Evan Sidrow

Evan Sidrow

Distinguished Postdoctoral Fellow in Statistics at Simon Fraser University

Talks and Presentations

Stochastic optimization for efficient inference in ecological hidden Markov models

June 11, 2024

Best student oral presentation (of 42 participants), Western North American Region of the International Biometric Society Annual Meeting, Fort Collins, CO, USA

Hidden Markov models (HMMs) are popular models to identify a finite number of latent states from sequential data. However, fitting them to large data sets can be computationally demanding because most likelihood maximization techniques require iterating through the entire underlying data set for every parameter update. We propose a novel optimization algorithm that updates the parameters of an HMM without iterating through the entire data set. Namely, we combine a partial E step with variance-reduced stochastic optimization within the M step. We prove the algorithm converges under certain regularity conditions. We test our algorithm empirically using a simulation study as well as a case study of kinematic data collected using suction-cup attached biologgers from eight northern resident killer whales (Orcinus orca) off the western coast of Canada. In both, our algorithm converges in fewer epochs and to regions of higher likelihood compared to standard numerical optimization techniques. Our algorithm allows practitioners to fit complicated HMMs to large time-series data sets more efficiently than existing baselines.

Variance-reduced stochastic optimization for efficient inference of hidden Markov models

March 09, 2024

Invited presentation, UBC-SFU Joint Statistics Seminar, Vancouver, BC, Canada

Hidden Markov models (HMMs) are popular models to identify a finite number of latent states from sequential data. However, fitting them to large data sets can be computationally demanding because most likelihood maximization techniques require iterating through the entire underlying data set for every parameter update. We propose a novel optimization algorithm that updates the parameters of an HMM without iterating through the entire data set. Namely, we combine a partial E step with variance-reduced stochastic optimization within the M step. We prove the algorithm converges under certain regularity conditions. We test our algorithm empirically using a simulation study as well as a case study of kinematic data collected using suction-cup attached biologgers from eight northern resident killer whales (Orcinus orca) off the western coast of Canada. In both, our algorithm converges in fewer epochs and to regions of higher likelihood compared to standard numerical optimization techniques. Our algorithm allows practitioners to fit complicated HMMs to large time-series data sets more efficiently than existing baselines.

Modelling multi-scale, state-switching functional data with hidden Markov models

May 31, 2023

Invited presentation, Statistical Society of Canada Annual Meeting, Ottawa, ON, Canada

Data sets composed of sequences of curves sampled at high frequencies in time are increasingly common in practice, but they can exhibit complicated dependence structures that cannot be modelled using common methods in functional data analysis. We detail a hierarchical approach that treats the curves as observations from a hidden Markov model. The distribution of each curve is then defined by another fine-scale model that may involve autoregression and require data transformations using moving-window summary statistics or Fourier analysis. This approach is broadly applicable to sequences of curves exhibiting intricate dependence structures. As a case study, we use this framework to model the fine-scale kinematic movements of a northern resident killer whale (Orcinus orca) off the western coast of Canada. Through simulations, we show that our model produces more interpretable state estimation and more accurate parameter estimates compared to existing methods.

Modelling functional data with hierarchical hidden Markov models: Applications to animal movement

May 30, 2022

Invited presentation, Statistical Society of Canada Annual Meeting, Virtual

Modern biologging sensors can record sequences of curves at very high frequencies, allowing researchers to observe biological processes such as animal movement at extremely fine scales. High-frequency data sets can exhibit state-switching, multi-scale dependence structures that are difficult to model with standard methods in functional data analysis. Inspired by data collected from a northern resident killer whale (Orcinus orca), we describe a hierarchical framework that treats curves as observations from a hidden Markov model. Each curve’s distribution is defined by a fine-scale model whose parameters depend upon a coarse-scale latent process. Through simulations, we show that our model produces more interpretable state estimates and more accurate parameter estimates compared to existing methods. We also consider several computational challenges when modelling state-switching functional data with hidden Markov models.