Modeling Irregular Time Series with Continuous Recurrent Units

Abstract: Recurrent neural networks (RNNs) are a popular choice for modeling sequential data. Modern RNN architectures assume constant time-intervals between observations. However, in many datasets (e.g. medical records) observation times are irregular and can carry important information. To address this challenge, we propose continuous recurrent units (CRUs) -- a neural architecture that can naturally handle irregular intervals between observations. The CRU assumes a hidden state, which evolves according to a linear stochastic differential equation and is integrated into an encoder-decoder framework. The recursive computations of the CRU can be derived using the continuous-discrete Kalman filter and are in closed form. The resulting recurrent architecture has temporal continuity between hidden states and a gating mechanism that can optimally integrate noisy observations. We derive an efficient parameterization scheme for the CRU that leads to a fast implementation f-CRU. We empirically study the CRU on a number of challenging datasets and find that it can interpolate irregular time series better than methods based on neural ordinary differential equations.

Short bio: Mona Schirmer is an ELLIS PhD student at the University of Amsterdam and a machine learning consultant at the World Bank. Before that, she completed a master’s in statistics at Humboldt University and Technical University in Berlin as well as a French engineering diploma at ENSAE. In her Bachelor’s, she studied economics and political science at Humboldt University of Berlin and the University of Munich. Her research interests lie in probabilistic machine learning und machine learning for social good. She visited ELLIS Alicante in May 2022.

Presenter: Mona Schirmer

Date: 2022-05-25 11:30 (CEST)

Location: Salon de Actos Politecnica I, Carretera San Vicente del Raspeig s/n, San Vicente del Raspeig 03690, Alicante ES