Generative deep learning models for inter- and extrapolation in medical time series

by Wenke Karbole

Temporal information plays a pivotal role in human comprehension of situations and processes. In natural time series, such as videos, events unfold sequentially, providing viewers with insights into the timing of the depicted processes. In medicine, evaluating temporal dynamics is crucial for tasks like disease staging, predicting disease progression, or monitoring treatment efficacy. However, unlike most videos with consistent frame rates, medical imaging time series are commonly sampled at irregular intervals over long periods of time. Moreover, individual time points are not intrinsically registered due to varying fields of view, patient positioning, heterogeneity in imaging protocols, or sudden anatomical changes. These irregular temporal representations pose challenges for conventional deep learning tools.

Our objective is to utilize generative deep learning models to inter- and extrapolate missing data points in medical time series. We aim to obtain longitudinal medical imaging data with consistent temporal representation while maintaining the complexities of physiological processes, such as non-linear dynamics and the sudden emergence or disappearance of tissue. Moreover, the underlying generative methods can not only visualize intricate medical processes for patients and clinicians but also allow for the generation of counterfactual scenarios. Counterfactuals, as a mode for casual reasoning, compare real-world data to hypothetical “What if?”-scenarios, evaluating the outcome under different conditions, and thereby can provide insights into cause-to-effect relationships. This capability can facilitate the assessment of biomarkers for various disease types and stages by understanding the implications of medical interventions, as well as the impact of risk factors, such as demographic factors, genotypes, or lifestyle on disease progression.