Salva Rühling Cachay

selected publications [full list]

1. NeurIPS

   DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting

   Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu

   In Advances in Neural Information Processing Systems

   Abstract BibTex PDF Blog Code Poster Slides

   While diffusion models can successfully generate data and make predictions, they are predominantly designed for static images. We propose an approach for efficiently training diffusion models for probabilistic spatiotemporal forecasting, where generating stable and accurate rollout forecasts remains challenging, Our method, DYffusion, leverages the temporal dynamics in the data, directly coupling it with the diffusion steps in the model. We train a stochastic, time-conditioned interpolator and a forecaster network that mimic the forward and reverse processes of standard diffusion models, respectively. DYffusion naturally facilitates multi-step and long-range forecasting, allowing for highly flexible, continuous-time sampling trajectories and the ability to trade-off performance with accelerated sampling at inference time. In addition, the dynamics-informed diffusion process in DYffusion imposes a strong inductive bias and significantly improves computational efficiency compared to traditional Gaussian noise-based diffusion models. Our approach performs competitively on probabilistic forecasting of complex dynamics in sea surface temperatures, Navier-Stokes flows, and spring mesh systems.

   @inproceedings{cachay2023dyffusion,
     title = {{DYffusion:} A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting},
     author = {R{\"u}hling Cachay, Salva and Zhao, Bo and Joren, Hailey and Yu, Rose},
     booktitle = {Advances in Neural Information Processing Systems},
     url = {https://openreview.net/forum?id=WRGldGm5Hz},
     year = {2023},
   }

2. NeurIPS

   ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models

   Salva Rühling Cachay*, Venkatesh Ramesh*, Jason N. S. Cole, Howard Barker, and David Rolnick

   In Advances in Neural Information Processing Systems Datasets and Benchmarks Track

   Abstract BibTex PDF Code Poster Slides

   Numerical simulations of Earth’s weather and climate require substantial amounts of computation. This has led to a growing interest in replacing subroutines that explicitly compute physical processes with approximate machine learning (ML) methods that are fast at inference time. Within weather and climate models, atmospheric radiative transfer (RT) calculations are especially expensive. This has made them a popular target for neural network-based emulators. However, prior work is hard to compare due to the lack of a comprehensive dataset and standardized best practices for ML benchmarking. To fill this gap, we build a large dataset, ClimART, with more than \emph10 million samples from present, pre-industrial, and future climate conditions, based on the Canadian Earth System Model. ClimART poses several methodological challenges for the ML community, such as multiple out-of-distribution test sets, underlying domain physics, and a trade-off between accuracy and inference speed. We also present several novel baselines that indicate shortcomings of datasets and network architectures used in prior work.

   @inproceedings{cachay2021climart,
     title = {{ClimART}: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models},
     author = {R{\"u}hling Cachay*, Salva and Ramesh*, Venkatesh and Cole, Jason N. S. and Barker, Howard and Rolnick, David},
     booktitle = {Advances in Neural Information Processing Systems Datasets and Benchmarks Track},
     url = {https://openreview.net/forum?id=FZBtIpEAb5J},
     year = {2021},
   }

3. NeurIPS

   End-to-End Weak Supervision

   Salva Rühling Cachay, Benedikt Boecking, and Artur Dubrawski

   In Advances in Neural Information Processing Systems

   Abstract BibTex PDF Code Poster Slides

   Aggregating multiple sources of weak supervision (WS) can ease the data-labeling bottleneck prevalent in many machine learning applications, by replacing the tedious manual collection of ground truth labels. Current state of the art approaches that do not use any labeled training data, however, require two separate modeling steps: Learning a probabilistic latent variable model based on the WS sources – making assumptions that rarely hold in practice – followed by downstream model training. Importantly, the first step of modeling does not consider the performance of the downstream model. To address these caveats we propose an end-to-end approach for directly learning the downstream model by maximizing its agreement with probabilistic labels generated by reparameterizing previous probabilistic posteriors with a neural network. Our results show improved performance over prior work in terms of end model performance on downstream test sets, as well as in terms of improved robustness to dependencies among weak supervision sources.

   @inproceedings{cachay2021endtoend,
     title = {End-to-End Weak Supervision},
     author = {R{\"u}hling Cachay, Salva and Boecking, Benedikt and Dubrawski, Artur},
     booktitle = {Advances in Neural Information Processing Systems},
     year = {2021},
   }

4. arXiv

   The World as a Graph: Improving El Niño Forecasts with Graph Neural Networks

   Salva Rühling Cachay, Emma Erickson, Arthur Fender C. Bucker, Ernest Pokropek, Willa Potosnak, Suyash Bire, Salomey Osei, and Björn Lütjens

   arXiv:2310.14189

   Abstract BibTex PDF Code Slides Media

   Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns. In comparison, graph neural networks (GNNs) are capable of modeling large-scale spatial dependencies and are more interpretable due to the explicit modeling of information flow through edge connections. We propose the first application of graph neural networks to seasonal forecasting. We design a novel graph connectivity learning module that enables our GNN model to learn large-scale spatial interactions jointly with the actual ENSO forecasting task. Our model, \graphino, outperforms state-of-the-art deep learning-based models for forecasts up to six months ahead. Additionally, we show that our model is more interpretable as it learns sensible connectivity structures that correlate with the ENSO anomaly pattern.

   @article{cachay2021world,
     title = {The World as a Graph: Improving El Ni\~no Forecasts with Graph Neural Networks},
     author = {R{\"u}hling Cachay, Salva and Erickson, Emma and Fender C. Bucker, Arthur and Pokropek, Ernest and Potosnak, Willa and Bire, Suyash and Osei, Salomey and Lütjens, Björn},
     journal = {arXiv:2310.14189},
     year = {2021},
   }