Alert button
Picture for Jessica Hwang

Jessica Hwang

Alert button

Gradient Estimation with Discrete Stein Operators

Feb 19, 2022
Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis K. Titsias, Lester Mackey

Figure 1 for Gradient Estimation with Discrete Stein Operators
Figure 2 for Gradient Estimation with Discrete Stein Operators
Figure 3 for Gradient Estimation with Discrete Stein Operators
Figure 4 for Gradient Estimation with Discrete Stein Operators

Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize the variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations.

Viaarxiv icon

Improving Subseasonal Forecasting in the Western U.S. with Machine Learning

Sep 19, 2018
Jessica Hwang, Paulo Orenstein, Karl Pfeiffer, Judah Cohen, Lester Mackey

Figure 1 for Improving Subseasonal Forecasting in the Western U.S. with Machine Learning
Figure 2 for Improving Subseasonal Forecasting in the Western U.S. with Machine Learning
Figure 3 for Improving Subseasonal Forecasting in the Western U.S. with Machine Learning
Figure 4 for Improving Subseasonal Forecasting in the Western U.S. with Machine Learning

Water managers in the western United States (U.S.) rely on longterm forecasts of temperature and precipitation to prepare for droughts and other wet weather extremes. To improve the accuracy of these longterm forecasts, the Bureau of Reclamation and the National Oceanic and Atmospheric Administration (NOAA) launched the Subseasonal Climate Forecast Rodeo, a year-long real-time forecasting challenge, in which participants aimed to skillfully predict temperature and precipitation in the western U.S. two to four weeks and four to six weeks in advance. Here we present and evaluate our machine learning approach to the Rodeo and release our SubseasonalRodeo dataset, collected to train and evaluate our forecasting system. Our system is an ensemble of two regression models. The first integrates the diverse collection of meteorological measurements and dynamic model forecasts in the SubseasonalRodeo dataset and prunes irrelevant predictors using a customized multitask model selection procedure. The second uses only historical measurements of the target variable (temperature or precipitation) and introduces multitask nearest neighbor features into a weighted local linear regression. Each model alone is significantly more accurate than the operational U.S. Climate Forecasting System (CFSv2), and our ensemble skill exceeds that of the top Rodeo competitor for each target variable and forecast horizon. We hope that both our dataset and our methods will serve as valuable benchmarking tools for the subseasonal forecasting problem.

Viaarxiv icon