RL Learns State-Dependent Climate Model Parameters
On July 24, 2026, researchers revised an arXiv study evaluating reinforcement learning for replacing fixed tunable parameters with state-dependent functions in idealized weather and climate models. The authors report that TQC, DDPG, and TD3 delivered the strongest stable performance across nine RL algorithms, while a six-agent federated DDPG configuration achieved the lowest area-weighted RMSE across tested tropical and mid-latitude regions.
Researchers have released a revised arXiv study evaluating reinforcement learning as a way to replace fixed, offline-tuned parameter values in weather and climate models with functions that update from the evolving model state. The paper, submitted in January and last revised on July 24, evaluates the approach in three idealized environments: simple climate bias correction (SCBC), radiative-convective equilibrium (RCE), and a zonal-mean energy balance model (EBM).
Weather and climate simulations use parametrizations to represent sub-grid processes that cannot be explicitly resolved at model resolution. According to the paper's abstract, conventional fixed coefficients can be weakly constrained and can contribute to persistent model biases. The authors use RL policies to update components of those parametrization schemes during simulation rather than rely solely on static parameter tuning.
Results across testbeds
The authors benchmarked nine RL algorithms and report that Truncated Quantile Critics (TQC), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3) had the highest skill and stable convergence. They assessed performance against a static baseline using area-weighted RMSE plus temperature and pressure-level diagnostics.
For the EBM experiments, the paper reports that single-agent RL outperformed static tuning, with the largest improvements in tropical and mid-latitude bands. In federated multi-agent experiments, the authors report that agents could specialize across regions and converge faster. A six-agent DDPG setup with frequent aggregation recorded the lowest area-weighted RMSE across the tested tropical and mid-latitude regions.
The reported learned controls included adjustments to EBM radiative parameters intended to reduce meridional bias, RCE lapse rates to match vertical temperature errors, and heating increments to limit drift. The reported experiments are limited to idealized testbeds and do not establish validation in full weather and climate models.
Code and reproducibility
The authors have made a public climate-rl repository available. Its README describes Gymnasium-based climate environments, CleanRL-styled RL implementations, Ray-based parameter-tuning scripts, datasets, notebooks, and experiment results. A Zenodo release published on April 16 packages the project software and identifies it as a release for the associated JAMES submission.
The work offers a concrete formulation of online parameter control in a scientific simulator: model state forms the RL observation, parameter updates form the action, and physical-error metrics contribute to the reward. Comparable efforts moving from idealized simulators to high-dimensional climate systems typically face more difficult questions around numerical stability, reward design, out-of-distribution states, computational cost, and physical constraints. The paper presents the reported experiments as a starting point for evaluation in full weather and climate models.
Key Points
- 1The study replaces fixed parametrization coefficients with RL-controlled state-dependent functions, targeting biases linked to offline parameter tuning in idealized climate simulations.
- 2TQC, DDPG, and TD3 achieved the paper's strongest stable results, while federated DDPG produced the lowest reported regional EBM RMSE.
- 3Public code and a Zenodo software release support reproducibility, though validation in full weather and climate models remains outside the reported testbeds.
Scoring Rationale
This is a technically relevant research contribution for practitioners working on scientific machine learning, control, and climate simulation. Its public code improves practical value, but the evidence is limited to idealized testbeds rather than full weather or climate systems.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
