On average every 2 years, the stratospheric polar vortex exhibits extreme perturbations known as sudden stratospheric warmings (SSWs).
The impact of these events is not limited to the stratosphere: but they can also influence the weather at the surface of the Earth for up to 3 months after their occurrence. This downward effect is observed in particular for SSW events with extended recovery timescales.
This long-lasting stratospheric impact on surface weather can be leveraged to significantly improve the performance of weather forecasts on timescales of weeks to months.
In this paper, we present a fully data-driven procedure to improve the performance of long-range forecasts of the stratosphere around SSW events with an extended recovery.
We first use unsupervised machine learning algorithms to capture the spatio-temporal dynamics of SSWs and to create a continuous scale index measuring both the frequency and the strength of persistent stratospheric perturbations.
We then uncover three-dimensional spatial patterns maximizing the correlation with positive index values, allowing us to assess when and where statistically significant early signals of SSW occurrence can be found.
Finally, we propose two machine learning (ML) forecasting models as competitors for the state-of-the-art sub-seasonal European Centre for Medium-Range Weather Forecasts (ECMWF) numerical prediction model S2S (sub-seasonal to seasonal): while the numerical model performs better for lead times of up to

In both hemispheres and during winter, the atmosphere above the polar regions is characterized by eastward winds centered around the poles with a mid-winter peak in wind intensity, the so-called “polar vortex”.
On average once every 2 years, in the Northern Hemisphere, upwardly propagating Rossby waves can disturb the polar vortex and induce a sudden warming of the polar stratosphere.
Known as sudden stratospheric warmings

The dynamics behind SSWs are not yet fully understood

SSWs with a stronger and more persistent impact on the stratosphere are generally associated with slowly descending zonal wind anomalies

One difficulty in uncovering relevant precursors to SSW events is the need to analyze large quantities of three-dimensional spatio-temporal data of different physical variables with complex interactions. For this reason, existing studies have relied on domain expertise to focus on regions of the atmosphere and mechanisms assumed relevant for SSWs. From a data analysis point of view, conducting a systematic search of the atmospheric dynamics to localize and uncover statistically significant signals with predictive potential for SSWs with long-lasting impacts would help in getting a deeper understanding of their dynamics. Machine learning (ML) and, more generally, data science techniques offer fully data-driven alternatives for a systematic search of new insights about relevant atmospheric dynamics that can be applied to large quantities of data. ML approaches can also represent competitive alternatives to numerical forecasts that can be leveraged to improve weather prediction performance.

Indeed, data-driven techniques have already been successfully applied to improve understanding of SSW dynamics. For instance, an application of empirical orthogonal functions (EOFs) to potential vorticity data during SSW events revealed unknown drivers as well as early signals of SSW onset

Following the suggestion of

The paper is structured as follows: Sect.

The analysis presented in this paper relies on the ERA-Interim reanalysis data produced by the European Centre for Medium-Range Weather Forecasts (ECMWF)

To characterize stratospheric perturbations and identify their precursors, we need to analyze a large quantity of reanalysis data: for each physical quantity, ERA-Interim provides

For temperature, we proceed as in

Alternatively, to directly study the perturbations of the vortex and not only the vertical structure of temperature anomalies, we analyze potential vorticity (PV), a quantity used in atmospheric dynamics and meteorology to characterize rotational fluids with a vertical stratification, where it is conserved for adiabatic frictionless motion.
Because the vortex can be displaced to fairly low latitudes, we analyze a slightly larger region for PV as compared to temperature, i.e., all grid points poleward of

Original (left) and reconstructed potential vorticity field using an increasing number (

Finally, we aim to illustrate how machine learning techniques can be used to improve the long-range performance of current numerical forecasts.
To do so, we leverage one data set of forecasts from the sub-seasonal to seasonal (S2S) prediction database

Most classical characterizations of sudden stratospheric warmings are binary indices that usually require a predefined threshold, e.g., most commonly, the inversion of the zonal-mean zonal wind at

From a machine learning perspective, the limited number of events in the database and the uncertainty of the event start dates make the application of most algorithms using binary indices difficult and potentially inconsistent. For these reasons, we first define a continuous scale index to quantify the strength and the occurrence of the vortex perturbations. For this task, we use simple unsupervised machine learning algorithms, i.e., algorithms that aim at discovering dynamical patterns in the data without an auxiliary source of information, including potentially unknown patterns.

We start by analyzing stratospheric perturbations as captured by temperature anomalies using an analysis similar to those of

Figure

We now aim at characterizing not only the vertical structure of the temperature perturbations at individual time steps but also their variation over time.
Applying PCA to a time-lagged embedding

First four principal directions corresponding to the largest variance obtained from PCA on polar-cap-averaged temperature anomalies at levels 150, 125, 100, 70, 50, 30, 20, 10, 7, 5, 2, and 1 hPa, indicated by the different colors in the legend, with a temporal embedding of

Figure

Figure

We have thus managed to produce a continuous scale index using unsupervised machine learning techniques, allowing us to characterize the dynamics of strong temperature perturbations of the polar vortex that are followed by a slow recovery. The index

We now aim at uncovering relevant variables, atmospheric regions, and associated spatio-temporal patterns that are indicative of likely future SSW events with a slow recovery.
We leverage supervised machine learning algorithms to predict the index

From a practical perspective, we aim at finding atmospheric patterns showing a statistically significant level of predictability, summarized here by correlation, for our index

As a supervised algorithm, sPCA is likely to suffer from overfitting, i.e., matching noisy variations in the data instead of relevant physical dynamics.
Indeed, as we explore multiple levels over a large region of the Northern Hemisphere, the dimension

We start by analyzing potential vorticity: this physical quantity is available from the ERA-Interim reanalysis at

Focusing first on anomalies, the leftmost panel in Fig.

For PV waves, we analyze the atmosphere only above

We also searched for early signals in other physical quantities of the climate system.
The divergence of the Eliassen–Palm flux

Maximum correlation as a function of the lead time for PV anomalies

Finally, we also consider tropical stratospheric winds, i.e., the zonal (

To summarize, sPCA has allowed us to find statistically significant early signals for the occurrence of SSWs with slow recovery up to 6 weeks prior to the onset of the event.
The signal is strongest for PV anomalies with a predictive pattern localized in the upper part of the stratosphere, i.e., from

Now that we know which physical quantities show statistically significant correlations, as well as where in the atmosphere and how much in advance these variables are relevant, we can use this knowledge to design machine learning techniques to forecast the index

Forecast performance being a relative notion, we need reference forecasting models against which new algorithms will be compared. The first most natural choice is the climatological forecast, i.e., a forecast whose prediction matches the corresponding day and month of the average seasonal cycle. This forecast is usually seen as the least informative as this strategy does not take into account the current state of the system; it is thus seen as the baseline that has to be beaten for a forecast to exhibit any kind of predictability.

A more informed and more accurate alternative, corresponding to the current state of the art, comprises the sub-seasonal forecasts produced by numerical models.
In particular, we analyze the performance of the S2S forecasts produced by the ECMWF presented in Sect.

Data-driven alternatives such as machine learning algorithms provide point forecasts by default, i.e., not an ensemble but only one value for each lead time.
Comparing probabilistic forecasts, such as the ECMWF ensemble members, with point forecasts is therefore challenging.
Modifying machine learning algorithms to output probabilistic forecasts is possible but requires either advanced techniques such as Bayesian computation or model ensemble methods. The link between numerical ensembles and probabilistic forecasts is an active field of research

To forecast the index

To enable a comparison with the ECMWF hindcasts, we first repeat the analysis described in Sect.

Therefore, we propose to post-process S2S forecasts based on ML forecasts to improve the overall model performance:
for a given initialization date, we start by producing a forecast

Mean absolute error (MAE) as a function of the lead time for the following forecasting models: climatology (solid black), ECMWF model (solid red), linear regression with sPCA output (double-dashed blue), multilayer perceptron (dashed orange), and post-processed ECMWF model (dotted red).

We showed here that ML methods can be used to improve long-range forecast performance in the stratosphere. Further fine-tuning of different ML models, by trying more combinations of variables or hyperparameters, is likely to further improve the performance of the ML models and the post-processed S2S model but is left for future work.

We presented in this work a framework to analyze and predict atmospheric dynamics using machine learning.
The methodology presented is a three-step procedure: we first use unsupervised machine learning techniques to produce a univariate index quantifying the occurrence and the strength of sudden stratospheric warmings with slow recovery.
The index is then used as input for supervised algorithms in order to assess “where” and “when” in the system we can find relevant predictors.
Finally, the answer to these questions is used to produce ML forecasts up to

The methodology presented in this paper has been developed to ensure both tractability and interpretability of the results.
As the drivers behind SSWs with slow recovery are large-scale circulation patterns, we successfully reduced the data dimensionality using a functional representation of the data that allows us to apply machine learning algorithms using reasonable computational times and resources, making it possible to test statistical significance using resampling techniques.
We focused here mostly on algorithms, such as linear regression, whose interpretability is straightforward and which not only improve the forecast performance but also potentially allow us to deepen our understanding of SSW dynamics. However, non-linear and more complex data-driven methods such as Laplacian eigenmaps

The major drawback of the post-processing presented in Sect.

First four principal components for winters 1980/99, 1999/2000, 2006/07, and 2007/08 obtained using PCA on polar-cap-averaged temperature anomalies at pressure levels 150, 125, 100, 70, 50, 30, 20, 10, 7, 5, 2, and 1 hPa with a temporal embedding of

Pattern of PV anomalies maximizing correlation with the SSW index at a lead time of

Pattern of PV anomalies maximizing correlation with the SSW index at a lead time of

Pattern of PV anomalies maximizing correlation with the SSW index at a lead time of

Pattern of PV anomalies maximizing correlation with the SSW index at a lead time of

Pattern of PV anomalies maximizing correlation with the SSW index at a lead time of

Pattern of PV waves maximizing correlation with the SSW index at a lead time of

Pattern of PV waves maximizing correlation with the SSW index at a lead time of

Pattern of PV waves maximizing correlation with the SSW index at a lead time of

Pattern of PV waves maximizing correlation with the SSW index at a lead time of

Pattern of PV waves maximizing correlation with the SSW index at a lead time of

Patterns of heat flux maximizing correlation with the SSW index at a lead time of

Patterns of heat flux maximizing correlation with the SSW index at a lead time of

Patterns of heat flux maximizing correlation with the SSW index at a lead time of

Patterns of heat flux maximizing correlation with the SSW index at a lead time of

ERA-Interim reanalysis was obtained from the ECMWF server (

RdF designed, implemented, and conducted all the data analysis. RdF also made the figures and wrote the manuscript draft. ZW, DD, ES, and GO contributed to designing the study and interpreting the results, as well as editing the manuscript.

At least one of the (co-)authors is a member of the editorial board of

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was funded through the project EXPECT (C18-08) of the Swiss Data Science Center. Partial support from the Swiss National Science Foundation through project PP00P2_170523 to Zheng Wu and projects PP00P2_170523 and PP00P2_198896 to Daniela I. V. Domeisen is gratefully acknowledged.

This research has been supported by the Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (grant nos. PP00P2_170523 and PP00P2_198896) and the Swiss Data Science Center (project no. C18-08).

This paper was edited by Pedram Hassanzadeh and reviewed by two anonymous referees.