Simulating the West African monsoon (WAM) system using numerical weather and climate models suffers from large uncertainties, which are difficult to assess due to nonlinear interactions between different components of the WAM. Here we present a fundamentally new approach to the problem by approximating the behavior of a numerical model – here the Icosahedral Nonhydrostatic (ICON) model – through a statistical surrogate model based on universal kriging, a general form of Gaussian process regression, which allows for a comprehensive global sensitivity analysis. The main steps of our analysis are as follows: (i) identify the most important uncertain model parameters and their probability density functions, for which we employ a new strategy dealing with non-uniformity in the kriging process. (ii) Define quantities of interest (QoIs) that represent general meteorological fields, such as temperature, pressure, cloud cover and precipitation, as well as the prominent WAM features, namely the tropical easterly jet, African easterly jet, Saharan heat low (SHL) and intertropical discontinuity. (iii) Apply a sampling strategy with regard to the kriging method to identify model parameter combinations which are used for numerical modeling experiments. (iv) Conduct ICON model runs for identified model parameter combinations over a nested limited-area domain from 28° W to 34° E and from 10° S to 34° N. The simulations are run for August in 4 different years (2016 to 2019) to capture the peak northward penetration of rainfall into West Africa, and QoIs are computed based on the mean response over the whole month in all years. (v) Quantify sensitivity of QoIs to uncertain model parameters in an integrated and a local analysis.

The results show that simple isolated relationships between single model parameters and WAM QoIs rarely exist. Changing individual parameters affects multiple QoIs simultaneously, reflecting the physical links between them and the complexity of the WAM system. The entrainment rate in the convection scheme and the terminal fall velocity of ice particles show the greatest effects on the QoIs. Larger values of these two parameters reduce cloud cover and precipitation and intensify the SHL. The entrainment rate primarily affects 2 m temperature and 2 m dew point temperature and causes latitudinal shifts, whereas the terminal fall velocity of ice mostly affects cloud cover. Furthermore, the parameter that controls the evaporative soil surface has a major effect on 2 m temperature, 2 m dew point temperature and cloud cover. The results highlight the usefulness of surrogate models for the analysis of model uncertainty and open up new opportunities to better constrain model parameters through a comparison of the model output with selected observations.

The West African monsoon (WAM) is a prominent seasonal large-scale circulation feature associated with a deep northward penetration of rainfall into West Africa during the boreal summer months, usually peaking in August

Schematic illustration of the WAM system in a height–latitude display

The monsoonal rain belt is enclosed by two distinctive dynamical features, the African easterly jet (AEJ) to the north and the tropical easterly jet (TEJ) to the south. The AEJ, a pronounced easterly jet at around 600–700 hPa maintained by the low-tropospheric meridional temperature gradient, regularly features wave disturbances. These so-called African easterly waves (AEWs) with wavelengths between 2000 and 5000 km and periods of 2–7 d

Despite its outstanding importance for the region, simulations of the WAM spanning timescales from weather to climate are fraught with substantial uncertainties. With respect to weather forecasts,

How can we improve model simulations over West Africa? The most obvious way is trying to improve the numerical model itself.

The abovementioned studies were conducted to assess isolated relationships between certain model parameters and simulated WAM quantities. A general problem of this approach is that it is very challenging to study the combined effects of several sources of uncertainty at once. Nonlinear interactions and buffering effects will make it nearly impossible to deduce such effects from single-parameter perturbation experiments. Ideally one should conduct experiments across a wide range of parameter combinations, but this will very quickly become too expensive, as a certain simulation period is required to separate differences from day-to-day weather noise.

An attractive alternative to such a costly approach is surrogate models – also known as emulators or meta-models – which allow for a comprehensive but resource-friendly statistical investigation of the sensitivity of QoIs to uncertain model parameters

There exists a range of methodological approaches for surrogate models. Among these, Gaussian process regression, also known as kriging, is the most popular one in meteorological literature and has for instance been applied by

Universal kriging

This study aims at quantifying the uncertainty contributions and effects of selected model parameters on a variety of QoIs and output fields that characterize the WAM system. There has been no such study that also includes potential interactions of multiple model parameters. The Icosahedral Nonhydrostatic (ICON) model, the operational weather prediction model of the German Weather Service (DWD), is used to simulate the rainy seasons in 4 years in limited-area mode. We investigate the influence of six model parameters that are expected to have substantial impacts on the WAM characteristics. For each of them, probability density functions (PDFs) are assigned based on the literature and expert knowledge. Maximin Latin hypercube sampling

The paper is organized as follows: in Sect.

This section details the applied methods and employed datasets. In Sect.

A crucial first step on the way to develop surrogate models is to identify relevant uncertain model parameters and to define meaningful PDFs representing our best knowledge of the associated epistemic uncertainty. Based on experience from sensitivity studies, literature review and expert judgment, we take into consideration six parameters which cover a fairly broad spectrum of the model's physics. These are the grid-scale microphysics (

Selected uncertain model parameters and a short description, the assumed PDF and physical unit.

The entrainment rate (

In various meteorological studies and applications, uniform parameter distributions over an estimated range of plausible values are assumed

In order to represent the relationship between the uncertain model parameters listed in Table

In order to build a surrogate model, training points for the model parameters have to be defined based on the PDFs specified in Sect.

For the sake of simplicity and interpretability of the results, the model parameters are kept temporarily and spatially constant during individual model runs. Therefore, one training point corresponds to a fixed set of the six model parameters which is used for one ICON model run.

The number of necessary training points strongly depends on the nonlinearity of the investigated problem. Therefore, validation (see Sect.

In this study, we aim at describing a relationship between six model parameters and selected QoIs. We construct a separate surrogate model for each QoI, which can later be used to employ sensitivity studies with significantly reduced computational cost.

Among available surrogate modeling methods, Gaussian process regression offers wide flexibility and potential for extensions and is therefore used in this study. We apply the universal kriging method, a general form of Gaussian process regression, where explicit basis functions can be incorporated. We base our choice on

Our aim is to build a surrogate model

For the purpose of universal kriging,

Here, the anisotropic form of the radial basis function

Here,

The prediction mean and prediction variance, as shown by

Here, additive i.i.d. Gaussian noise with variance

The selection of basis functions for universal kriging is a crucial step because prediction accuracy of the surrogate model may strongly depend on it.

In our work, we consider transformed input spaces.

In our study, we assume linear basis functions in the physical input space, which are transformed into the i.i.d. uniform space by the Rosenblatt transformation. Assuming linear basis functions in the physical input space is considered reasonable, since most parameter ranges are relatively small compared to their absolute values, and linear relationships may be sufficient in order to represent a global trend. Furthermore, quadratic basis functions would in a general case imply the inclusion of

Obtained surrogate models need to be validated to assess their accuracy, which depends on various factors, e.g., the number of training points, the choice of basis functions and nonlinearities in the physical model. As a validation criterion for a surrogate model

Because of high computation cost, using a separate validation set that is not used for model training is not effective. Therefore, cross-validation techniques, such as leave-one-out validation or leave-

Model accuracy is considered to be high if NMSE values are close to 0 and low if NMSE values are close to 1. By definition, values are non-negative and should not exceed 1, as the covariance between the surrogate model and data would in that case be higher than the variance of the data itself. Interpretation of the NMSE could become problematic if QoI values do not substantially change and the variance

In order to quantify the relative magnitude of the dependency of the QoIs on the uncertain model parameters, global sensitivity analysis is used. We apply FAST

The Icosahedral Nonhydrostatic (ICON) model

ICON model setup, outer domain with 26 km grid spacing (green), inner domain with 13 km grid spacing (brown) and domain for which output data are stored (blue).

Simulation data are stored with a horizontal resolution of 0.1° within the region from 0 to 25° N and 15° W to 15° E (see Fig.

cloud cover at high (

column-integrated water vapor (kg m

precipitation (mm per month), 3-hourly;

2 m temperature (K), 3-hourly;

2 m dew point temperature (K), 3-hourly;

mean sea level pressure (Pa), 3-hourly;

The output quantities from the ICON simulations are validated. In Sect.

In this section, we describe the QoIs we selected to characterize the WAM and explain how these quantities are determined from the ICON model output. The results for all QoIs are averaged over the study time (1 to 31 August of the years 2016, 2017, 2018 and 2019) using all data with the temporal resolution given in Sect.

The accumulated precipitation fields are computed and averaged over the study region to obtain one scalar value representing the overall precipitation.

Illustrations of selected QoI computations:

The latitude of the rain belt is determined to investigate the potential influence of model parameters on a north–south shift in the average precipitation. For this purpose, the latitudinal center of the accumulated precipitation is computed in the study region between 12° W and 2° E (Fig.

The averaged column-integrated water vapor over the study region is computed.

The averaged cloud cover at high, middle and low levels over the study region is computed.

TEJ and AEJ are the main zonal wind features of the WAM system. They are characterized by the same measures applied at different pressure levels (200 hPa for TEJ, 600 hPa for AEJ). The averaged latitudes of the jet streams are considered in order to investigate the potential influence of model parameters on the north–south shift of the jet streams. Computing only the latitude of the maximum zonal wind speeds turned out to be a non-robust measure, since it is very sensitive and likely to fluctuate for small parameter changes due to the chaotic nature of the atmosphere. Therefore, we introduce a more robust measure that takes the neighboring latitudes into account to compute an averaged latitude of maximum zonal wind speeds. First, we compute the averaged zonal wind speeds for each latitude on the grid. This step simplifies the data to a manageable form with one average wind speed value for each latitude on the grid. The distribution of these wind speeds is still relatively flat, making it difficult to robustly determine the latitude of maximum wind speed. We thus employ a strategy where we exponentiate the average wind values (here, an exponent of 3 yielded useful results) to assign higher weight to the highest values and to reduce the influence of values far from the jet center which are still relatively high (Figs.

The jet speed is determined by averaging the zonal wind along the obtained jet latitude. It should be noted that the maximum jet speed at instantaneous points in time is much higher, since we consider the average of wind speeds over time for the sake of a greater robustness.

The ITD indicates the location where dry northeasterly winds from the Sahara and moist southwesterly winds from the tropical Atlantic Ocean meet. The ITD is characterized by a marked jump in moisture content near the surface. We use the 14 °C 2 m dew point temperature as a measure for the ITD latitude

The characteristic heat low in the region of the Sahara, one of the main drivers of the WAM, is characterized by its strength. For this purpose, the average pressure field is determined within the region from 15 to 25° N and 15° W to 5° E, where the heat low is expected on the basis of climatological results

The latitude of the SHL is characterized by the southern boundary of the SHL region based on a MSLP threshold of 1009 hPa between 9 and 1° W as shown in Fig.

The average 2 m temperature over the whole study region is computed.

The average 2 m dew point temperature over the whole study region is computed.

In Sect.

This procedure is applied to all combinations of model parameters and available output data. To quantify the significance of such investigations, a Kruskal–Wallis test

As a reference, the mean field plots can be obtained for each meteorological variable by averaging output data obtained from all available training points. These reference plots together with the variability plots can then serve as a basis for interpretation of regional influences of model parameters.

Validation is an essential step before discussing the results of the conducted studies. It offers insight into how informative and significant the analysis of this work is. We conducted validation for the outputs of the ICON model simulations (see Sect.

The validation results for the averaged ICON model outputs with respect to ERA5 data – and additionally GPM IMERG data for precipitation – are shown in Table

Validation results for the ICON model outputs with ERA5 data.

For validation of the surrogate models, leave-

Validation results for the surrogate models of all QoIs with cross-validation.

Main and total effect sensitivity indices of the six selected uncertain model parameters for all QoIs, resulting from the global sensitivity analysis FAST.

The results of the global sensitivity analysis are shown in Fig.

Sensitivities of cloud cover (leftmost columns in Fig.

Column water vapor is mostly influenced by the deep-cloud parameters, similar to high clouds, but the boundary layer parameters also play a minor role. This suggests that this variable is in fact more sensitive to interactions with clouds at middle and high levels than to changes in evaporation and vertical mixing at low levels. Somewhat unexpectedly, 2 m temperature and 2 m dew point temperature are mostly influenced by the deep-cloud parameters, too, with the entrainment rate playing the biggest role. This suggests that these parameters must cause substantial indirect effects outside of the clouds. More obviously,

The eight rightmost double columns in Fig.

Results of the parameter studies for the QoIs based on the surrogate models described in Sect.

As surrogate model predictions depend on all six parameters, the full relationship cannot be visualized graphically. Instead, it is possible to illustrate one-at-a-time changes. Since parameter interactions were shown to be relatively low in Sect.

Dependencies of all QoIs (ordinate) with respect to the six selected uncertain model parameter (abscissa). The shaded area around the curves illustrates prediction variance (see Eq.

Averages of all output variables over all available ICON simulations and the entire evaluation period (Augusts 2016–2019) are shown in Fig.

Average of selected output fields over the evaluation period (August 2016–2019), averaged over all available ICON simulations. Figure numbers are chosen in accordance with the variables and labels in Figs.

Spatial variability of selected output fields for the uncertain model parameters

Spatial variability of selected output fields for the uncertain model parameters

Spatial variability of selected output fields for the uncertain model parameters

Results from the statistical Kruskal–Wallis test for the variability fields are denoted in Figs.

The effect of the investigated deep-cloud parameters, entrainment rate (

As shown in Fig.

Figure

The column-integrated water vapor (Fig.

As already pointed out in the discussion of Fig.

Comparing the effect of enhanced entrainment with that of a faster terminal fall velocity of ice, we see many commonalities despite the fundamentally different microphysical processes at play. With respect to the overall effects displayed in Fig.

The investigated below-cloud parameters, namely the relative humidity threshold for onset of evaporation (

Signals that stand out in Fig.

Effects of the scaling factor for minimum vertical diffusion for heat and moisture (

For a higher value of

In contrast,

Both

The aim of this study was to quantify uncertainty contributions of selected uncertain ICON model parameters for a set of QoIs that characterizes the WAM system. Findings should help to improve parameter specifications to make long-term simulations and forecasts more accurate. Due to computational cost, surrogate models are used as a resource-friendly alternative to describe the relationship between model parameters and QoIs. The study was based on a novel approach by

The dependency of QoIs on multiple model parameters and the influence of single parameters on multiple QoIs reflect the complex coupled relationships in the WAM system. Although the magnitude of the impact of individual model parameters varies quite strongly, most parameters show distinct effects on many facets of the system, which are illustrated schematically for the four most important parameters in Fig.

Illustration of the qualitative effects on the WAM system due to an increase in the investigated model parameters that have the strongest impacts:

The entrainment rate (

The parameters

The scaling factor for vertical diffusion of heat and moisture (

Concerning the selected uncertain model parameters (Sect.

This study has shown that it is mainly the entrainment rate, the fall speed of ice and surface evaporation that should be specified more accurately. This can be done by including further investigations, measurements and expert knowledge, including a more complex representation in parameterizations. Moreover, these parameters could be optimized with respect to the WAM simulation through parameter identification studies by including reanalysis and satellite data as observational references. The surrogate models that were obtained in this study can serve as the basis to conduct such identification studies. However, the outcome would be limited to the West African region. Thus, it might be possible to specify parameters that should only be valid in regions for which they have been optimized, as is already the case for

The computational framework used in this study primarily relies on publicly available software packages, along with some custom extensions. Gaussian process regression analyses were performed using the

The data used for model validation in this study include the ERA5 reanalysis data

PK conceived the overall concept of the study, including all necessary steps to quantify uncertainties in the selected model parameters. MF designed the study, including the experimental design, surrogate models, computation of QoIs, sensitivity analysis and local parameter studies, with input from all co-authors. GP set up the ICON model including ERA5 data. RVDL, GP and PK contributed to the meteorological aspects in the model setup. CP contributed to the methodical aspects of the study. PK, AL and JM contributed to the interpretation of the results and strategies for post-analysis. MF prepared the paper with input from all co-authors.

At least one of the (co-)authors is a member of the editorial board of

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

Peter Knippertz acknowledges the C2 Prediction of wet and dry periods of the West African Monsoon project of the Transregional Collaborative Research Center SFB/TRR 165 Waves to Weather funded by the German Science Foundation (DFG).

This research has been supported by the German Research Foundation (DFG) (grant no. SFB/TRR 165, Waves to Weather).

This paper was edited by Stephan Pfahl and reviewed by two anonymous referees.