- A Python-driven MPAS-A dynamic-core-only is tested
- The tangent linear and adjoint models of the MPAS-A are developed and their correctness is verified
- The relative sensitivities obtained by using the MPAS adjoint model shows interesting results for a baroclinic instability test case
Numerically solving atmospheric dynamical and physical equations has been the primary approach for making weather forecasts. The increasing computational capabilities allow for resolving atmospheric motions at finer and finer scales. Nonetheless, simulations permitting non-hydrostatic scales everywhere on the globe are still challenging and highly expensive (Satoh et al., 2008). An alternative solution is offered by the Model for Prediction Across Scales – Atmosphere (MPAS-A), which employs finite-volume irregular centroidal Voronoi meshes on a C-grid with an option of smoothly varying resolutions (Ringler et al., 2008; Skamarock et al., 2012; Park et al., 2013). Under the framework of MPAS-A, regions of interest may be covered with grids of the highest possible resolution and the remaining part of the globe with coarser grids and smooth transitions from high to coarse resolutions. Hagos et al. (2013) examined and compared the error characteristics of such smoothly varying, variable-resolution meshes in MPAS-A to those from a two-way nesting approach in the Weather Research and Forecasting (WRF) model. It was found that errors appeared near the edges of high-resolution nesting regions in WRF simulations but were avoided by the variable-resolution meshes in MPAS-A. Rauscher and Ringler (2014) showed that the regionally refined mesh of MPAS-A helps to resolve mid-latitude eddy activities that are often induced by orography, land–sea contrasts and sea surface temperature anomalies.
In addition to an advanced numerical forecast model, a complete and modern numerical weather prediction system also encompasses a data assimilation procedure to produce the optimal initial conditions for making forecasts. Atmospheric data assimilation problems, in general, involve large dimensions. The introduction of adjoint techniques into data assimilation has drastically reduced computational costs as well as opened new opportunities in meteorological research areas (Le Dimet and Talagrand, 1986; Courtier and Talagrand, 1990; Rabier et al., 1992, 1996; Courtier et al., 1993, 1994). Based on the Penn State – NCAR Mesoscale Model (MM5), Zou et al. (1995) developed the adjoint model of MM5 to formulate the four-dimensional variational (4D-Var) assimilation system and found significant positive impacts on short-range forecasts after assimilating rainfall observations (Zou and Kuo, 1996). Similarly, aimed at building a WRF-based 4D-Var assimilation system, Xiao et al. (2008) developed the tangent linear (TL) and adjoint (AD) models of the WRF model, with the aid of an automatic differentiation tool (Giering and Kaminski, 2003), and showed that adjusting initial conditions using the WRF adjoint model could significantly improve the Antarctic windstorm forecast. In addition to its essential role in variational data assimilation, adjoint models also have remarkable and versatile potentials in other meteorological applications. Many questions posed in the field of atmospheric science involve sensitivities. An example is how some weather features of interest may change if certain small perturbations are added to the initial conditions. Traditionally, answering these questions would involve multiple model runs with sequentially perturbed components in an initial condition vector. By contrast, with the aid of adjoint models, a comprehensive sensitivity field at the model starting time can be obtained with a single adjoint model run. Errico (1997) and Zou et al. (1997) comprehensively described the applicability of adjoint models in various topics, including parameter estimation, model stability analysis and synoptic studies.
During the last few years, the programming language Python has become increasingly popular among atmospheric science researchers as well as those from other Earth science disciplines (Lin, 2012). Due to its concise and natural syntax, programmes written in Python are clear and easy to read. Well-developed packages are readily available to handle input/output (IO) tasks in various encoding formats commonly seen in the field of Earth science, such as Gridded Binary, Network Common Data Form, Hierarchical Data Format (HDF) 4 and HDF5. In tasks involving modelling, the handling of date and time is also frequently dealt with, which is also made exceedingly simple in Python (Lutz, 2013). Nonetheless, Python comes with disadvantages, including much slower computational speeds than compiled programming languages like Fortran. In this study, the driver layer of the non-linear MPAS-A is redeveloped in Python to control the simulation workflow, including IO, date and time in model simulations to reconcile the advantages and disadvantages of both Python and Fortran. The heavy-duty dynamical calculations are retained in Fortran so that the model’s computational speed is hardly compromised. Such a structural arrangement makes the installation, compilation and modification of the model even more user- and platform-friendly than without using the Python driver. As this research involves only the dynamical core of the MPAS-A without moist physics for the moment, conducted are numerical experiments with the baroclinic wave case of Jablonowski and Williamson (JW), a case commonly adopted for testing the performance and stability of numerical weather prediction (NWP) models’ dynamical cores in numerous previous studies (Jablonowski and Williamson, 2006a, 2006b; Park et al., 2013). Section 2 briefly introduces some technical features of the non-hydrostatic MPAS-A model and the redeveloped Python driver layer. Section 3 presents the development of the TL model using the same combined Python–Fortran structure and tests its correctness. Section 4 involves the development of correctness checks of the MPAS-A adjoint model, and Section 5 presents some results from two adjoint relative sensitivity studies. Section 6 gives a summary and conclusions.
A brief description of the MPAS atmospheric solver and case selection
The MPAS is a global modelling system that uses spherical centroidal Voronoi tessellations discretization, allowing for both quasi-uniform and smoothly variable horizontal resolution meshes. The MPAS framework houses multiple geophysical dynamical solvers, including the atmosphere, ocean, sea ice and land ice. MPAS-A is a compressible, non-hydrostatic atmospheric model. The variable-resolution-permitting centroidal mesh is suitable for regional NWPs and regional climatological studies over high-resolution regions (Skamarock et al., 2012; Hagos et al., 2013; Park et al., 2013). Figure 1 is an example of a refined circular area centred at [50° N, 170° W] with a horizontal resolution of about 25 km between neighbouring cell centres, which progressively becomes coarser and coarser to 92 km in remote parts of the globe. Figure 1b demonstrates the actual distributions of centroidal grid cells within the transitional region indicated by a white box in Fig. 1a. The mesh with gradually varying resolutions is to be contrasted with the traditionally two-way nested grids (Skamarock and Klemp, 2008). In the settings of nested grids, horizontal resolutions change abruptly over the boundaries of the refined areas, possibly generating additional systematic errors in the simulation results (Hagos et al., 2013).
The MPAS-A inherited the approach of the Advanced Research WRF (ARW) for solving the non-hydrostatic dynamical and physical equations, including the staggered C-grid spatial discretization and the third-order Runge–Kutta temporal integration scheme. In addition to the Runge–Kutta scheme, also adopted is a time-splitting technique to integrate gravity and acoustic waves. Spatially, the vertical coordinate in the MPAS-A model is terrain-following and height-based, different from the ARW. The MPAS-A model has five prognostic state variables, i.e. horizontal momentum (u) perpendicular to the cell edges; coupled potential temperature (θ), coupled density (ρ) and specific humidity (q) residing at the cell centres; and vertical momentum (w) located at the cell centres horizontally but between layers vertically. Ringler et al. (2008) and Skamarock et al. (2012) provided more details about the unstructured centroidal mesh and MPAS-A dynamics solver, respectively.
In this study, the MPAS-A is restructured so that the driver layer is written in Python, and the mediation and model layers are retained in Fortran, in which the model layer is where the actual dynamical computations take place and the mediation layer is an intermediate layer of the software placed between the driver and model layers. Under this framework, the functionalities in the driver layer, including installation, compilation, controlling of the simulation time flow and IO, are made exceedingly user- and platform-friendly (see Appendix A), which will also be the case for a prospective 4D-Var assimilation system. The fast computation efficiency of Fortran is maintained under the Python-driving framework since the heavy-duty mediation and model layers are still written in Fortran. The Python intrinsic utility F2Py, i.e. Fortran to Python interface generator, builds the interfaces between the Python driver layer and Fortran mediation/model layers, which compiles Fortran codes with common Fortran compilers (e.g. gfortran, ifort and pgfortran), providing a connection between the two programming languages (Peterson, 2009). In this study, the TL/AD Fortran programmes are developed line-by-line manually following the guidance detailed in the work by Zou et al. (1997), without using any automatic differentiation tool. Under this Python-driving structure, the non-linear forward, TL, and adjoint models of the MPAS-A performs simulations exactly like the original MPAS-A, regardless of the types of meshes chosen in different numerical experiments.
The JW test case is adopted in this research to test the simulation performances of the MPAS-A non-hydrostatic dynamical core with both quasi-uniform and variable-resolution meshes as well as those of the accuracy of both the TL and adjoint models. For a demonstration of the benefits from the smooth variable-resolution mesh, conducted are three separate numerical experiments with the same Python-driving MPAS-A dynamical core and the same initial conditions but with three different meshes: (1) uniform resolution (UR) at 120 km with 40,962 cells, (2) UR at 30 km with 655,362 cells and (3) variable resolution (VR) ranging from 25 to 92 km with 163,842 cells (Fig. 1a). Starting with the initial conditions of the baroclinic instability test case, the model integrates for a period of nine days. Figure 2 shows the surface pressure features of an evolving baroclinic wave on day 9 from the UR-120-km and UR-30-km experiments and their differences. While the patterns in Fig. 2a,b are similar, differences in the magnitude of surface pressure exceed 15 hPa in places. Figure 3a shows the wave features on day 9 from the VR experiment. As described earlier, the area with the finest resolution is centred at [50° N, 170° W], where most of the wave patterns may be well confined in finer resolutions, guided by the black contours. Comparing results from the VR experiment (Fig. 3a) and from the UR-30-km experiment (Fig. 2b), differences (Fig. 3b) are noticeably smaller than those between the UR-120-km and UR-30-km experiments (Fig. 2c). As indicated by the number of grid cells, the MPAS-A loops through the VR mesh consists of only 25% of the number of cells in the UR-30-km mesh, which is a direct indicator of a significant saving in the computational cost. In reality, for 24-h forecasts with the same integration time step (150 sec), the VR experiment took only 16% of the computational time of the UR-30-km experiment (Table 1). This implies that for a given region of interest, the smoothly variable resolution offered in the MPAS-A may help to achieve high-resolution simulation performances for only a fraction of the computational cost of an experiment with a high-resolution uniform mesh.
Development of the MPAS-A TL model
The non-hydrostatic MPAS-A can be symbolically written as
The correctness of the MPAS-A TL model can be verified by checking whether the following equation is satisfied:
Conducted was a set of five experiments with a 24-h integration period for the TL correctness check. In the first four tests, each model prognostic variable, i.e. u, w, ρ and θ, in vector h in Equation (3) was separately assigned non-zero values. In the fifth experiment, all four prognostic variables in vector h were simultaneously assigned non-zero values. As the scale factor α becomes smaller and smaller, the function Φ(α) is expected to approach unity in an approximately linear way. Figure 4 shows the results of the correctness tests. As anticipated, the values of Φ(α) progressively become closer and closer to unity as the scale factor α gets smaller and smaller for certain ranges of α values in each of the testing experiments. When α is too small, the accuracies of the Φ(α) values start to be affected by the machine round-off errors and drift away from unity. Figure 5 shows the temporal evolutions of Φ(α) with an integration period of 72 h when α equals to 10−3, 10−4 and 10−5. In all three cases, the values of |Φ(α) – 1| gradually increase as time moves on. However, the approximations of perturbation evolutions by the TL model are noticeably better when the initial perturbation sizes are small (α = 10−5, Fig. 5c) than when perturbation sizes are relatively big (α = 10−3, Fig. 5a).
A separate set of experiments is included following Equation (3), in which both x and h in Equation (3) were given the values of the simulation results on day 5 of the JW baroclinic wave case, and α is set as 10−3 (Fig. 6). After a four-day integration, the simulation results reach the same time as the results shown in Figs. 2 and 3, i.e. day 9. Figure 7a shows the differences between the two non-linear model simulations calculated with the MPAS dynamic core only, i.e. in Equation (3), and Fig. 7b shows the results from the TL model taking αh as input, i.e. in Equation (3). Both results appear almost identical, indicating a close approximation of the TL model forecast to the evolution of perturbations as the non-linear model differences. Figure 7c shows pixel-by-pixel comparisons of the results in Fig. 7a,b, suggesting a high correlation between the two, with a root-mean-square error of 0.00454 hPa in the linear regression fit.
Development of the MPAS-A adjoint model
The MPAS-A adjoint model is essentially the transpose of the MPAS-A TL model, expressed as
The correctness of the MPAS-A adjoint model can be verified based on the following equation:
The left-hand side (LHS) of Equation (5) is the inner product of the TL model forecast at time tr from an initial condition at time t0, i.e. with a vector and the right-hand side (RHS) is the inner product of with the output of the adjoint model integration from the time tr to t0, i.e. If the adjoint model is developed correctly, the LHS and RHS of Equation (5) is expected to agree with the machine accuracy of the data type declared in the program, which is double precision in the MPAS-A, for any vectors of and Following Equation (5), the first set of five experiments is included with the integration time tr equal to 1, 3, 6, 9 and 12 h with and The second set of testing experiments is the same as the first set except for changing to the 48-h non-linear model forecasts 48h(x0). The resulting LHS and RHS from the five tests of both sets agree with the precision of machine accuracies, indicating the correctness of the MPAS-A adjoint model (Table 2).
A brief example of the coding in the restructured Python–Fortran MPAS-A, as well as how divergence is calculated under the Voronoi mesh in MPAS with its corresponding TL/AD can be found in Appendices A and B, respectively. The naming convention for a given prognostic or diagnostic variables in the MPAS-A model, such as ‘divergence’, is ‘divergence_tl’ and ‘divergence_ad’ in the TL and adjoint models, respectively. The divergence at a cell centre can be found with a summation of wind vectors at all edges of the cell. All other quantities in the atmospheric dynamical equations have also been converted similarly in the TL/AD Fortran codes. Following the MPAS-A Fortran code, the TL and adjoint models are finally developed.
MPAS-A adjoint sensitivity analysis
An advantageous application of the adjoint model is sensitivity analysis (Errico and Vukicevic, 1992; Errico, 1997; Zou et al., 1997). A given quantity of interest (J) at the model forecast time, such as mean squared error (MSE), vorticity, surface pressure etc., can be denoted as
The limitations of this method are: (1) the magnitudes of the perturbation in the initial conditions have to be sufficiently small to ensure a good approximation of the gradients of J with respect to the perturbed variable and (2) many model runs are required to find a comprehensive profile of J’s sensitivities to all of the variables at various geographical locations. By contrast, the sensitivities of the response functions with respect to the model variables may be expressed as (Errico, 1997)
Compared with the term in Equation (7), the term in Equation (8) is the transpose representing the adjoint model. Note that the sensitivities of J, calculated using a single adjoint model run, are (1) independent from the perturbation sizes and (2) with respect to all model variables over the entire model domain in the initial conditions.
In the experiments of this study, the response function in the adjoint sensitivity analysis is defined as the surface pressure at two individual points, A and B, on day 9, as marked in Fig. 3a:
The adjoint sensitivity results in the experiments discussed above serve as a preliminary example for the strength of the MPAS-A adjoint model, which can efficiently help to find the areas and fields in the initial conditions that have the greatest impacts on a given scalar of interest in the forecasts. The sensitivity field shows a strong dependence on the state flow near the point of interest, which also implies the reason that flow-dependent perturbations are implicitly accounted for in 4D-Var assimilation systems. Similar applications can readily be inferred such as finding the sensitivity field of the vorticity or central surface pressure at a preceding time in a tropical cyclone case.
Summary and conclusions
The weather forecast capabilities over the advanced irregular centroidal mesh offered by the MPAS-A is unique for weather and climate research. In this study, the MPAS-A is restructured in such a way that the driver layer is developed in Python for its versatility and convenience when controlling the workflow of the model and that the mediation and model layers are retained in Fortran for its efficient computational speed. In the case of the Jablonowski and Williamson baroclinic wave, compared are the simulation results from the experiments using the Python-driving MPAS-A with both quasi-uniform and variable-resolution meshes. High-resolution simulation performances over regions of interest can be achieved using the variable-resolution mesh for a small fraction of the computational cost compared to using the global uniformly high-resolution mesh.
An adjoint model is a remarkably versatile tool for various sensitivity-involved meteorological research topics and serves a vital role in a four-dimensional variational assimilation system. The tangent linear (TL) and adjoint models of the MPAS-A dynamical core are developed under the same Python-driving framework in this research. Necessary correctness verification procedures are carried out and passed to ensure that both the TL and adjoint models generate accurate results. The TL model closely approximates the evolutions of perturbations after a four-day forecast when compared with the simulation differences between the perturbed and original non-linear MPAS-A model. Additional experiments are included to demonstrate the applicability of the MPAS-A adjoint model in efficiently calculating sensitivities of a given quantity of interest at forecast time, or the so-called response function, to the initial conditions or forecast model inputs. Specifically, the relative sensitivities of the surface pressure on day 9 at two separate locations, one at the centre of a low-pressure system (point A) and the other in a fair-weather region (point B), to the model variables 12 and 24 h earlier than day 9 are calculated with the adjoint model. The surface pressure at point A is primarily sensitive to areas upstream of the isobars near the centre of the low-pressure system horizontally and exhibits a westward vertical tilting. By comparison, with calm weather at point B, areas showing significant sensitivities are located near point B, regardless of the simulation time.
The physical options currently available in the MPAS-A model can readily be inserted into the Python–Fortran structure proposed in this study. Future research efforts will focus on first developing an MPAS TL/AD system with moist physics and planetary boundary layer physics parameterization schemes and then building a comprehensive global 4D-Var assimilation system.