1. Overview¶
Welcome! In this section we’ll describe what the Model Diagnostics Task Force (MDTF) framework is, how it works, and how you can contribute your own diagnostic scripts.
1.1. Purpose¶
The scientific motivation and content behind the framework was described in E. D. Maloney et al. (2019): Process-Oriented Evaluation of Climate and Weather Forecasting Models. BAMS, 100 (9), 1665–1686, doi:10.1175/BAMS-D-18-0042.1.
1.2. Framework operation¶
The design goal of the MDTF framework is to provide a portable and adaptable means to run process-oriented diagnostic scripts, abbreviated as PODs below. By “portability,” we mean the ideal of “run once, run anywhere”: the purpose of the framework is to automate retrieval of model data from different local or remote sources, and transform that data into a layout (field names, variable units, etc.) your script expects. This will empower your analysis to be run by a wider range of researchers on a wider range of models.
As shown in the figure above, the MDTF framework itself performs common data management and support tasks (gray boxes) before and after the individual POD scripts are run. The PODs (colored boxes) are developed by different research groups and run independently of one another. Each POD takes as input
requested variables from the model run, along with
any required observational or supporting data, performs an analysis, and produces
a set of figures which are presented to the user in a series of .html files.
We do not include or require a mechanism for publishing these webpages on the internet; html is merely used as a convenient way to present a multimedia report to the user.
1.3. Getting started for users¶
The rest of the documentation in this section describes next steps for end users of the framework:
We provide instructions on how to download and install the framework and run it on sample model data.
We describe the most common configuration options for running the framework on your own model data. Also see the full list of command-line options.
If you encounter a bug, check the GitHub issue tracker.
1.4. Getting started for POD developers¶
Information for researchers wishing to contribute a POD to the framework is provided in the Developer Information section The framework is designed to require minimal changes to existing analysis scripts. We recommend that developers of new PODs start independently of the framework and adapt it for the framework’s use once it’s fully debugged. As summarized in the figure above, the changes needed to convert an existing analysis script for use in the framework are:
Provide a settings file which tells the framework what it needs to do: what languages and libraries your code need to run, and what model data your code takes as input.
Adapt your code to load data files from locations set in unix shell environment variables (we use this as a language-independent way for the framework to communicate information to the POD).
Provide a template web page which links to, and briefly describes, the plots generated by the script.