4. Running the MDTF-diagnostics package in “multirun” mode¶
Version 3 and later of the MDTF-diagnostics package provides support for “multirun” diagnostics that analyze output from multiple model and/or observational datasets. At this time, the multirun implementation is experimental, and may only be run on appropriately-formatted PODs. “Single-run” PODs that analyze one model dataset and/or one observational dataset must be run separately because the configuration for single-run and multi-run analyses is different. Users and developers should open issues when they encounter bugs or require additional features to support their PODs, or run existing PODs on new datasets.
4.1. The example_multicase POD and configuration¶
A multirun test POD called example_multicase is available in diagnostics/example_multicase that demonstrates
how to configure “multirun” diagnostics that analyze output from multiple datasets.
The multirun_config_template.jsonc file
contains separate pod_list
and case_list
blocks. As with the single-run configuration, the pod_list
may contain
multiple PODs separated by commas. The case_list
contains multiple blocks of information for each case that the
POD(s) in the pod_list
will analyze. The CASENAME
, convention
, FIRSTYR
, and LASTYR
attributes must be
defined for each case. The convention
must be the same for each case, but FIRSTYR
and LASTYR
may differ among cases.
Directions for generating the synthetic data in the configuration file are provided in the file comments, and in the
quickstart section of the README file
The multirun implementation is triggered by setting data_type
to “multi_run” in the environment settings section
of the configuration file, or via the command line. The default data_type
value “single_run” if no value is defined
for data_type
in the configuration file.
As with single_run mode, the OBS_DATA_ROOT
, MODEL_DATA_ROOT
, and WORKING_DIR
must be defined.
However, the OBS_DATA_ROOT
does not require a subdirectory for the POD unless the POD analyzes an observational
dataset. The assumption for now is that multirun PODs will only analyze model datasets; settings for observational
data are retained for backwards compatibility, and needs of multirun POD developers will inform the modification
of the data management options moving forward.
All other settings are identical to those described in the configuration section.
4.2. POD output¶
The framework defines a root directory $WORKING_DIR/[POD name]
for each
POD in the pod_list. $WORKING_DIR/[POD name]
contains the the main framework log files, and subdirectories for each
case. Temporary copies of processed data for each case are placed in
$WORKING_DIR/[pod_name]/[CASENAME]/[data output frequency]
.
The pod html file is written to $OUTPUT_DIR/[POD name]/[POD_name].html
($OUTPUT_DIR
defaults to $WORKING_DIR
if it is not defined), and the output figures are placed in
$OUTPUT_DIR/[POD name]/model
depending on how the paths are defined in the
POD’s html template.
Note that an obs directory is created by default, but will be empty unless the POD developer
opts to use an observational dataset and write observational data figures to this directory.
Figures that are generated as .eps files before conversion to .png files are written to
$WORKING_DIR/[POD name]/model/PS
.
4.3. Multirun environment variables¶
Multirun PODs obtain information for environment variables for the case and variable attributes
described in the configuration section
from a yaml file named case_info.yaml that the framework generates at runtime. The case_info.yaml file is written
to $WORKING_DIR/[POD name]
, and has a corresponding environment variable case_env_file that the POD uses to
parse the file. The example_multicase.py script demonstrates to how to read the environment variables from
case_info.yaml using the case_env_file environment variable into a dictionary,
then loop through the dictionary to obtain the post-processed data for analysis. An example case_info.yaml file
with environment variables defined for the synthetic test data is located in the example_multicase directory.