# 7. Walkthrough of framework operation¶

In this section, we describe the actions that are taken when the framework is run, focusing on aspects that are relevant for the operation of individual PODs. The Example Diagnostic POD (short name: example) is used as a concrete example here to illustrate how a POD is implemented and integrated into the framework.

We begin with a reminder that there are 2 essential files for the operation of the framework and POD:

• src/default_tests.jsonc: configuration input for the framework.

• diagnostics/example/settings.jsonc: settings file for the example POD.

To setup for running the example POD, (1) download the necessary supporting and NCAR-CAM5.timeslice sample data and unzip them under inputdata/, and (2) open default_tests.jsonc, uncomment the whole NCAR-CAM5.timeslice section in case_list, and comment out the other cases in the list. We also recommend setting both save_ps and save_nc to true.

## 7.1. Step 1: Framework invocation¶

The user runs the framework by executing the framework’s main driver script $CODE_ROOT/mdtf, rather than executing the PODs directly. This is where the user specifies the model run to be analyzed, and chooses which PODs to run via the pod_list section in default_tests.jsonc. • Some of the configuration options can be input through command line, see the command line reference or run %$CODE_ROOT/mdtf --help.

At this stage, the framework also creates the directory $OUTPUT_DIR/ (default: mdtf/wkdir/) and all subdirectories therein for hosting the output files by the framework and PODs from each run. • If you’ve run the framework with both save_ps and save_nc in default_tests.jsonc set to true, check the output directory structure and files therein. Note that when running, the framework will keep collecting the messages relevant to individual PODs, including (1) the status of required data and environment, and (2) texts printed out by PODs during execution, and will save them as log files under each POD’s output directory. These log files can be viewed via the top-level results page index.html and, together with messages printed in the terminal, are useful for debugging. ### Example diagnostic¶ Run the framework using the NCAR-CAM5.timeslice case. After successful execution, open the index.html under the output directory in a web browser. The plots links to the webpage produced by the example POD with figures, and log to example.log including all example-related messages collected by the framework. The messages displayed in the terminal are not identical to those in the log files, but also provide a status update on the framework-POD operation. ## 7.2. Step 2: Data request¶ Each POD describes the model data it requires as input in the varlist section of its settings.jsonc, with each entry in varlist corresponding to one model data file used by the POD. The framework goes through all the PODs to be run in pod_list and assembles a list of required model data from their varlist. It then queries the source of the model data ($DATADIR/) for the presence of each requested variable with the requested characteristics (e.g., frequency, units, etc.).

• The most important features of settings.jsonc are described in the settings documentation and full detail on the reference page.

• Variables are specified in varlist following CF convention wherever possible. If your POD requires derived quantities that are not part of the standard model output (e.g., column weighted averages), incorporate necessary preprocessing for computing these from standard output variables into your code. PODs are allowed to request variables outside of the CF conventions (by requiring an exact match on the variable name), but this will severely limit the POD’s application.

• Some of the requested variables may be unavailable or without the requested characteristics (e.g., frequency). You can specify a backup plan for this situation by designating sets of variables as alternates if feasible: when the framework is unable to obtain a variable that has the alternates attribute in varlist, it will then (and only then) query the model data source for the variables named as alternates.

• If no alternates are defined or the alternate variables are also unavailable, the framework will skip executing your POD, and an error log will be presented in index.html.

Once the framework has determined which PODs are able to run given the model data, it prepares the necessary environment variables, including directory paths and the requested variable names (as defined in data/fieldlist_$convention.jsonc) for PODs’ operation. • At this step, the framework also checks the PODs’ observational/supporting data under inputdata/obs_data/. If the directory of any of the PODs in pod_list is missing, the framework would terminate with error messages showing on the terminal. Note that the framework only checks the presence of the directory, but not the files therein. ### Example diagnostic¶ The example POD uses only one model variable in its varlist: surface air temperature, recorded at monthly frequency. • In the beginning of example.log, the framework reports finding the requested model data file under Found files. • If the framework could not locate the file, the log would instead record Skipping execution with the reason being missing data. ## 7.3. Step 3: Runtime environment configuration¶ The framework reads the other parts of your POD’s settings.jsonc, e.g., pod_env_vars, and generates additional environment variables accordingly (on top of those being defined through default_tests.jsonc). Furthermore, in the runtime_requirements section of settings.jsonc, we request that you provide a list of languages and third-party libraries your POD uses. The framework will check that all these requirements are met by one of the Conda environments under $CONDA_ENV_DIR/.

• The requirements should be satisfied by one of the existing generic Conda environments (updated by you if necessary), or a new environment you created specifically for your POD.

• If there isn’t a suitable environment, the POD will be skipped.

Note that the framework’s information about the Conda environments all comes from the YAML (.yml) files under src/conda/ (and their contents) by assuming that the corresponding Conda environments have been installed using (thus are consistent with) the YAML files.

4. plots model figure $WK_DIR/model/PS/example_model_plot.eps, 5. reads the digested data in time-averaged form at $OBS_DATA/example_tas_means.nc, and plots the figure to $WK_DIR/obs/PS/example_obs_plot.eps. Note that these tasks correspond to the code blocks 1) through 5) in the script. • When the script is called and running, it prints out messages which are saved in example.log. These are helpful to determine when and how the POD execution is interrupted if there’s a problem. • The script is organized to deal with model data first, and then to process digested observations. Thus if something goes wrong with the digested data, the script is still able to produce the html page with model figures. This won’t happen if code block 5) is moved before 4), i.e., well-organized code is more robust and may be able to produce partial results even when it encounters problems. In code block 7) of example-diag.py, we include an example of exception handling by trying to access a non-existent file (the final block is just to confirm that the error would not interrupt the script’s execution because of exception-handling). • The last few lines of example.log demonstrate the script is able to finish execution despite an error having occurred. Exception handling makes code robust. ## 7.5. Step 5: Output and cleanup¶ At this point, your POD has successfully finished running, and all remaining tasks are handled by the framework. The framework converts the postscript plots to bitmaps according to the following rule: • $WK_DIR/model/PS/filename.eps$WK_DIR/model/filename.png • $WK_DIR/obs/PS/filename.eps$WK_DIR/obs/filename.png The html template for each POD is then copied to $WK_DIR by the framework.

• In writing the template file all plots should be referenced as relative links to this location, e.g., “<A href=model/filename.png>”. See templates from existing PODs.

• Values of all environment variables referenced in the html template are substituted by the framework, allowing you to show the run’s CASENAME, date range, etc. Text you’d like to change at runtime must be changed through environment variables (the v3 framework doesn’t allow other ways to alter the text of your POD’s output webpage at runtime).

• If save_ps and save_nc are set to false, the .eps and .nc files will be deleted.

Finally, the framework links your POD’s html page to the top-level index.html, and copies all files to the specified output location (OUTPUT_DIR in default_tests.jsonc; same as WK_DIR by default).

• If make_variab_tar in default_tests.jsonc is set to true, the framework will create a tar file for the output directory, in case you’re working on a server, and have to move the file to a local machine before viewing it.

### Example diagnostic¶

Open the html template diagnostics/example/example.html and the output \$WK_DIR/example.html in a text editor, and compare. All the environment variables in the template have been substituted, e.g., {EXAMPLE_FAV_COLOR} becomes blue (defined in pod_env_vars in settings.jsonc).