src.core module¶
Definition of the MDTF framework main loop and classes implementing basic, supporting functionality.
- class src.core.ObjectStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)¶
Bases:
MDTFEnum
util.MDTFEnum
used to track the status of an object hierarchy object (child class ofMDTFObjectBase
):NOTSET: the object hasn’t been fully initialized.
ACTIVE: the object is currently being processed by the framework.
INACTIVE: the object has been initialized, but isn’t being processed (e.g., alternate
VarlistEntry
s).FAILED: processing of the object has encountered an error, and no further work will be done.
SUCCEEDED: Processing finished successfully.
- NOTSET = 1¶
- ACTIVE = 2¶
- INACTIVE = 3¶
- FAILED = 4¶
- SUCCEEDED = 5¶
- class src.core.MDTFObjectBase(*args, **kwargs)[source]¶
Bases:
object
Base class providing shared functionality for the object hierarchy, which is:
The framework itself (
MDTFFramework
);DataSourceBase
s belonging to a run of the package;Diagnostic
s (PODs) belonging to aDataSourceBase
;VarlistEntry
s (requested model variables) belonging to aDiagnostic
.
- status: ObjectStatus = 1¶
- property full_name¶
- property failed¶
- property active¶
- iter_children(child_type=None, status=None, status_neq=None)[source]¶
Generator iterating over child objects associated with this object.
- Parameters:
child_type – None or Type type; default None. If None, iterates over
type (all child objects regardless of their) –
status – None or
ObjectStatus
, default None. If None, iterates over all child objects, regardless of status. If aObjectStatus
value is passed, only iterates over child objects with that status.status_neq – None or
ObjectStatus
, default None. If set, iterates over child objects which don’t have the given status. If status is set, this setting is ignored.
- class src.core.ConfigTuple(name, backup_filename, contents)¶
Bases:
tuple
Class wrapping general structs used for configuration.
- backup_filename¶
- contents¶
- name¶
- class src.core.ConfigManager(*args, **kwargs)[source]¶
-
- backup_config(cli_obj, case_d)[source]¶
Copy serializable version of parsed settings, in order to write backup config file.
- classmethod fromDict(x)¶
Recursively transforms a dictionary into a NameSpace via copy. (Note: as dicts are not hashable, they cannot be nested in sets/frozensets.)
- fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
- toDict()¶
Recursively converts a NameSpace back into a dictionary.
- class src.core.PathManager(*args, **kwargs)[source]¶
-
Singleton
holding the root directories for all paths used by the code.- classmethod fromDict(x)¶
Recursively transforms a dictionary into a NameSpace via copy. (Note: as dicts are not hashable, they cannot be nested in sets/frozensets.)
- fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
- toDict()¶
Recursively converts a NameSpace back into a dictionary.
- class src.core.TranslatedVarlistEntry(coords: InitVar = None, name: str = sentinel.Mandatory, standard_name: str = sentinel.Mandatory, units: Units = sentinel.Mandatory, modifier: str = '', component: str = '', associated_files: str = '', rename_coords: bool = True, convention: str = sentinel.Mandatory, log: Any = sentinel.Mandatory)[source]¶
Bases:
DMVariable
Class returned by
VarlistTranslator.translate()
. Marks some attributes inherited fromDMVariable
as being queryable inquery_dataset()
.- property T¶
Return T axis dimension coordinate if defined, else None.
- property X¶
Return X axis dimension coordinate if defined, else None.
- property Y¶
Return Y axis dimension coordinate if defined, else None.
- property Z¶
Return Z axis dimension coordinate if defined, else None.
- add_scalar(ax, ax_value, **kwargs)¶
Metadata operation corresponding to taking a slice of a higher-dimensional variable (extracting its values at axis ax = ax_value). The coordinate corresponding to ax is removed from the list of coordinate dimensions and added to the list of scalar coordinates.
- property axes¶
Superset of the
dim_axes()
dict (whose values contain coordinate dimensions only) that includes axes corresponding to scalar coordinates.
- property axes_set¶
Superset of the
dim_axes_set()
frozenset (which contains axes labels corresponding to coordinate dimensions only) that includes axes labels corresponding to scalar coordinates.
- build_axes(*coords, verify=True)¶
Constructs a dict mapping axes labels to dimension coordinates (of type
AbstractDMCoordinate
.)
- change_coord(ax_name, new_class=None, **kwargs)¶
Replace attributes on a given coordinate, but also optionally cast them to new classes.
- Parameters:
ax_name – Name of the coordinate to modify.
new_class (optional) – new class to cast the returned coordinate to.
kwargs – Set of attribute names and values to replace on the returned copy.
- coords: InitVar = None¶
- property dim_axes¶
Retrun dict mapping axes labels (‘X’, ‘Y’, etc.) to corresponding dimension coordinate objects.
- property dim_axes_set¶
Return frozenset of dimension coordinate axes labels.
- property full_name¶
Object’s full name, to be used in logging and debugging. Preferred because it eliminates irrelevant information in repr(), which is lengthy.
- get_scalar(ax_name)¶
If the axis label ax_name is a scalar coordinate, return the corresponding
AbstractDMCoordinate
object, otherwise return None.
- property is_static¶
Whether the variable has time dependence (bool).
- remove_scalar(ax, position=-1, **kwargs)¶
Metadata operation that’s the inverse of
add_scalar()
. Given an axis label ax that’s currently a scalar coordinate, remove the slice value and add it to the list of dimension coordinates at position (default end of the list.)
- class src.core.FieldlistEntry(coords: ~dataclasses.InitVar = None, name: str = sentinel.Mandatory, standard_name: str = sentinel.Mandatory, units: ~src.units.Units = '', modifier: str = '', component: str = '', associated_files: str = '', rename_coords: bool = True, scalar_coord_templates: dict = <factory>)[source]¶
Bases:
DMDependentVariable
Class corresponding to an entry in a fieldlist file.
- scalar_name(old_coord, new_coord, log=<Logger>)[source]¶
Uses one of the scalar_coord_templates to construct the translated variable name for this variable on a scalar coordinate slice (eg. pressure level).
- property T¶
Return T axis dimension coordinate if defined, else None.
- property X¶
Return X axis dimension coordinate if defined, else None.
- property Y¶
Return Y axis dimension coordinate if defined, else None.
- property Z¶
Return Z axis dimension coordinate if defined, else None.
- add_scalar(ax, ax_value, **kwargs)¶
Metadata operation corresponding to taking a slice of a higher-dimensional variable (extracting its values at axis ax = ax_value). The coordinate corresponding to ax is removed from the list of coordinate dimensions and added to the list of scalar coordinates.
- property axes¶
Superset of the
dim_axes()
dict (whose values contain coordinate dimensions only) that includes axes corresponding to scalar coordinates.
- property axes_set¶
Superset of the
dim_axes_set()
frozenset (which contains axes labels corresponding to coordinate dimensions only) that includes axes labels corresponding to scalar coordinates.
- build_axes(*coords, verify=True)¶
Constructs a dict mapping axes labels to dimension coordinates (of type
AbstractDMCoordinate
.)
- change_coord(ax_name, new_class=None, **kwargs)¶
Replace attributes on a given coordinate, but also optionally cast them to new classes.
- Parameters:
ax_name – Name of the coordinate to modify.
new_class (optional) – new class to cast the returned coordinate to.
kwargs – Set of attribute names and values to replace on the returned copy.
- coords: dc.InitVar = None¶
- property dim_axes¶
Retrun dict mapping axes labels (‘X’, ‘Y’, etc.) to corresponding dimension coordinate objects.
- property dim_axes_set¶
Return frozenset of dimension coordinate axes labels.
- property full_name¶
Object’s full name, to be used in logging and debugging. Preferred because it eliminates irrelevant information in repr(), which is lengthy.
- get_scalar(ax_name)¶
If the axis label ax_name is a scalar coordinate, return the corresponding
AbstractDMCoordinate
object, otherwise return None.
- property is_static¶
Whether the variable has time dependence (bool).
- remove_scalar(ax, position=-1, **kwargs)¶
Metadata operation that’s the inverse of
add_scalar()
. Given an axis label ax that’s currently a scalar coordinate, remove the slice value and add it to the list of dimension coordinates at position (default end of the list.)
- units: src.units.Units = ''¶
- class src.core.Fieldlist(name: str = sentinel.Mandatory, axes: ~src.util.basic.WormDict = <factory>, axes_lut: ~src.util.basic.WormDict = <factory>, entries: ~src.util.basic.WormDict = <factory>, lut: ~src.util.basic.WormDict = <factory>, env_vars: dict = <factory>)[source]¶
Bases:
object
Class corresponding to a single variable naming convention (single file in data/fieldlist_*.jsonc).
TODO: implement more robust indexing/lookup scheme. standard_name is not a unique identifier, but should include cell_methods, etc. as well as dimensionality.
- to_CF(var_or_name)[source]¶
Returns
FieldlistEntry
for the variable having the given name in this convention.
- to_CF_name(var_or_name)[source]¶
Like
to_CF()
, but only return the CF standard name, given the name in this convention.
- from_CF(var_or_name, modifier=None, num_dims=0, has_scalar_coords_att=False, name_only=False)[source]¶
Look up
FieldlistEntry
corresponding to the given standard name, optionally providing a modifier to resolve ambiguity.TODO: this is a hacky implementation; FieldlistEntry needs to be expanded with more ways to uniquely identify variable (eg cell methods). :param var_or_name: variable or name of the variable :param modifier: optional string to distinguish a 3-D field from a 4-D field with :param the same var_or_name value: :param num_dims: number of dimensions of the POD variable corresponding to var_or_name :param has_scalar_coords_att: boolean indicating that the POD variable has a scalar_coords :param attribute: :param and therefore requires a level from a 4-D field: :param name_only: boolean indicating to not return a modifier–hacky way to accommodate :param a from_CF_name call that does not provide other metadata:
- from_CF_name(var_or_name, modifier=None)[source]¶
Like
from_CF()
, but only return the variable’s name in this convention.- Parameters:
var_or_name – variable or name of the variable
modifier – optional string to distinguish a 3-D field from a 4-D field with
value (the same var_or_name) –
- translate_coord(coord, log=<Logger>)[source]¶
Given a
DMCoordinate
, look up the corresponding translatedDMCoordinate
in this convention.
- translate(var)[source]¶
Returns
TranslatedVarlistEntry
instance, with populated coordinate axes. Units of scalar coord slices are translated to the units of the conventions’ coordinates. Includes logic to translate and rename scalar coords/slices, e.g.VarlistEntry
for ‘ua’ (intrinsically 4D) @ 500mb could produce aTranslatedVarlistEntry
for ‘u500’ (3D slice), depending on naming convention.
- class src.core.NoTranslationFieldlist(*args, **kwargs)[source]¶
Bases:
Singleton
Class which partially implements the
Fieldlist
interface but does no variable translation.VarlistEntry
objects from the POD are passed through to createTranslatedVarlistEntry
objects.- translate(var)[source]¶
Returns
TranslatedVarlistEntry
instance, populated with contents of inputVarlistEntry
instance.Note
We return a copy of the
VarlistEntry
because logic inDefaultDatasetParser
alters the translation based on the file’s actual contents.
- class src.core.VariableTranslator(*args, **kwargs)[source]¶
Bases:
Singleton
Singleton
containing information for different variable naming conventions. These are defined in thedata/fieldlist_*.jsonc
files.- read_conventions(code_root, unittest=False)[source]¶
Read in the conventions from the Fieldlists and populate the convention attribute.
- get_convention_name(conv_name)[source]¶
Resolve the naming convention associated with a given
Fieldlist
object from among a set of possible aliases.
- class src.core.MDTFFramework(*args, **kwargs)[source]¶
Bases:
MDTFObjectBase
- property full_name¶
- configure(cli_obj, pod_info_tuple, log_config)[source]¶
Wrapper for all configuration done based on CLI arguments.
- parse_mdtf_args(cli_obj, pod_info_tuple)[source]¶
Parse script options returned by the CLI. For greater customizability, most of the functionality is spun out into sub-methods.
- property active¶
- child_deactivation_handler(child, exc)¶
- child_status_update(exc=None)¶
- deactivate(exc, level=None)¶
- iter_children(child_type=None, status=None, status_neq=None)¶
Generator iterating over child objects associated with this object.
- Parameters:
child_type – None or Type type; default None. If None, iterates over
type (all child objects regardless of their) –
status – None or
ObjectStatus
, default None. If None, iterates over all child objects, regardless of status. If aObjectStatus
value is passed, only iterates over child objects with that status.status_neq – None or
ObjectStatus
, default None. If set, iterates over child objects which don’t have the given status. If status is set, this setting is ignored.
- status: ObjectStatus = 1¶
- property failed¶
Overall success/failure of this run of the framework. Return True if any case or any POD has failed, else return False.