src.util.basic module

Classes and functions that define and operate on basic data structures.

src.util.basic.abstract_attribute(obj=None)

Decorator for abstract attributes in abstract base classes by analogy with abc.abstract_method(). Based on https://stackoverflow.com/a/50381071.

class src.util.basic.MDTFABCMeta(name, bases, namespace, /, **kwargs)

Bases: ABCMeta

Wrap the metaclass for abstract base classes to enable definition of abstract attributes via abstract_attribute(). Based on https://stackoverflow.com/a/50381071.

Raises:

NotImplementedError – If a child class doesn’t define an abstract attribute, by analogy with abc.abstract_method().

mro()

Return a type’s method resolution order.

class src.util.basic.Singleton

Bases: type

mro()

Return a type’s method resolution order.

class src.util.basic.MultiMap(*args, **kwargs)

Bases: defaultdict

Extension of the dict class that allows doing dictionary lookups from either keys or values.

Syntax for lookup from keys is unchanged, while lookup from values is done on the inverse() attribute and returns a list of matching keys if more than one match is present. See https://stackoverflow.com/a/21894086.

Example:

>>> d = MultiMap({'key1': 'val', 'key2':'val'})

>>> d['key1']
'val'

>>> d.inverse['val']
['key1', 'key2']

__init__(*args, **kwargs)

Inherited from collections.defaultdict. Construct by passing an ordinary dict.

get_(key)

Re-implement __getitem__ to handle returning possibly multiple items.

to_dict()

Convert to ordinary dict.

inverse()

Construct inverse dict mapping values to keys.

inverse_get_(val)

Construct inverse dict and return keys corresponding to val.

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

class src.util.basic.WormDict(dict=None, /, **kwargs)

Bases: UserDict, dict

Dict which raises exceptions when trying to overwrite or delete an existing entry. “WORM” is an acronym for “write once, read many.”

Raises:

WormKeyError – If code attempts to reassign or delete an existing key.

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

class src.util.basic.ConsistentDict(dict=None, /, **kwargs)

Bases: WormDict

Like WormDict, but we only raise WormKeyError if we try to reassign to a different value.

Raises:

WormKeyError – If code attempts to reassign an existing key to a value different than the current one.

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

class src.util.basic.WormDefaultDict(default_factory=None, *args, **kwargs)

Bases: WormDict

src.util.basic.WormDict with collections.defaultdict functionality.

__init__(default_factory=None, *args, **kwargs)

Inherited from collections.defaultdict and takes same arguments.

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

class src.util.basic.NameSpace

Bases: dict

A dictionary that provides attribute-style access.

For example, d[‘key’] = value becomes d.key = value. All methods of dict are supported.

Note

Recursive access (d.key.subkey, as in C-style languages) is not supported.

Implementation is based on https://github.com/Infinidat/munch.

Raises:

AttributeError – In cases where dict would raise a KeyError.

toDict() → dict

Recursively converts a NameSpace back into a dictionary.

classmethod fromDict(x)

Recursively transforms a dictionary into a NameSpace via copy. (Note: as dicts are not hashable, they cannot be nested in sets/frozensets.)

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

class src.util.basic.MDTFEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: _MDTFEnumMixin, Enum

Customize behavior of Enum:

1. Assign (integer) values automatically to the members of the enumeration.

2. Provide a from_struct() method to simplify instantiating an instance from a string. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass(). To avoid potential confusion with reserved keywords, we use the Python convention that members of the enumeration are all uppercase.

name = None

classmethod from_struct(str_)

Instantiate from string.

class src.util.basic.ObjectStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: MDTFEnum

util.MDTFEnum used to track the status of an object hierarchy object: - NOTSET: the object hasn’t been fully initialized. - ACTIVE: the object is currently being processed by the framework. - INACTIVE: the object has been initialized, but isn’t being processed (e.g.,

alternate VarlistEntrys).

• FAILED: processing of the object has encountered an error, and no further work will be done.

• SUCCEEDED: Processing finished successfully.

name = None

classmethod from_struct(str_)

Instantiate from string.

NOTSET = 1

ACTIVE = 2

INACTIVE = 3

FAILED = 4

SUCCEEDED = 5

src.util.basic.sentinel_object_factory(obj_name)

Return a unique singleton object/class (same difference for singletons). For implementation, see python docs.

class src.util.basic.MDTF_ID(id_=None)

Bases: object

Class wrapping UUID, to provide unique ID numbers for members of the object hierarchy (cases, pods, variables, etc.), so that we don’t need to require that objects in these classes have unique names.

__init__(id_=None)

Parameters:

id_ (optional) – hard-code an ID instead of generating one.

src.util.basic.is_iterable(obj)

Test if obj is an iterable collection.

Parameters:

obj – Object to test.

Returns:

bool – True if obj is an iterable collection and not a string.

src.util.basic.to_iter(obj, coll_type=<class 'list'>)

Cast arbitrary object obj to an iterable collection. If obj is not a collection, returns a one-element list containing obj.

Parameters:

• obj – Object to cast to collection.

• coll_type – One of list, set or tuple, default list. Class to cast obj to.

Returns:

obj, cast to an iterable collection of type coll_type.

src.util.basic.from_iter(obj)

Inverse of to_iter(). If obj is a single-element iterable collection, return its only element.

src.util.basic.remove_prefix(s1, s2)

If string s1 starts with string s2, return s1 with s2 removed. Otherwise return s1 unmodified.

src.util.basic.remove_suffix(s1, s2)

If string s1 ends with string s2, return s1 with s2 removed. Otherwise return s1 unmodified.

src.util.basic.filter_kwargs(kwarg_dict, function)

Given keyword arguments kwarg_dict, return only those kwargs accepted by function.

Parameters:

• kwarg_dict (dict) – Keyword arguments to be passed to function.

• function (function) – Function to be called.

Returns:

dict – Subset of key:value entries of kwarg_dict where keys are keyword arguments recognized by function.

src.util.basic.splice_into_list(list_, splice_d, key_fn=None, log=<Logger>)

Splice sub-lists (values of splice_d) into list list_ after their corresponding entries (keys of slice_d). Example:

>>> splice_into_list(['a','b','c'], {'b': ['b1', 'b2']})
['a', 'b', 'b1', 'b2', 'c']

Parameters:

• list_ (list) – Parent list to splice sub-lists into.

• splice_d (dict) – Sub-lists to splice in. Keys are entries in list_ and values are the sub-lists to insert after that entry. Duplicate or missing entries are handled appropriately.

• key_fn (function) – Optional. If supplied, function applied to elements of list_ to compare to keys of splice_d.

Returns:

Spliced list_ as described above.

src.util.basic.canonical_arg_name(str_)

Convert a flag or other specification to a destination variable name. The destination variable name always has underscores, never hyphens, in accordance with PEP8.

E.g., canonical_arg_name('--GNU-style-flag') returns “GNU_style_flag”.

src.util.basic.plugin_key(plugin_name)

Convert user input for plugin options to string used to lookup plugin value from options defined in cli_plugins.jsonc files.

Ignores spaces and underscores in supplied choices for CLI plugins, and make matching of plugin names case-insensititve.

src.util.basic.word_wrap(str_)

Clean whitespace and perform 80-column word wrapping for multi-line help and description strings. Explicit paragraph breaks must be encoded as a double newline (\n\n).

src.util.basic.iterdict(d)

Iterate through a nested dictionary Return the key-value pair, and a level index for the deepest level

src.util.basic.deactivate(obj, exc, level=None)

Deactivate an object and dependencies and set object status

class src.util.basic.RegexDict

Bases: dict

Utilities to find dictionary entries using regular expressions Credit: https://stackoverflow.com/questions/21024822/python-accessing-dictionary-with-wildcards

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get_matching_value(query)

Return the value corresponding to query

get_all_matching_values(queries: list)

Return a tuple of all matching values corresponding to each entry in a list of queries