src.cli module

Classes which parse the framework’s command line interface configuration files and implement the dynamic CLI; see Framework configuration and parsing.

Familiarity with the python argparse module is recommended.

src.cli.canonical_arg_name(str_)[source]

Convert a flag or other specification to a destination variable name. The destination variable name always has underscores, never hyphens, in accordance with PEP8.

E.g., canonical_arg_name('--GNU-style-flag') returns “GNU_style_flag”.

src.cli.plugin_key(plugin_name)[source]

Convert user input for plugin options to string used to lookup plugin value from options defined in cli_plugins.jsonc files.

Ignores spaces and underscores in supplied choices for CLI plugins, and make matching of plugin names case-insensititve.

src.cli.word_wrap(str_)[source]

Clean whitespace and perform 80-column word wrapping for multi-line help and description strings. Explicit paragraph breaks must be encoded as a double newline (\n\n).

src.cli.read_config_files(code_root, file_name, site='')[source]

Utility function to read a pair of configuration files: one for the framework defaults, another optional one for site-specific configuration.

Parameters
Returns

A tuple of the two files’ contents. First element is the site specific file (empty dict if that file isn’t found) and second is the framework file (if not found, fatal error and exit immediately.)

src.cli.read_config_file(code_root, file_name, site='')[source]

Return the site’s config file if present, else the framework’s file. Wraps read_config_files().

Parameters
Returns

Path to the configuration file.

class src.cli.CustomHelpFormatter(*args, **kwargs)[source]

Bases: argparse.RawDescriptionHelpFormatter, argparse.ArgumentDefaultsHelpFormatter

Modify help text formatter to only display variable placeholder text (“metavar”) once, to save space. Taken from https://stackoverflow.com/a/16969505. Also inherit from argparse.RawDescriptionHelpFormatter in order to preserve line breaks in description only (https://stackoverflow.com/a/18462760).

__init__(*args, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

class src.cli.RecordDefaultsAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)[source]

Bases: argparse.Action

Argparse Action that adds a boolean to record if user actually set the argument’s value, or if we’re using the default value specified in the parser. From https://stackoverflow.com/a/50936474. This also re-implements the ‘store_true’ and ‘store_false’ actions, in order to give defaults information on boolean flags.

If the user specifies a value for an option named <option>, the __call__() method adds a variable named <option>_is_default_ to the returned argparse.Namespace. This information is used by MDTFArgParser.parse_args() to populate the is_default attribute of MDTFArgParser.

Subclasses of argparse.Action are only called on user-supplied values, not default values. If the call_on_defaults flag is set on a subclass, MDTFArgParser.parse_args() will also call the action on default values.

default_value_suffix = '_is_default_'
call_on_defaults = False
__init__(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

class src.cli.PathAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)[source]

Bases: src.cli.RecordDefaultsAction

Argparse Action that performs shell environment variable expansion and resolution of relative paths, using resolve_path(). Should be specified as the CLI action for every option taking paths as a value.

call_on_defaults = True
__init__(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

default_value_suffix = '_is_default_'
class src.cli.ClassImportAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)[source]

Bases: src.cli.RecordDefaultsAction

Argparse Action to import classes on demand. Values are looked up from the ‘cli_plugins.jsonc’ file. This is a placeholder used to trigger behavior when arguments are parsed.

call_on_defaults = False
__init__(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

default_value_suffix = '_is_default_'
class src.cli.PluginArgAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)[source]

Bases: src.cli.ClassImportAction

Argparse Action to invoke the CLI plugin functionality, specificially importing the plugin’s entry point via ClassImportAction. All CLI options which define a plugin should specify this as the action.

call_on_defaults = False
__init__(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

default_value_suffix = '_is_default_'
class src.cli.CLIArgument(name: str, action: str = None, nargs: str = None, const: Any = None, default: Any = None, type: Any = None, choices: Iterable = None, required: bool = False, help: str = None, metavar: str = None, dest: str = None, is_positional: bool = False, short_name: str = None, hidden: bool = False)[source]

Bases: object

Class which stores configuration options for a single argument of an argparse.ArgumentParser, with several custom options to simplify the parsing of CLIs defined in JSON files. Attributes correspond to arguments to add_argument().

name: str
action: str = None
nargs: str = None
const: Any = None
default: Any = None
type: Any = None
choices: Iterable = None
required: bool = False
help: str = None
metavar: str = None
dest: str = None
arg_flags: list
is_positional: bool = False
short_name: str = None
hidden: bool = False
__post_init__()[source]

Post-initialization type conversion of attributes.

add(target_p)[source]

Adds the CLI argument to the parser target_p. Wraps add_argument().

Parameters

target_p – Parser object (or argument group, or subparser) to which the argument will be added.

__init__(name: str, action: str = None, nargs: str = None, const: Any = None, default: Any = None, type: Any = None, choices: Iterable = None, required: bool = False, help: str = None, metavar: str = None, dest: str = None, is_positional: bool = False, short_name: str = None, hidden: bool = False)None

Initialize self. See help(type(self)) for accurate signature.

class src.cli.CLIArgumentGroup(title: str, description: str = None, arguments: list = <factory>)[source]

Bases: object

Class holding configuration options for an argparse.ArgumentParser argument group. Attributes correspond to arguments to add_argument_group().

title: str
description: str = None
arguments: list
classmethod from_dict(d)[source]

Initialize an instance of this object from a nested dict d obtained from reading a JSON file.

add(target_p)[source]

Adds the CLI argument group, as well as all arguments it contains, to the parser target_p. Wraps add_argument_group().

Parameters

target_p – Parser object (or subparser) to which the argument group will be added.

__init__(title: str, description: str = None, arguments: list = <factory>)None

Initialize self. See help(type(self)) for accurate signature.

class src.cli.CLIParser(prog: str = None, usage: str = None, description: str = None, epilog: str = None, arguments: list = <factory>, argument_groups: list = <factory>)[source]

Bases: object

Class holding configuration options for an instance of argparse.ArgumentParser (or equivalently a subcommand parser or a CLI plugin). Attributes correspond to arguments given to the argparse.ArgumentParser constructor.

prog: str = None
usage: str = None
description: str = None
epilog: str = None
arguments: list
argument_groups: list
__post_init__()[source]

Post-initialization type conversion of attributes.

classmethod from_dict(d)[source]

Initialize an instance of this object from a nested dict d obtained from reading a JSON file.

iter_args(filter_class=None)[source]

Iterator over all CLIArgument objects associated with this parser; if filter_class is specified, only iterate over objects having (a subclass of) filter_class as their action.

configure(target_p)[source]

Configures a parser object by setting top-level attributes and adding all arguments and argument groups.

Parameters

target_p – Parser object to configure.

add_plugin_args(preparsed_d)[source]

Revise arguments after we know what plugins are being used. This annotates the help string of the plugin selector argument and configures its choices attribute (which lists the allowed values for each option in the online help). It then inserts the plugin-specifc CLI arguments following that argument.

Parameters

preparsed_d – dict of results of the preparsing operation. Keys are the destination strings of the plugin selector arguments (identified by having their action set to PluginArgAction), and values are the values assigned to them by preparsing.

__init__(prog: str = None, usage: str = None, description: str = None, epilog: str = None, arguments: list = <factory>, argument_groups: list = <factory>)None

Initialize self. See help(type(self)) for accurate signature.

class src.cli.CLICommand(name: str, entry_point: str, help: str = '', cli_file: str = None, cli: dict = None, code_root: dataclasses.InitVar = '')[source]

Bases: object

Class holding configuration options for a subcommand (invoked via a subparser) or a plugin.

name: str
entry_point: str
help: str = ''
cli_file: str = None
cli: dict = None
parser: Any = None
code_root: dataclasses.InitVar = ''
__post_init__(code_root)[source]

Post-initialization type converstion of attributes.

import_target()[source]

Imports the function or class referred to by the entry_point attribute.

call(*args, **kwargs)[source]

Imports the function or class referred to by the entry_point attribute, and calls it with the passed arguments.

__init__(name: str, entry_point: str, help: str = '', cli_file: str = None, cli: dict = None, code_root: dataclasses.InitVar = '')None

Initialize self. See help(type(self)) for accurate signature.

class src.cli.DefaultsFileTypes(value)

Bases: src.util.basic.MDTFEnum

MDTFEnum to distinguish the three different categories of input settings files. In order of precedence:

  1. USER: Input settings read from a file supplied by the user.

  2. SITE: Settings specific to the given site (set with the --site flag.)

  3. GLOBAL: Settings applicable to all sites. The main intended use case of this file is to enable the user to configure a default site at install time.

USER = 1
SITE = 2
GLOBAL = 3
class src.cli.CLIConfigManager(*args, **kwargs)[source]

Bases: src.util.basic.Singleton

Singleton to handle search, loading and parsing of configuration files for the CLI and CLI default values. We encapsulate this functionality in its own class, instead of MDTFArgParser or its children, to try to make the code easier to understand (not out of necessity).

Note

This is initialized in MDTFTopLevelArgParser; as a Singleton, it must be properly initialized before being referenced by the classes in this module.

__init__(code_root=None, skip_defaults=False)[source]

Initialize self. See help(type(self)) for accurate signature.

default_site = 'local'

Name of the default value for the --site option.

defaults_filename = 'defaults.jsonc'

Name of the JSONC file for site-specific default settings.

subcommands_filename = 'cli_subcommands.jsonc'

Name of the JSONC files defining site-specific and built-in CLI subcommands.

plugins_filename = 'cli_plugins.jsonc'

Name of the JSONC files defining site-specific and built-in CLI plugins.

property framework_dir

Absolute path to the framework code directory, <CODE_ROOT>/src.

property sites_dir

Absolute path to the directory for site-specific code, <CODE_ROOT>/sites.

property site_dir

Absolute path to the directory for the site-specific code for the chosen site, <CODE_ROOT>/sites/<site>.

read_defaults(def_type, path=None)[source]

Populate one of the entries in self.defaults by reading from the appropriate defaults file.

Parameters
  • def_type (DefaultsFileTypes) – Type of defaults file to read.

  • path (str, optional) – path of the file. Only used for user-specified defaults.

site_default_text()[source]

Return text to be used in online help describing the paths to the defaults files that are being used.

read_subcommands()[source]

Populates subcommands and subparser_kwargs attributes with contents of CLI plugin files for the framework and site. Site-specific subcommand definitions override those defined on the framework.

read_plugins()[source]

Populates plugins attribute with contents of CLI plugin files for the framework and site.

get_plugin(plugin_name, choice_of_plugin=None)[source]

Look up requested CLI plugin from plugins attribute, logging appropriate errors where KeyErrors would be raised.

Parameters
  • plugin_name (str) – Name of the plugin selected.

  • choice_of_plugin (str, optional) – if provided, the name of the choice of plugin.

Returns

CLICommand object corresponding to the requested plugin choice if both arguments are given, or a dict of recognized choices if only the first argument is given.

class src.cli.MDTFArgParser(*args, **kwargs)[source]

Bases: argparse.ArgumentParser

Customized argparse.ArgumentParser. Added functionality:

  • Configuring the parser from an external file (configure()).

  • Customized help text formatting provided by CustomHelpFormatter.

  • Recording whether the user specified each argument value, or whether the default was used, via RecordDefaultsAction.

  • Better bookkeeping of argument groups, e.g. which arguments belong to which group.

__init__(*args, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

static split_args(argv)[source]

Wrapper for shlex.split().

iter_actions()[source]

Iterator over Action objects associated with all user-defined arguments in parser, as well as those for any subcommands.

parse_known_args(args=None, namespace=None)[source]

Wrapper for parse_known_args() which handles intermediate levels of default settings derived from the user’s settings files. These override defaults defined in the parser itself. The precedence order is:

  1. Argument values explictly given by the user on the command line, as recorded in the is_default attribute of MDTFArgParser.

  2. Argument values from a file the user gave via the -f flag. (CLIConfigManager.defaults[DefaultsFileTypes.USER]).

  3. Argument values specified as the default values in the argument parser, which in turn are set with the following precedence order:

    1. Default values from a site-specfic file (defaults.jsonc), stored in CLIConfigManager.defaults[DefaultsFileTypes.SITE].

    2. Default values from a defaults.jsonc file in the /sites directory, stored in CLIConfigManager.defaults[DefaultsFileTypes.GLOBAL].

    3. Default values hard-coded in the CLI definition file itself.

Parameters
  • args (optional) – String or list of strings to parse. If a single string is passed, it’s split using split_args(). If not supplied, the default behavior parses sys.argv().

  • namespace (optional) – An object to store the parsed arguments. The default is a new empty argparse.Namespace object.

Returns

Tuple of 1) populated namespace containing parsed arguments and 2) unrecognized arguments, as with argparse.ArgumentParser.parse_known_args().

parse_args(args=None, namespace=None)[source]

Subclassed implementation of parse_args() which wraps parse_known_args().

class src.cli.MDTFArgPreparser[source]

Bases: src.cli.MDTFArgParser

Parser class used to “preparse” plugin selector arguments, to determine what site and plugin values were set by the user. Plugin selector arguments are identified by having their action set to PluginArgAction.

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

parse_site(argv=None, default_site=None)[source]

Wrapper for parse_known_args() used to determine what site to use.

parse_input_file(argv=None)[source]

Wrapper for parse_known_args() used to determine what user input file to use.

parse_plugins(argv=None)[source]

Wrapper for parse_known_args() used to parse the plugin selector arguments.

iter_actions()

Iterator over Action objects associated with all user-defined arguments in parser, as well as those for any subcommands.

parse_args(args=None, namespace=None)

Subclassed implementation of parse_args() which wraps parse_known_args().

parse_known_args(args=None, namespace=None)

Wrapper for parse_known_args() which handles intermediate levels of default settings derived from the user’s settings files. These override defaults defined in the parser itself. The precedence order is:

  1. Argument values explictly given by the user on the command line, as recorded in the is_default attribute of MDTFArgParser.

  2. Argument values from a file the user gave via the -f flag. (CLIConfigManager.defaults[DefaultsFileTypes.USER]).

  3. Argument values specified as the default values in the argument parser, which in turn are set with the following precedence order:

    1. Default values from a site-specfic file (defaults.jsonc), stored in CLIConfigManager.defaults[DefaultsFileTypes.SITE].

    2. Default values from a defaults.jsonc file in the /sites directory, stored in CLIConfigManager.defaults[DefaultsFileTypes.GLOBAL].

    3. Default values hard-coded in the CLI definition file itself.

Parameters
  • args (optional) – String or list of strings to parse. If a single string is passed, it’s split using split_args(). If not supplied, the default behavior parses sys.argv().

  • namespace (optional) – An object to store the parsed arguments. The default is a new empty argparse.Namespace object.

Returns

Tuple of 1) populated namespace containing parsed arguments and 2) unrecognized arguments, as with argparse.ArgumentParser.parse_known_args().

static split_args(argv)

Wrapper for shlex.split().

class src.cli.MDTFTopLevelArgParser(code_root, skip_defaults=False, argv=None)[source]

Bases: src.cli.MDTFArgParser

Class for constructing the command-line interface, parsing the options, and handing off execution to the selected subcommand.

__init__(code_root, skip_defaults=False, argv=None)[source]

Initialize self. See help(type(self)) for accurate signature.

iter_arg_groups(subcommand=None)[source]

Iterate over all arguments defined on the parser for the subcommand subcommand.

iter_group_actions(subcommand=None, group=None)[source]

Iterate over all arguments defined on the parser group group for the subcommand subcommand.

add_input_file_arg(target_p)[source]

Convenience method to add the flag to pass a user-designated defaults file to the parser target_p (either the top-level parser, or the preparser.)

init_user_defaults()[source]

Set user defaults using values read in from a configuration file in one of two formats. The format is determined from context: either

  1. A JSON/JSONC file of key-value pairs. This is parsed using parse_json().

  2. A plain text file containing flags and arguments as they would be passed on the command line (except shell expansions are not performed). This is parsed by the MDTFArgParser.parse_args() method of the configured parser.

The file’s path is determined from the argument to the -f flag via MDTFArgPreparser.parse_input_file().

Raises

ValueError – if the string cannot be parsed.

add_site_arg(target_p)[source]

Convenience method to add the argument flag to select which site-specific code to use, to the parser target_p (either the top-level parser, or a preparser.)

init_site()[source]

We allow site-specific installations to customize the CLI, so before we construct the CLI parser we need to determine what site to use. We do this by running a parser that only looks for the --site flag.

This sets the site attribute and populates the global and site-specific settings dicts.

add_contents(target_p)[source]

Convenience method to fully configure a parser target_p (either the top-level parser, or a preparser), adding subparsers for all subcommands.

setup()[source]

Method to wrap all configuration methods needed to configure the parser before calling parse_args(): reading the defaults files and configuring plugins based on existing values.

configure()[source]

Method that assembles the top-level CLI parser; called at the end of setup(). Options specific to the script are hard-coded here; CLI options for each subcommand are given in jsonc configuration files for each command which are read in by setup(). See documentation for parse_known_args() for information on the configuration file mechanism.

parse_args(args=None, namespace=None)[source]

Wrapper for parse_args() which handles intermediate levels of default settings. See documentation for parse_known_args().

parse_known_args(args=None, namespace=None)[source]

Wrapper for parse_known_args() which handles intermediate levels of default settings; see documentation for parse_known_args().

dispatch(args=None)[source]

Parse args, and call the subcommand that was selected.

iter_actions()

Iterator over Action objects associated with all user-defined arguments in parser, as well as those for any subcommands.

static split_args(argv)

Wrapper for shlex.split().

class src.cli.MDTFTopLevelSubcommandArgParser(code_root, skip_defaults=False, argv=None)[source]

Bases: src.cli.MDTFTopLevelArgParser

Extends MDTFTopLevelArgParser to add support for subcommands. Currently unused, intended for a future release.

__init__(code_root, skip_defaults=False, argv=None)

Initialize self. See help(type(self)) for accurate signature.

add_input_file_arg(target_p)

Convenience method to add the flag to pass a user-designated defaults file to the parser target_p (either the top-level parser, or the preparser.)

add_site_arg(target_p)

Convenience method to add the argument flag to select which site-specific code to use, to the parser target_p (either the top-level parser, or a preparser.)

init_site()

We allow site-specific installations to customize the CLI, so before we construct the CLI parser we need to determine what site to use. We do this by running a parser that only looks for the --site flag.

This sets the site attribute and populates the global and site-specific settings dicts.

init_user_defaults()

Set user defaults using values read in from a configuration file in one of two formats. The format is determined from context: either

  1. A JSON/JSONC file of key-value pairs. This is parsed using parse_json().

  2. A plain text file containing flags and arguments as they would be passed on the command line (except shell expansions are not performed). This is parsed by the MDTFArgParser.parse_args() method of the configured parser.

The file’s path is determined from the argument to the -f flag via MDTFArgPreparser.parse_input_file().

Raises

ValueError – if the string cannot be parsed.

iter_actions()

Iterator over Action objects associated with all user-defined arguments in parser, as well as those for any subcommands.

iter_arg_groups(subcommand=None)

Iterate over all arguments defined on the parser for the subcommand subcommand.

iter_group_actions(subcommand=None, group=None)

Iterate over all arguments defined on the parser group group for the subcommand subcommand.

parse_args(args=None, namespace=None)

Wrapper for parse_args() which handles intermediate levels of default settings. See documentation for parse_known_args().

parse_known_args(args=None, namespace=None)

Wrapper for parse_known_args() which handles intermediate levels of default settings; see documentation for parse_known_args().

setup()

Method to wrap all configuration methods needed to configure the parser before calling parse_args(): reading the defaults files and configuring plugins based on existing values.

static split_args(argv)

Wrapper for shlex.split().

add_contents(target_p)[source]

Convenience method to fully configure a parser target_p (either the top-level parser, or a preparser), adding subparsers for all subcommands.

configure()[source]

Method that assembles the top-level CLI parser. Options specific to the script are hard-coded here; CLI options for each subcommand are given in jsonc configuration files for each command which are read in here. See documentation for parse_known_args() for information on the configuration file mechanism.

dispatch()[source]

Parse args, and call the subcommand that was selected.