src.util.filesystem module

Utility functions for interacting with the local filesystem and configuration files.

src.util.filesystem.abbreviate_path(path: str, old_base: str, new_base=None) → str

Express path as a path relative to old_base, optionally prepending new_base.

src.util.filesystem.resolve_path(rel_path: str, root_path: str = '', env_vars: dict = None, log=<Logger>) → str

Abbreviation to resolve relative paths, expanding environment variables if necessary.

Parameters:

• log – logger object

• rel_path (str) – Path to resolve.

• root_path (str) – Optional. Root path to resolve path with. If not given, resolves relative to os.getcwd().

• env_vars (dict) – global environment variables

Returns:

str – Absolute version of path.

src.util.filesystem.recursive_copy(src_files, src_root: str, dest_root: str, copy_function=None, overwrite: bool = False)

Copy src_files to dest_root, preserving relative subdirectory structure.

Copies a subset of files in a directory subtree rooted at src_root to an identical subtree structure rooted at dest_root, creating any subdirectories as needed. For example, recursive_copy('/A/B/C.txt', '/A', '/D') will first create the destination subdirectory /D/B and copy /A/B/C.txt to /D/B/C.txt.

Parameters:

• src_files (str or iterable) – Absolute path, or list of absolute paths, to files to copy.

• src_root (str) – Root subtree of all files in src_files.

• dest_root (str) – Destination directory in which to create the copied subtree.

• copy_function (function) – Function to use to copy individual files. Must take two arguments, the source and destination paths, respectively. Defaults to shutil.copy2().

• overwrite (bool) – Optional, default False. Determines whether to raise error if files would be overwritten.

Raises:

• ValueError – If all files in src_files are not contained in the src_root directory.

• OSError – If overwrite is False, raise if any destination files already exist, otherwise silently overwrite.

src.util.filesystem.check_executable(exec_name: str) → bool

Tests if the executable exec_name is found on the current $PATH.

Parameters:

exec_name (:py:obj:`str`) – Name of the executable to search for.

src.util.filesystem.find_files(src_dirs: str | list, filename_globs: str | list, n_files=None) → list

Return list of files in src_dirs, or any subdirectories, matching any of filename_globs. Wraps Python glob.glob.

Parameters:

• src_dirs – Directory, or a list of directories, to search for files in. The function will also search all subdirectories.

• filename_globs – Glob, or a list of globs, for filenames to match. This is a shell globbing pattern, not a full regex.

• n_files (int) – Optional. Number of files expected to be found.

Raises:

MDTFFileNotFoundError – If n_files is supplied and the number of files found is different than this number.

Returns:

List of paths to files matching any of the criteria. If no files are found, the list is empty.

src.util.filesystem.check_dir(dir_path: str, attr_name: str = '', create: bool = False)

Check existence of directories. No action is taken for directories that already exist; nonexistent directories either raise a MDTFFileNotFoundError or cause the creation of that directory.

Parameters:

• dir_path – If a string, the absolute path to check; otherwise, assume the path to check is given by the attr_name attribute on this object.

• attr_name – Name of the attribute being checked (used in log messages).

• create – (bool, default False): if True, nonexistent directories are created.

src.util.filesystem.bump_version(path: str, new_v=None, extra_dirs=None)

Append a version number to path, if necessary, so that it doesn’t conflict with existing files.

Parameters:

• path (str) – Path to test and append version number to.

• new_v (int) – Optional. Version number to begin incrementing at.

• extra_dirs (str or iterable) – Optional. If supplied, increment the version number of path so that it doesn’t conflict with pre-existing files at these locations either.

Returns:

str – path with a version number appended to it, if path exists. For files, the version number is appended before the extension. For example, repeated application would create a series of files file.txt, file.v1.txt, file.v2.txt, …

src.util.filesystem.append_html_template(template_file: str, target_file: str, template_dict: dict = {}, create: bool = True, append: bool = True)

Perform substitutions on template_file and write result to target_file.

Variable substitutions are done with custom templating, replacing double curly bracket-delimited keys with their values in template_dict. For example, if template_dict is {'A': 'foo'}, all occurrences of the string {{A}} in template_file are replaced with the string foo. Spaces between the braces and variable names are ignored.

Double-curly-bracketed strings that don’t correspond to keys in template_dict are ignored (instead of raising a KeyError.)

Double curly brackets are chosen as the delimiter to match the default syntax of, e.g., jinja2. Using single curly braces would lead to conflicts with CSS syntax.

Parameters:

• template_file (str) – Path to template file.

• target_file (str) – Destination path for result.

• template_dict (dict) – Template name-value pairs. Both names and values must be strings.

• create (bool) – Optional, default True. If True, create target_file if it doesn’t exist, otherwise raise an OSError.

• append (bool) – Optional, default True. If target_file exists and this is True, append the substituted contents of template_file to it. If False, overwrite target_file with the substituted contents of template_file.

class src.util.filesystem.TempDirManager(config)

Bases: object

temp_root: str = ''

keep_temp: bool = False

make_tempdir(hash_obj=None)

rm_tempdir(path: str)

cleanup()

tempdir_cleanup_handler(signum=None)