src.util.processes module

Utility functions for dealing with subprocesses.

class src.util.processes.ExceptionPropagatingThread(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)[source]

Bases: threading.Thread

Class to propagate exceptions raised in a child thread back to the caller thread when the child is join()ed. Adapted from https://stackoverflow.com/a/31614591.

run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

join(timeout=None)[source]

Wait until the thread terminates.

This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

property daemon

A boolean value indicating whether this thread is a daemon thread.

This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.

The entire Python program exits when only daemon threads are left.

property ident

Thread identifier of this thread or None if it has not been started.

This is a nonzero integer. See the get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.

property name

A string used for identification purposes only.

It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.

src.util.processes.poll_command(command, shell=False, env=None)[source]

Runs a command in a subprocess and prints stdout in real-time. Wraps Popen.

Parameters
  • command – list of command + arguments, or the same as a single string. See subprocess syntax. Note this interacts with the shell setting.

  • shell (bool) – Optional. Whether to run command in a shell. Default False.

  • env (dict) – Optional. Environment variables to set.

src.util.processes.run_command(command, env=None, cwd=None, timeout=0, dry_run=False, log=<Logger>)[source]

Subprocess wrapper to facilitate running a single command without starting a shell.

Note

We hope to save some process overhead by not running the command in a shell, but this means the command can’t use piping, quoting, environment variables, or filename globbing etc.

See documentation for Popen.

Parameters
  • command (list of str) – List of commands to execute.

  • env (dict) – Optional. Environment variables to set.

  • cwd (str) – Optional. Child processes’ working directory. Default is None, which uses the current working directory.

  • timeout (int) – Optionally, kill the command’s subprocess and raise a MDTFCalledProcessError if the command doesn’t finish in timeout seconds. Set to 0 to disable.

Returns

List of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact.

Raises

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in the output attribute of the exception.

src.util.processes.run_shell_command(command, env=None, cwd=None, dry_run=False, log=<Logger>)[source]

Subprocess wrapper to facilitate running shell commands. See documentation for Popen.

Parameters
  • commands (list of str) – List of commands to execute.

  • env (dict) – Optional. Environment variables to set.

  • cwd (str) – Optional. Child processes’ working directory. Default is None, which uses the current working directory.

Returns

List of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact, so if commands give != 1 lines of output this will not map to the list of commands given.

Raises

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in the output attribute of the exception.