src.util.processes module

Utility functions for dealing with subprocesses.

class src.util.processes.ExceptionPropagatingThread(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)

Bases: Thread

Class to propagate exceptions raised in a child thread back to the caller thread when the child is join()ed. Adapted from https://stackoverflow.com/a/31614591.

ret = None

exc = None

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

join(timeout=None)

Wait until the thread terminates.

This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

__init__(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is a list or tuple of arguments for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

property daemon

A boolean value indicating whether this thread is a daemon thread.

This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.

The entire Python program exits when only daemon threads are left.

property ident

Thread identifier of this thread or None if it has not been started.

This is a nonzero integer. See the get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.

property name

A string used for identification purposes only.

It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.

property native_id

Native integral thread ID of this thread, or None if it has not been started.

This is a non-negative integer. See the get_native_id() function. This represents the Thread ID as reported by the kernel.

src.util.processes.poll_command(command, shell=False, env=None)

Runs a command in a subprocess and prints stdout in real-time. Wraps Popen.

Parameters:

• command – list of command + arguments, or the same as a single string. See subprocess syntax. Note this interacts with the shell setting.

• shell (bool) – Optional. Whether to run command in a shell. Default False.

• env (dict) – Optional. Environment variables to set.

src.util.processes.run_command(command, env=None, cwd=None, timeout=0, log=<Logger>)

Subprocess wrapper to facilitate running a single command without starting a shell.

Note

We hope to save some process overhead by not running the command in a shell, but this means the command can’t use piping, quoting, environment variables, or filename globbing etc.

See documentation for Popen.

Parameters:

• command (list of str) – List of commands to execute.

• env (dict) – Optional. Environment variables to set.

• cwd (str) – Optional. Child processes’ working directory. Default is None, which uses the current working directory.

• timeout (int) – Optionally, kill the command’s subprocess and raise a MDTFCalledProcessError if the command doesn’t finish in timeout seconds. Set to 0 to disable.

Returns:

List of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact.

Raises:

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in the output attribute of the exception.

src.util.processes.run_shell_command(command, env=None, cwd=None, log=<Logger>)

Subprocess wrapper to facilitate running shell commands. See documentation for Popen.

Parameters:

• command (list of str) – List of commands to execute.

• env (dict) – Optional. Environment variables to set.

• cwd (str) – Optional. Child processes’ working directory. Default is None, which uses the current working directory.

• log – log file

Returns:

List of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact, so if commands give != 1 lines of output this will not map to the list of commands given.

Raises:

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in the output attribute of the exception.