src.util.datelabel module

Classes for serializing and deserializing dates and times expressed as strings in filenames and paths.

Intended use case is, e.g., determining if a file contains data for a given year based on the filename alone, without having to open it and parse the header.

Warning

Classes are implemented on top of the Python standard library datetime package, and as such always assume a proleptic Gregorian calendar. This is adequate for the intended filename-parsing use case.

Timezone support is not currently implemented, for the same reason.

Warning

These classes should not be used for detailed calendar math. We currently implement and test comparison logic only, not anything more (e.g. addition, subtraction, although increment/decrement are supported).

Properties and use of DateRange, Date and DateFrequency objects are best illustrated by examples:

>>> Date('20001215').month
12

>>> Date('200012') == datetime(2000, 12, 1)
True

>>> DateRange('2010-2020') in DateRange('2008-2019')
False

>>> DateRange('2010-2020').overlaps(DateRange('2008-2019'))
True

>>> DateFrequency('daily') < DateFrequency('24hr')
True

class src.util.datelabel.AtomicInterval(left, lower, upper, right)

Bases: object

This class represents an atomic interval. An atomic interval is a single interval, with a lower and upper bound, and two (closed or open) boundaries.

CLOSED = True

OPEN = False

__init__(left, lower, upper, right)

Create an atomic interval. If a bound is set to infinity (regardless of its sign), the corresponding boundary will be exclusive.

Parameters:

• left – Boolean indicating if left boundary is inclusive (True) or exclusive (False).

• lower – value of the lower bound.

• upper – value of the upper bound.

• right – Boolean indicating if right boundary is inclusive (True) or exclusive (False).

property left

Boolean indicating whether the left boundary is inclusive (True) or exclusive (False).

property lower

Lower bound value.

property upper

Upper bound value.

property right

Boolean indicating whether the right boundary is inclusive (True) or exclusive (False).

is_empty()

Test interval emptiness.

Returns:

True if interval is empty, False otherwise.

replace(left=None, lower=None, upper=None, right=None, ignore_inf=True)

Create a new interval based on the current one and the provided values. Callable can be passed instead of values. In that case, it is called with the current corresponding value except if ignore_inf if set (default) and the corresponding bound is an infinity.

Parameters:

• left – (a function of) left boundary.

• lower – (a function of) value of the lower bound.

• upper – (a function of) value of the upper bound.

• right – (a function of) right boundary.

• ignore_inf – ignore infinities if functions are provided (default is True).

Returns:

An Interval instance.

overlaps(other, adjacent=False)

Test if intervals have any overlapping value. If ‘adjacent’ is set to True (default is False), then it returns True for adjacent intervals as well (e.g., [1, 2) and [2, 3], but not [1, 2) and (2, 3]).

Parameters:

• other – an atomic interval.

• adjacent – set to True to accept adjacent intervals as well.

Returns:

True if intervals overlap, False otherwise.

intersection(other)

Return the intersection of two intervals.

Parameters:

other – an interval.

Returns:

The intersection of the intervals.

union(other)

Return the union of two intervals. If the union cannot be represented using a single atomic interval, return an Interval instance (which corresponds to an union of atomic intervals).

Parameters:

other – an interval.

Returns:

The union of the intervals.

contains(item)

Test if given item is contained in this interval. This method accepts atomic intervals, intervals and arbitrary values.

Parameters:

item – an atomic interval, an interval or any arbitrary value.

Returns:

True if given item is contained, False otherwise.

adjoins_left(other)

Returns True if other follows self with no gap or overlap.

adjoins_right(other)

Returns True if self follows other with no gap or overlap.

adjoins(other)

Returns True if there is no gap or overlap between self and other.

classmethod span(*args)

Return an AtomicInterval covering the collection of intervals in args.

classmethod contiguous_span(*args)

Return an AtomicInterval covering the collection of intervals in args if those intervals are contiguous and nonoverlapping.

Raises:

ValueError – If collection of intervals is not contiguous and nonoverlapping.

class src.util.datelabel.DatePrecision(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: IntEnum

IntEnum to encode the recognized levels of precision for Dates and DateRanges. Example:

>>> Date('200012').precision == DatePrecision.MONTH
True

because the date in the example, “December 2000,” is only defined up to a month, and hence is represented by the interval from 1 Dec 2000 to 31 Dec 2000.

STATIC = -1

YEAR = 1

MONTH = 2

DAY = 3

HOUR = 4

MINUTE = 5

SECOND = 6

MICROSECOND = 7

conjugate()

Returns self, the complex conjugate of any int.

bit_length()

Number of bits necessary to represent self in binary.

>>> bin(37)
'0b100101'
>>> (37).bit_length()
6

bit_count()

Number of ones in the binary representation of the absolute value of self.

Also known as the population count.

>>> bin(13)
'0b1101'
>>> (13).bit_count()
3

to_bytes(length=1, byteorder='big', *, signed=False)

Return an array of bytes representing an integer.

length

Length of bytes object to use. An OverflowError is raised if the integer is not representable with the given number of bytes. Default is length 1.

byteorder

The byte order used to represent the integer. If byteorder is ‘big’, the most significant byte is at the beginning of the byte array. If byteorder is ‘little’, the most significant byte is at the end of the byte array. To request the native byte order of the host system, use `sys.byteorder’ as the byte order value. Default is to use ‘big’.

signed

Determines whether two’s complement is used to represent the integer. If signed is False and a negative integer is given, an OverflowError is raised.

from_bytes(byteorder='big', *, signed=False)

Return the integer represented by the given array of bytes.

bytes

Holds the array of bytes to convert. The argument must either support the buffer protocol or be an iterable object producing bytes. Bytes and bytearray are examples of built-in objects that support the buffer protocol.

byteorder

The byte order used to represent the integer. If byteorder is ‘big’, the most significant byte is at the beginning of the byte array. If byteorder is ‘little’, the most significant byte is at the end of the byte array. To request the native byte order of the host system, use `sys.byteorder’ as the byte order value. Default is to use ‘big’.

signed

Indicates whether two’s complement is used to represent the integer.

as_integer_ratio()

Return integer ratio.

Return a pair of integers, whose ratio is exactly equal to the original int and with a positive denominator.

>>> (10).as_integer_ratio()
(10, 1)
>>> (-10).as_integer_ratio()
(-10, 1)
>>> (0).as_integer_ratio()
(0, 1)

real

the real part of a complex number

imag

the imaginary part of a complex number

numerator

the numerator of a rational number in lowest terms

denominator

the denominator of a rational number in lowest terms

class src.util.datelabel.DateMixin

Bases: object

Utility methods for dealing with dates.

static date_format(dt, precision=None)

Print date dt in YYYYMMDDHHMMSS format, with length being set automatically from precision.

Note

strftime() is broken for dates prior to 1900 in python < 3.3, see https://bugs.python.org/issue1777412 and https://stackoverflow.com/q/10263956. For this reason, the workaround implemented here should be used instead.

classmethod increment(dt, precision)

Return a copy of dt advanced by one time unit as specified by the precision attribute.

classmethod decrement(dt, precision)

Return a copy of dt moved back by one time unit as specified by the precision attribute.

class src.util.datelabel.DateRange(start, end=None, precision=None, log=<Logger>)

Bases: AtomicInterval, DateMixin

Class representing a range of dates specified with variable precision.

Endpoints of the interval are represented internally as datetime objects.

Note

In keeping with convention, this is always defined as a closed interval (containing both endpoints). E.g., DateRange(‘1990-1999’) starts at 0:00 on 1 Jan 1990 and ends at 23:59 on 31 Dec 1999, inclusive.

precision

Precision to which both endpoints of the DateRange are known. E.g., DateRange(‘1990-1999’) has a precision of DatePrecision.YEAR.

Type:

:class:`DatePrecision`

property is_static

Property indicating time-independent data (e.g., fx in the CMIP6 DRS.)

property start_datetime

Start of the interval, returned as a datetime object.

property start

Start of the interval, returned as a Date object of appropriate precision.

property end_datetime

End of the interval, returned as a datetime object.

property end

End of the interval, returned as a Date object of appropriate precision.

classmethod from_contiguous_span(*args)

Given multiple DateRanges, return interval containing them only if their time intervals are contiguous and non-overlapping.

classmethod from_date_span(*args)

Return a DateRange coresponding to the interval containing a set of Dates. Differs from from_contiguous_span() in that we don’t expect intervals to be contiguous.

format(precision=None)

Return string representation of this DateRange, of the form YYYYMMDD…-YYYYMMDD…. The length of the YYYYMMDD… representation is determined by the precision attribute if not manually given.

contains(item)

Override AtomicInterval.__contains__() to handle differences in datelabel precision. Finite precision means that the interval endpoints are ranges, not points (which is why Date inherits from DateRange and not vice-versa). We replace strict equality of endpoints (==) with appropriate conditions on the overlap of these ranges.

overlaps(item)

Test if intervals have any overlapping value. If ‘adjacent’ is set to True (default is False), then it returns True for adjacent intervals as well (e.g., [1, 2) and [2, 3], but not [1, 2) and (2, 3]).

Parameters:

• other – an atomic interval.

• adjacent – set to True to accept adjacent intervals as well.

Returns:

True if intervals overlap, False otherwise.

intersection(item, precision=None)

Return the intersection of two intervals.

Parameters:

other – an interval.

Returns:

The intersection of the intervals.

CLOSED = True

OPEN = False

adjoins(other)

Returns True if there is no gap or overlap between self and other.

adjoins_left(other)

Returns True if other follows self with no gap or overlap.

adjoins_right(other)

Returns True if self follows other with no gap or overlap.

classmethod contiguous_span(*args)

Return an AtomicInterval covering the collection of intervals in args if those intervals are contiguous and nonoverlapping.

Raises:

ValueError – If collection of intervals is not contiguous and nonoverlapping.

static date_format(dt, precision=None)

Print date dt in YYYYMMDDHHMMSS format, with length being set automatically from precision.

Note

strftime() is broken for dates prior to 1900 in python < 3.3, see https://bugs.python.org/issue1777412 and https://stackoverflow.com/q/10263956. For this reason, the workaround implemented here should be used instead.

classmethod decrement(dt, precision)

Return a copy of dt moved back by one time unit as specified by the precision attribute.

classmethod increment(dt, precision)

Return a copy of dt advanced by one time unit as specified by the precision attribute.

is_empty()

Test interval emptiness.

Returns:

True if interval is empty, False otherwise.

property left

Boolean indicating whether the left boundary is inclusive (True) or exclusive (False).

property lower

Lower bound value.

replace(left=None, lower=None, upper=None, right=None, ignore_inf=True)

Create a new interval based on the current one and the provided values. Callable can be passed instead of values. In that case, it is called with the current corresponding value except if ignore_inf if set (default) and the corresponding bound is an infinity.

Parameters:

• left – (a function of) left boundary.

• lower – (a function of) value of the lower bound.

• upper – (a function of) value of the upper bound.

• right – (a function of) right boundary.

• ignore_inf – ignore infinities if functions are provided (default is True).

Returns:

An Interval instance.

property right

Boolean indicating whether the right boundary is inclusive (True) or exclusive (False).

classmethod span(*args)

Return an AtomicInterval covering the collection of intervals in args.

union(other)

Return the union of two intervals. If the union cannot be represented using a single atomic interval, return an Interval instance (which corresponds to an union of atomic intervals).

Parameters:

other – an interval.

Returns:

The union of the intervals.

property upper

Upper bound value.

class src.util.datelabel.Date(*args, **kwargs)

Bases: DateRange

Defines a single date with variable level precision.

The date is represented as an interval, with precision setting the length of the interval (which is why this inherits from DateRange and not vice versa.)

Date objects are mapped to datetimes representing the start of the interval implied by their precision, e.g. Date(‘2000-05’) maps to 0:00 on 1 May 2000.

year, month, day, hour, minute, second

Components of the datetime representing the start of the interval defined by Date. We do not check that the attribute access is appropriate to the Date’s precision.

precision

Precision to which both endpoints of the DateRange are known. E.g., Date(1990) has a precision of DatePrecision.YEAR.

Type:

:class:`DatePrecision`

format(precision=None)

Return YYYYMMDD… string representation of this Date. The length of the representation is determined by the precision attribute if not manually given.

isoformat()

Return string representation of the start datetime of this Date interval in YYYY-MM-DD HH:MM:SS format.

CLOSED = True

OPEN = False

adjoins(other)

Returns True if there is no gap or overlap between self and other.

adjoins_left(other)

Returns True if other follows self with no gap or overlap.

adjoins_right(other)

Returns True if self follows other with no gap or overlap.

contains(item)

Override AtomicInterval.__contains__() to handle differences in datelabel precision. Finite precision means that the interval endpoints are ranges, not points (which is why Date inherits from DateRange and not vice-versa). We replace strict equality of endpoints (==) with appropriate conditions on the overlap of these ranges.

classmethod contiguous_span(*args)

Return an AtomicInterval covering the collection of intervals in args if those intervals are contiguous and nonoverlapping.

Raises:

ValueError – If collection of intervals is not contiguous and nonoverlapping.

static date_format(dt, precision=None)

Print date dt in YYYYMMDDHHMMSS format, with length being set automatically from precision.

Note

strftime() is broken for dates prior to 1900 in python < 3.3, see https://bugs.python.org/issue1777412 and https://stackoverflow.com/q/10263956. For this reason, the workaround implemented here should be used instead.

classmethod decrement(dt, precision)

Return a copy of dt moved back by one time unit as specified by the precision attribute.

property end

End of the interval, returned as a Date object of appropriate precision.

property end_datetime

End of the interval, returned as a datetime object.

classmethod from_contiguous_span(*args)

Given multiple DateRanges, return interval containing them only if their time intervals are contiguous and non-overlapping.

classmethod from_date_span(*args)

Return a DateRange coresponding to the interval containing a set of Dates. Differs from from_contiguous_span() in that we don’t expect intervals to be contiguous.

classmethod increment(dt, precision)

Return a copy of dt advanced by one time unit as specified by the precision attribute.

intersection(item, precision=None)

Return the intersection of two intervals.

Parameters:

other – an interval.

Returns:

The intersection of the intervals.

is_empty()

Test interval emptiness.

Returns:

True if interval is empty, False otherwise.

property is_static

Property indicating time-independent data (e.g., fx in the CMIP6 DRS.)

property left

Boolean indicating whether the left boundary is inclusive (True) or exclusive (False).

property lower

Lower bound value.

overlaps(item)

Test if intervals have any overlapping value. If ‘adjacent’ is set to True (default is False), then it returns True for adjacent intervals as well (e.g., [1, 2) and [2, 3], but not [1, 2) and (2, 3]).

Parameters:

• other – an atomic interval.

• adjacent – set to True to accept adjacent intervals as well.

Returns:

True if intervals overlap, False otherwise.

replace(left=None, lower=None, upper=None, right=None, ignore_inf=True)

Create a new interval based on the current one and the provided values. Callable can be passed instead of values. In that case, it is called with the current corresponding value except if ignore_inf if set (default) and the corresponding bound is an infinity.

Parameters:

• left – (a function of) left boundary.

• lower – (a function of) value of the lower bound.

• upper – (a function of) value of the upper bound.

• right – (a function of) right boundary.

• ignore_inf – ignore infinities if functions are provided (default is True).

Returns:

An Interval instance.

property right

Boolean indicating whether the right boundary is inclusive (True) or exclusive (False).

classmethod span(*args)

Return an AtomicInterval covering the collection of intervals in args.

property start

Start of the interval, returned as a Date object of appropriate precision.

property start_datetime

Start of the interval, returned as a datetime object.

union(other)

Return the union of two intervals. If the union cannot be represented using a single atomic interval, return an Interval instance (which corresponds to an union of atomic intervals).

Parameters:

other – an interval.

Returns:

The union of the intervals.

property upper

Upper bound value.

src.util.datelabel.FXDateRange = _FXDateRange()

Singleton placeholder/sentinel object for use in describing static data with no time dependence.

class src.util.datelabel.DateFrequency(quantity, unit=None)

Bases: timedelta

Class representing a frequency or time period.

Warning

Period lengths are not defined accurately, eg. a year is taken as 365 days and a month is taken as 30 days. For this reason, we do not implement addition and subtraction of DateFrequency objects to Dates, as is possible for timedelta and datetime.

property is_static

Property indicating time-independent data (e.g., fx in CMIP6 DRS.)

classmethod from_struct(str_)

Object instantiation method used by src.util.dataclass.mdtf_dataclass() for type coercion.

format()

Return string representation of the DateFrequency.

format_local()

String representation as used in framework’s local directory hierarchy (defined in src.data_manager.DataManager.dest_path().)

max = datetime.timedelta(days=999999999, seconds=86399, microseconds=999999)

min = datetime.timedelta(days=-999999999)

resolution = datetime.timedelta(microseconds=1)

src.util.datelabel.FXDateFrequency = _FXDateFrequency('fx')

Singleton placeholder/sentinel object for use in describing static data with no time dependence.

class src.util.datelabel.AbstractDateRange

Bases: ABC

Defines interface (set of attributes) for DateRange objects.

class src.util.datelabel.AbstractDate

Bases: ABC

Defines interface (set of attributes) for Date objects.

class src.util.datelabel.AbstractDateFrequency

Bases: ABC

Defines interface (set of attributes) for DateFrequency objects.