Skip to content

Dataset Validation References

A check tests one rule on one dataset and returns a CheckResult. A DatasetValidator runs a set of checks and returns a ValidationReport.

from mesqual.validation import ConstraintCheck, DatasetValidator

validator = DatasetValidator([
    ConstraintCheck('Line.Constraints.capacity_up', min_value=0),
    ConstraintCheck('Generator.Results.generation', min_value=0, isna_ok=False),
])
report = validator.run(dataset)
report.failed                      # the FAIL results
report.to_markdown('validation.md')

validation

Checks on datasets that return results, and a validator that collects them in a report.

DatasetCheck

Bases: ABC

Base class of a check: a rule on one dataset, run by run(dataset) -> CheckResult.

A subclass implements _run. It declares the flags it reads in FLAGS, or overrides flags when they depend on the instance. Settings, for example tolerances, are constructor arguments. A subclass can be a dataclass. A subclass keeps no state between runs: run gives the same result for the same dataset and settings.

run returns SKIP when a declared flag is accepted but not available on the dataset (Dataset.availability). A flag with unknown availability counts as available: only a fetch tells. A FlagUnavailableError raised in _run also gives SKIP. A declared flag that the dataset does not accept raises UnknownFlagError, so a typo in a flag name is an ERROR in the validator, not a SKIP. A subclass with more needs, for example a dataset class or a model input, overrides the hook _unmet(dataset) and calls super()._unmet(dataset).

Examples:

>>> from mesqual.validation import CheckStatus, Comparison
>>> class PricesNotNegative(DatasetCheck):
...     FLAGS = ('Node.Results.price',)
...
...     def _run(self, dataset):
...         prices = Comparison.long(dataset.fetch('Node.Results.price'))
...         negative = prices[prices < 0].rename('price').reset_index()
...         if len(negative):
...             return self.result(CheckStatus.FAIL, f'{len(negative)} negative node prices.', negative)
...         return self.result(CheckStatus.PASS, f'{len(prices)} node prices, none negative.')
>>> result = PricesNotNegative().run(dataset)
>>> result.status.name, result.summary
('PASS', '1328 node prices, none negative.')

Attributes:

Name Type Description
FLAGS tuple[FlagType, ...]

The flags the check reads.

Source code in src/mesqual/validation/check.py
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
class DatasetCheck(ABC):
    """Base class of a check: a rule on one dataset, run by `run(dataset) -> CheckResult`.

    A subclass implements `_run`. It declares the flags it reads in `FLAGS`, or
    overrides `flags` when they depend on the instance. Settings, for example
    tolerances, are constructor arguments. A subclass can be a dataclass.
    A subclass keeps no state between runs: `run` gives the same result for
    the same dataset and settings.

    `run` returns `SKIP` when a declared flag is accepted but not available on
    the dataset (`Dataset.availability`). A flag with unknown availability
    counts as available: only a fetch tells. A `FlagUnavailableError` raised in
    `_run` also gives `SKIP`. A declared flag that the dataset does not accept
    raises `UnknownFlagError`, so a typo in a flag name is an `ERROR` in the
    validator, not a `SKIP`. A subclass with more needs, for example a dataset
    class or a model input, overrides the hook `_unmet(dataset)` and calls
    `super()._unmet(dataset)`.

    Examples:
        >>> from mesqual.validation import CheckStatus, Comparison
        >>> class PricesNotNegative(DatasetCheck):
        ...     FLAGS = ('Node.Results.price',)
        ...
        ...     def _run(self, dataset):
        ...         prices = Comparison.long(dataset.fetch('Node.Results.price'))
        ...         negative = prices[prices < 0].rename('price').reset_index()
        ...         if len(negative):
        ...             return self.result(CheckStatus.FAIL, f'{len(negative)} negative node prices.', negative)
        ...         return self.result(CheckStatus.PASS, f'{len(prices)} node prices, none negative.')
        >>> result = PricesNotNegative().run(dataset)
        >>> result.status.name, result.summary
        ('PASS', '1328 node prices, none negative.')

    Attributes:
        FLAGS: The flags the check reads.
    """

    FLAGS: ClassVar[tuple[FlagType, ...]] = ()

    @property
    def name(self) -> str:
        """The check name: the class name. A validator holds one check per name."""
        return type(self).__name__

    @property
    def flags(self) -> tuple[FlagType, ...]:
        """The flags the check reads: `FLAGS`."""
        return tuple(self.FLAGS)

    def run(self, dataset: Dataset) -> CheckResult:
        """Run the check on a dataset.

        Returns:
            The result. `SKIP` if a need is not met (`_unmet`) or the check
                fetches an unavailable flag.

        Raises:
            UnknownFlagError: The dataset does not accept a flag of `flags`.
        """
        try:
            return self._unmet(dataset) or self._run(dataset)
        except FlagUnavailableError as error:
            return self.skip(str(error))

    def _unmet(self, dataset: Dataset) -> CheckResult | None:
        """Hook: a `SKIP` result if the dataset does not meet a need of the check, else `None`.

        Default: `SKIP` if a flag of `flags` is not available. The details list
        each such flag with the reason. A subclass with more needs overrides it
        and calls `super()._unmet(dataset)`.

        Raises:
            UnknownFlagError: The dataset does not accept a flag of `flags`.
        """
        self._check_accepted(dataset, self.flags)
        missing = {flag: dataset.availability(flag) for flag in self.flags}
        missing = {flag: a for flag, a in missing.items() if not self._counts_as_available(a)}
        if not missing:
            return None
        details = pd.DataFrame({'flag': list(missing), 'reason': [a.reason for a in missing.values()]})
        return self.skip(f'flags not available: {", ".join(map(str, missing))}.', details)

    @abstractmethod
    def _run(self, dataset: Dataset) -> CheckResult:
        """Hook: test the rule on a dataset that meets the needs of the check."""

    def result(self, status: CheckStatus, summary: str, details: pd.DataFrame | None = None) -> CheckResult:
        """Return a result of this check."""
        return CheckResult(self.name, status, summary, pd.DataFrame() if details is None else details)

    def skip(self, reason: str, details: pd.DataFrame | None = None) -> CheckResult:
        """Return a `SKIP` result with the reason."""
        return self.result(CheckStatus.SKIP, reason, details)

    def compared(self, comparison: Comparison, what: str) -> CheckResult:
        """Return `PASS` or `FAIL` for a comparison, `SKIP` when it compared nothing.

        The summary is `'<violations> of <compared> <what>.'`.

        Args:
            comparison: The comparison.
            what: The noun phrase of a violation, for example `'line MTUs off'`.
        """
        if not comparison.compared:
            return self.skip('nothing to compare.')
        violations = comparison.violations
        status = CheckStatus.FAIL if len(violations) else CheckStatus.PASS
        return self.result(status, f'{len(violations)} of {comparison.compared} {what}.', violations)

    @classmethod
    def has_flags(cls, dataset: Dataset, flags: Collection[FlagType]) -> bool:
        """Return `True` if every flag counts as available on the dataset (see the class docstring).

        Raises:
            UnknownFlagError: The dataset does not accept a flag.
        """
        cls._check_accepted(dataset, flags)
        return all(cls._counts_as_available(dataset.availability(flag)) for flag in flags)

    @staticmethod
    def _check_accepted(dataset: Dataset, flags: Collection[FlagType]) -> None:
        unknown = [flag for flag in flags if not dataset.flag_is_accepted(flag)]
        if unknown:
            raise UnknownFlagError(f'{dataset.name} does not accept the flags {", ".join(map(str, unknown))}.')

    @staticmethod
    def _counts_as_available(availability: Availability) -> bool:
        return availability.available is not False

name property

name: str

The check name: the class name. A validator holds one check per name.

flags property

flags: tuple[FlagType, ...]

The flags the check reads: FLAGS.

run

run(dataset: Dataset) -> CheckResult

Run the check on a dataset.

Returns:

Type Description
CheckResult

The result. SKIP if a need is not met (_unmet) or the check fetches an unavailable flag.

Raises:

Type Description
UnknownFlagError

The dataset does not accept a flag of flags.

Source code in src/mesqual/validation/check.py
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def run(self, dataset: Dataset) -> CheckResult:
    """Run the check on a dataset.

    Returns:
        The result. `SKIP` if a need is not met (`_unmet`) or the check
            fetches an unavailable flag.

    Raises:
        UnknownFlagError: The dataset does not accept a flag of `flags`.
    """
    try:
        return self._unmet(dataset) or self._run(dataset)
    except FlagUnavailableError as error:
        return self.skip(str(error))

result

result(status: CheckStatus, summary: str, details: DataFrame | None = None) -> CheckResult

Return a result of this check.

Source code in src/mesqual/validation/check.py
104
105
106
def result(self, status: CheckStatus, summary: str, details: pd.DataFrame | None = None) -> CheckResult:
    """Return a result of this check."""
    return CheckResult(self.name, status, summary, pd.DataFrame() if details is None else details)

skip

skip(reason: str, details: DataFrame | None = None) -> CheckResult

Return a SKIP result with the reason.

Source code in src/mesqual/validation/check.py
108
109
110
def skip(self, reason: str, details: pd.DataFrame | None = None) -> CheckResult:
    """Return a `SKIP` result with the reason."""
    return self.result(CheckStatus.SKIP, reason, details)

compared

compared(comparison: Comparison, what: str) -> CheckResult

Return PASS or FAIL for a comparison, SKIP when it compared nothing.

The summary is '<violations> of <compared> <what>.'.

Parameters:

Name Type Description Default
comparison Comparison

The comparison.

required
what str

The noun phrase of a violation, for example 'line MTUs off'.

required
Source code in src/mesqual/validation/check.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
def compared(self, comparison: Comparison, what: str) -> CheckResult:
    """Return `PASS` or `FAIL` for a comparison, `SKIP` when it compared nothing.

    The summary is `'<violations> of <compared> <what>.'`.

    Args:
        comparison: The comparison.
        what: The noun phrase of a violation, for example `'line MTUs off'`.
    """
    if not comparison.compared:
        return self.skip('nothing to compare.')
    violations = comparison.violations
    status = CheckStatus.FAIL if len(violations) else CheckStatus.PASS
    return self.result(status, f'{len(violations)} of {comparison.compared} {what}.', violations)

has_flags classmethod

has_flags(dataset: Dataset, flags: Collection[FlagType]) -> bool

Return True if every flag counts as available on the dataset (see the class docstring).

Raises:

Type Description
UnknownFlagError

The dataset does not accept a flag.

Source code in src/mesqual/validation/check.py
127
128
129
130
131
132
133
134
135
@classmethod
def has_flags(cls, dataset: Dataset, flags: Collection[FlagType]) -> bool:
    """Return `True` if every flag counts as available on the dataset (see the class docstring).

    Raises:
        UnknownFlagError: The dataset does not accept a flag.
    """
    cls._check_accepted(dataset, flags)
    return all(cls._counts_as_available(dataset.availability(flag)) for flag in flags)

Comparison dataclass

Values of two frames compared where both have a value.

Attributes:

Name Type Description
compared int

Number of compared values.

violations DataFrame

One row per value off by more than the tolerance: the index levels and column levels of the frames as columns (see long), then actual, expected and difference.

Examples:

>>> actual = pd.DataFrame({'N1': [10.0, 20.0], 'N2': [30.0, 41.0]})
>>> expected = pd.DataFrame({'N1': [10.0, 20.0], 'N2': [30.0, 40.0]})
>>> comparison = Comparison.of(actual, expected, tol=0.5)
>>> comparison.compared, len(comparison.violations)
(4, 1)
Source code in src/mesqual/validation/comparison.py
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
@dataclass(frozen=True, eq=False)
class Comparison:
    """Values of two frames compared where both have a value.

    Attributes:
        compared: Number of compared values.
        violations: One row per value off by more than the tolerance: the index
            levels and column levels of the frames as columns (see `long`), then
            `actual`, `expected` and `difference`.

    Examples:
        >>> actual = pd.DataFrame({'N1': [10.0, 20.0], 'N2': [30.0, 41.0]})
        >>> expected = pd.DataFrame({'N1': [10.0, 20.0], 'N2': [30.0, 40.0]})
        >>> comparison = Comparison.of(actual, expected, tol=0.5)
        >>> comparison.compared, len(comparison.violations)
        (4, 1)
    """

    compared: int
    violations: pd.DataFrame

    _VALUE_COLUMNS = ('actual', 'expected', 'difference')

    @classmethod
    def of(
        cls,
        actual: pd.DataFrame | pd.Series,
        expected: pd.DataFrame | pd.Series,
        tol: float = 0.0,
        rel_tol: float = 0.0,
        empty_is_off: bool = False,
    ) -> Comparison:
        """Compare `actual` with `expected` on the index and columns they share.

        A value fits when `|actual - expected| <= tol + rel_tol * max(|actual|, |expected|)`,
        as in `math.isclose`. A cell with NaN on either side is not compared,
        unless `empty_is_off`: then an empty actual cell with an expected value
        is a violation.

        The index levels and column levels of both sides are matched by name
        when all levels on both sides have unique names; `expected` takes the
        level order of `actual`. Else they are matched by position.

        Args:
            actual: The values to test.
            expected: The values they must equal.
            tol: Absolute tolerance.
            rel_tol: Relative tolerance, a share of the larger absolute value.
            empty_is_off: Count an empty actual cell with an expected value as a violation.

        Raises:
            ValueError: The level names of both sides differ (all levels named),
                or the number of levels differs (matched by position); a level
                is named `actual`, `expected` or `difference`; a side has a
                duplicate index entry.
        """
        index_levels = actual.index.nlevels
        actual, expected = cls._stacked(actual), cls._stacked(expected)
        expected = cls._aligned(actual, expected)
        actual = cls._named(actual, index_levels)
        expected = expected.set_axis(expected.index.set_names(actual.index.names))
        cls._check_levels(actual, 'actual')
        cls._check_levels(expected, 'expected')
        both = pd.concat({'actual': actual, 'expected': expected}, axis=1, join='inner')
        both = both[both['expected'].notna()] if empty_is_off else both.dropna()
        difference = both['actual'] - both['expected']
        scale = both[['actual', 'expected']].abs().max(axis=1)
        off = (difference.abs() > tol + rel_tol * scale) | both['actual'].isna()
        violations = both[off].assign(difference=difference[off]).reset_index()
        return cls(len(both), violations)

    @classmethod
    def combine(cls, parts: Mapping[str, Comparison], label: str) -> Comparison:
        """Join comparisons into one.

        Args:
            parts: The comparisons by key.
            label: Name of the first column of the violations, which holds the
                key of their part.

        Raises:
            ValueError: `label` is a column of the violations of a part.
        """
        if any(label in part.violations.columns for part in parts.values()):
            raise ValueError(f'label {label!r} is a column of the violations of a part.')
        frames = [part.violations.assign(**{label: key}) for key, part in parts.items() if len(part.violations)]
        violations = pd.concat(frames, ignore_index=True) if frames else pd.DataFrame()
        if len(violations):
            violations = violations[[label, *violations.columns.drop(label)]]
        return cls(sum(part.compared for part in parts.values()), violations)

    @classmethod
    def long(cls, values: pd.DataFrame | pd.Series) -> pd.Series:
        """Return the values as one float series, with all column levels of a frame moved into the index.

        An unnamed index level gets the name `index` (`index_<i>` for level `i`
        of several), an unnamed column level `column` (`column_<i>`).
        """
        return cls._named(cls._stacked(values), values.index.nlevels)

    @staticmethod
    def _stacked(values: pd.DataFrame | pd.Series) -> pd.Series:
        if isinstance(values, pd.DataFrame):
            values = values.stack(list(range(values.columns.nlevels)), future_stack=True)
        return values.astype('float64')

    @staticmethod
    def _named(values: pd.Series, index_levels: int) -> pd.Series:
        """Name the unnamed levels: the first `index_levels` as index levels, the others as column levels."""
        index_names = list(values.index.names)
        groups = [('index', index_names[:index_levels]), ('column', index_names[index_levels:])]
        names = [
            name if name is not None else (stem if len(group) == 1 else f'{stem}_{i}')
            for stem, group in groups
            for i, name in enumerate(group)
        ]
        return values.set_axis(values.index.set_names(names))

    @staticmethod
    def _aligned(actual: pd.Series, expected: pd.Series) -> pd.Series:
        names_a, names_e = list(actual.index.names), list(expected.index.names)
        by_name = all(None not in names and len(set(names)) == len(names) for names in (names_a, names_e))
        if by_name:
            if set(names_a) != set(names_e):
                raise ValueError(f'The level names differ: actual {names_a}, expected {names_e}.')
            return expected.reorder_levels(names_a) if names_a != names_e else expected
        if len(names_a) != len(names_e):
            raise ValueError(f'actual has {len(names_a)} index and column levels, expected {len(names_e)}.')
        return expected

    @classmethod
    def _check_levels(cls, values: pd.Series, side: str) -> None:
        reserved = set(cls._VALUE_COLUMNS) & set(values.index.names)
        if reserved:
            raise ValueError(f'{side} has levels named {sorted(reserved)}; rename them.')
        if values.index.has_duplicates:
            raise ValueError(f'{side} has duplicate index entries.')

of classmethod

of(actual: DataFrame | Series, expected: DataFrame | Series, tol: float = 0.0, rel_tol: float = 0.0, empty_is_off: bool = False) -> Comparison

Compare actual with expected on the index and columns they share.

A value fits when |actual - expected| <= tol + rel_tol * max(|actual|, |expected|), as in math.isclose. A cell with NaN on either side is not compared, unless empty_is_off: then an empty actual cell with an expected value is a violation.

The index levels and column levels of both sides are matched by name when all levels on both sides have unique names; expected takes the level order of actual. Else they are matched by position.

Parameters:

Name Type Description Default
actual DataFrame | Series

The values to test.

required
expected DataFrame | Series

The values they must equal.

required
tol float

Absolute tolerance.

0.0
rel_tol float

Relative tolerance, a share of the larger absolute value.

0.0
empty_is_off bool

Count an empty actual cell with an expected value as a violation.

False

Raises:

Type Description
ValueError

The level names of both sides differ (all levels named), or the number of levels differs (matched by position); a level is named actual, expected or difference; a side has a duplicate index entry.

Source code in src/mesqual/validation/comparison.py
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
@classmethod
def of(
    cls,
    actual: pd.DataFrame | pd.Series,
    expected: pd.DataFrame | pd.Series,
    tol: float = 0.0,
    rel_tol: float = 0.0,
    empty_is_off: bool = False,
) -> Comparison:
    """Compare `actual` with `expected` on the index and columns they share.

    A value fits when `|actual - expected| <= tol + rel_tol * max(|actual|, |expected|)`,
    as in `math.isclose`. A cell with NaN on either side is not compared,
    unless `empty_is_off`: then an empty actual cell with an expected value
    is a violation.

    The index levels and column levels of both sides are matched by name
    when all levels on both sides have unique names; `expected` takes the
    level order of `actual`. Else they are matched by position.

    Args:
        actual: The values to test.
        expected: The values they must equal.
        tol: Absolute tolerance.
        rel_tol: Relative tolerance, a share of the larger absolute value.
        empty_is_off: Count an empty actual cell with an expected value as a violation.

    Raises:
        ValueError: The level names of both sides differ (all levels named),
            or the number of levels differs (matched by position); a level
            is named `actual`, `expected` or `difference`; a side has a
            duplicate index entry.
    """
    index_levels = actual.index.nlevels
    actual, expected = cls._stacked(actual), cls._stacked(expected)
    expected = cls._aligned(actual, expected)
    actual = cls._named(actual, index_levels)
    expected = expected.set_axis(expected.index.set_names(actual.index.names))
    cls._check_levels(actual, 'actual')
    cls._check_levels(expected, 'expected')
    both = pd.concat({'actual': actual, 'expected': expected}, axis=1, join='inner')
    both = both[both['expected'].notna()] if empty_is_off else both.dropna()
    difference = both['actual'] - both['expected']
    scale = both[['actual', 'expected']].abs().max(axis=1)
    off = (difference.abs() > tol + rel_tol * scale) | both['actual'].isna()
    violations = both[off].assign(difference=difference[off]).reset_index()
    return cls(len(both), violations)

combine classmethod

combine(parts: Mapping[str, Comparison], label: str) -> Comparison

Join comparisons into one.

Parameters:

Name Type Description Default
parts Mapping[str, Comparison]

The comparisons by key.

required
label str

Name of the first column of the violations, which holds the key of their part.

required

Raises:

Type Description
ValueError

label is a column of the violations of a part.

Source code in src/mesqual/validation/comparison.py
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
@classmethod
def combine(cls, parts: Mapping[str, Comparison], label: str) -> Comparison:
    """Join comparisons into one.

    Args:
        parts: The comparisons by key.
        label: Name of the first column of the violations, which holds the
            key of their part.

    Raises:
        ValueError: `label` is a column of the violations of a part.
    """
    if any(label in part.violations.columns for part in parts.values()):
        raise ValueError(f'label {label!r} is a column of the violations of a part.')
    frames = [part.violations.assign(**{label: key}) for key, part in parts.items() if len(part.violations)]
    violations = pd.concat(frames, ignore_index=True) if frames else pd.DataFrame()
    if len(violations):
        violations = violations[[label, *violations.columns.drop(label)]]
    return cls(sum(part.compared for part in parts.values()), violations)

long classmethod

long(values: DataFrame | Series) -> Series

Return the values as one float series, with all column levels of a frame moved into the index.

An unnamed index level gets the name index (index_<i> for level i of several), an unnamed column level column (column_<i>).

Source code in src/mesqual/validation/comparison.py
102
103
104
105
106
107
108
109
@classmethod
def long(cls, values: pd.DataFrame | pd.Series) -> pd.Series:
    """Return the values as one float series, with all column levels of a frame moved into the index.

    An unnamed index level gets the name `index` (`index_<i>` for level `i`
    of several), an unnamed column level `column` (`column_<i>`).
    """
    return cls._named(cls._stacked(values), values.index.nlevels)

ConstraintCheck

Bases: DatasetCheck

Checks that the values of one flag are within bounds.

The check fetches the flag and tests each value: it must be at least min_value and at most max_value. For an exact value, use equal. An empty value (NaN) is a violation unless isna_ok; it is not tested against the bounds. Without values to test, the result is SKIP.

A bound is a number, a Series or a DataFrame:

  • A number applies to every value.
  • A DataFrame aligns to the flag data by index and columns.
  • A Series on a frame flag applies per object when it matches the columns, per timestep when it matches the index. It has all levels of that axis; a bound on one level of a MultiIndex axis is not supported. The level names decide first: the bound index names equal the column level names (per object) or the index names (per timestep). Else the labels decide: the bound labels overlap the columns and not the index (per object), or the other way round (per timestep). Extra labels are allowed.
  • A Series on a series flag aligns to its index.

A value without a bound (its label is not in the bound, or the bound is NaN) is not tested against that bound. A Series or DataFrame bound that covers no value of the data, for example a tz-naive bound on a tz-aware index, raises ValueError in the run. So do a Series bound that matches neither axis or both, an empty or a duplicate bound label, a DataFrame bound on a series flag, a data level named value, min_value or max_value, and an index level and a column level with the same name. The validator reports these as ERROR.

FAIL lists one row per violation: the index levels and column levels of the flag data, value, then min_value and max_value for the bounds that are set.

Examples:

>>> loading = ConstraintCheck('Line.Results.loading', min_value=0, max_value=1, isna_ok=False)
>>> generation = ConstraintCheck(  # per object: a Series by generator id
...     'Generator.Results.generation', min_value=0, max_value=dataset.fetch('Generator.Model')['capacity'],
... )
>>> flow = ConstraintCheck(  # per value: a DataFrame like the flag data
...     'Line.Results.flow_sent_up', max_value=dataset.fetch('Line.Constraints.capacity_up'),
... )
>>> [check.run(dataset).status.name for check in (loading, generation, flow)]
['PASS', 'PASS', 'PASS']
>>> generation.run(dataset).summary
'1660 values of Generator.Results.generation meet the rule >= 0, <= max_value per object, empty allowed.'

Parameters:

Name Type Description Default
flag FlagType

The flag to check.

required
min_value float | Series | DataFrame | None

Lowest allowed value, or None.

None
max_value float | Series | DataFrame | None

Highest allowed value, or None.

None
isna_ok bool

An empty value is not a violation.

True
object_subset list[int | str] | None

Columns of a frame flag to check, for example line ids. None checks all columns. A column that the data does not have raises KeyError in the run, and a flag that gives a Series raises ValueError; the validator reports both as ERROR.

None
name str | None

The check name. Default: ConstraintCheck[<flag>].

None

Raises:

Type Description
ValueError

object_subset is empty, or a bound is NaN or a bool.

Attributes:

Name Type Description
flag

The flag to check.

min_value

Lowest allowed value, or None.

max_value

Highest allowed value, or None.

isna_ok

An empty value is not a violation.

object_subset

Columns of the flag data to check, or None for all.

Source code in src/mesqual/validation/constraint_check.py
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
class ConstraintCheck(DatasetCheck):
    """Checks that the values of one flag are within bounds.

    The check fetches the flag and tests each value: it must be at least
    `min_value` and at most `max_value`. For an exact value, use `equal`.
    An empty value (NaN) is a violation unless `isna_ok`; it is not tested
    against the bounds. Without values to test, the result is `SKIP`.

    A bound is a number, a `Series` or a `DataFrame`:

    - A number applies to every value.
    - A `DataFrame` aligns to the flag data by index and columns.
    - A `Series` on a frame flag applies per object when it matches the
      columns, per timestep when it matches the index. It has all levels of
      that axis; a bound on one level of a `MultiIndex` axis is not supported.
      The level names decide first: the bound index names equal the column
      level names (per object) or the index names (per timestep). Else the
      labels decide: the bound labels overlap the columns and not the index
      (per object), or the other way round (per timestep). Extra labels are
      allowed.
    - A `Series` on a series flag aligns to its index.

    A value without a bound (its label is not in the bound, or the bound is
    NaN) is not tested against that bound. A `Series` or `DataFrame` bound that
    covers no value of the data, for example a tz-naive bound on a tz-aware
    index, raises `ValueError` in the run. So do a `Series` bound that matches
    neither axis or both, an empty or a duplicate bound label, a `DataFrame`
    bound on a series flag, a data level named `value`, `min_value` or
    `max_value`, and an index level and a column level with the same name. The
    validator reports these as `ERROR`.

    `FAIL` lists one row per violation: the index levels and column levels of
    the flag data, `value`, then `min_value` and `max_value` for the bounds
    that are set.

    Examples:
        >>> loading = ConstraintCheck('Line.Results.loading', min_value=0, max_value=1, isna_ok=False)
        >>> generation = ConstraintCheck(  # per object: a Series by generator id
        ...     'Generator.Results.generation', min_value=0, max_value=dataset.fetch('Generator.Model')['capacity'],
        ... )
        >>> flow = ConstraintCheck(  # per value: a DataFrame like the flag data
        ...     'Line.Results.flow_sent_up', max_value=dataset.fetch('Line.Constraints.capacity_up'),
        ... )
        >>> [check.run(dataset).status.name for check in (loading, generation, flow)]
        ['PASS', 'PASS', 'PASS']
        >>> generation.run(dataset).summary
        '1660 values of Generator.Results.generation meet the rule >= 0, <= max_value per object, empty allowed.'

    Args:
        flag: The flag to check.
        min_value: Lowest allowed value, or `None`.
        max_value: Highest allowed value, or `None`.
        isna_ok: An empty value is not a violation.
        object_subset: Columns of a frame flag to check, for example line ids.
            `None` checks all columns. A column that the data does not have
            raises `KeyError` in the run, and a flag that gives a `Series`
            raises `ValueError`; the validator reports both as `ERROR`.
        name: The check name. Default: `ConstraintCheck[<flag>]`.

    Raises:
        ValueError: `object_subset` is empty, or a bound is NaN or a bool.

    Attributes:
        flag: The flag to check.
        min_value: Lowest allowed value, or `None`.
        max_value: Highest allowed value, or `None`.
        isna_ok: An empty value is not a violation.
        object_subset: Columns of the flag data to check, or `None` for all.
    """

    def __init__(
        self,
        flag: FlagType,
        min_value: float | pd.Series | pd.DataFrame | None = None,
        max_value: float | pd.Series | pd.DataFrame | None = None,
        isna_ok: bool = True,
        object_subset: list[int | str] | None = None,
        name: str | None = None,
    ):
        if object_subset is not None and not len(object_subset):
            raise ValueError('object_subset is empty; use None to check all columns.')
        for label, bound in (('min_value', min_value), ('max_value', max_value)):
            self._check_scalar(label, bound)
        self.flag = flag
        self.min_value = min_value
        self.max_value = max_value
        self.isna_ok = isna_ok
        self.object_subset = object_subset
        self._name = name

    @classmethod
    def equal(
        cls,
        flag: FlagType,
        value: float | pd.Series | pd.DataFrame,
        *,
        isna_ok: bool = True,
        object_subset: list[int | str] | None = None,
        name: str | None = None,
    ) -> ConstraintCheck:
        """Return a check that the values of `flag` equal `value`: `min_value` and `max_value` are both `value`.

        `value` is a number, a `Series` or a `DataFrame`, aligned as a bound
        (see the class docstring). The other arguments are those of the
        constructor. Example: `ConstraintCheck.equal('Line.Results.flow_sent_up', capacities, isna_ok=False)`.
        """
        return cls(flag, value, value, isna_ok=isna_ok, object_subset=object_subset, name=name)

    @property
    def name(self) -> str:
        """The check name: the name given, else `ConstraintCheck[<flag>]`."""
        return self._name or f'{type(self).__name__}[{self.flag}]'

    @property
    def flags(self) -> tuple[FlagType, ...]:
        """The flags the check reads: `flag`."""
        return (self.flag,)

    def _run(self, dataset: Dataset) -> CheckResult:
        data = dataset.fetch(self.flag)
        if self.object_subset is not None:
            if not isinstance(data, pd.DataFrame):
                raise ValueError(f'{self.flag} gives a {type(data).__name__}; object_subset selects frame columns.')
            data = data[self.object_subset]
        values = Comparison.long(data)
        self._check_levels(values)
        what = f'values of {self.flag}{self._subset_text()}'
        if values.empty:
            return self.skip(f'no {what}.')
        bounds = {label: self._aligned(label, bound, data) for label, bound in self._bounds().items()}
        broken = self._broken(values, {label: series for label, (_, series) in bounds.items()})
        rule = self._rule_text({label: form for label, (form, _) in bounds.items()})
        if broken.any():
            details = pd.concat(
                {'value': values[broken], **{label: series[broken] for label, (_, series) in bounds.items()}},
                axis=1,
            ).reset_index()
            summary = f'{int(broken.sum())} of {len(values)} {what} break the rule {rule}.'
            return self.result(CheckStatus.FAIL, summary, details)
        return self.result(CheckStatus.PASS, f'{len(values)} {what} meet the rule {rule}.')

    def _broken(self, values: pd.Series, bounds: dict[str, pd.Series]) -> pd.Series:
        empty = values.isna()
        off = pd.Series(False, index=values.index)
        if 'min_value' in bounds:
            off |= values < bounds['min_value']
        if 'max_value' in bounds:
            off |= values > bounds['max_value']
        return (off & ~empty) | (empty & (not self.isna_ok))

    def _bounds(self) -> dict[str, float | pd.Series | pd.DataFrame]:
        bounds = {'min_value': self.min_value, 'max_value': self.max_value}
        return {label: bound for label, bound in bounds.items() if bound is not None}

    @staticmethod
    def _check_scalar(label: str, bound: float | pd.Series | pd.DataFrame | None) -> None:
        if bound is None or isinstance(bound, (pd.Series, pd.DataFrame)):
            return
        if isinstance(bound, (bool, np.bool_)):
            raise ValueError(f'{label} is a bool; give a number.')
        if pd.isna(bound):
            raise ValueError(f'{label} is NaN; give a number or None.')

    @staticmethod
    def _check_levels(values: pd.Series) -> None:
        names = list(values.index.names)
        reserved = {'value', 'min_value', 'max_value'} & set(names)
        if reserved:
            raise ValueError(f'The flag data has levels named {sorted(reserved)}; rename them.')
        if len(set(names)) != len(names):
            raise ValueError(f'The flag data has index and column levels with the same name: {names}.')

    def _aligned(
        self,
        label: str,
        bound: float | pd.Series | pd.DataFrame,
        data: pd.DataFrame | pd.Series,
    ) -> tuple[str, pd.Series]:
        """Return the form of the bound and the bound as a long series aligned to the values of the data."""
        form = self._form(bound, data)
        shaped = Comparison.long(self._shaped(bound, data, form))
        if form != 'scalar' and shaped.isna().all():
            raise ValueError(f'{label} covers no value of {self.flag}; check its labels, level names and time zone.')
        return form, shaped

    @staticmethod
    def _shaped(
        bound: float | pd.Series | pd.DataFrame,
        data: pd.DataFrame | pd.Series,
        form: str,
    ) -> pd.DataFrame | pd.Series:
        """Return the bound with the index and columns of the data; NaN where it has no value."""
        if form == 'scalar':
            if isinstance(data, pd.DataFrame):
                return pd.DataFrame(bound, index=data.index, columns=data.columns)
            return pd.Series(bound, index=data.index)
        if form == 'per value':
            if isinstance(bound, pd.DataFrame):
                return bound.reindex(index=data.index, columns=data.columns)
            return bound.reindex(data.index)
        if form == 'per object':
            row = bound.reindex(data.columns).to_numpy(dtype='float64')
            return pd.DataFrame(np.broadcast_to(row, data.shape), index=data.index, columns=data.columns)
        column = bound.reindex(data.index).to_numpy(dtype='float64')
        return pd.DataFrame(np.broadcast_to(column[:, None], data.shape), index=data.index, columns=data.columns)

    @classmethod
    def _form(cls, bound: float | pd.Series | pd.DataFrame, data: pd.DataFrame | pd.Series) -> str:
        """`'scalar'`, `'per value'`, `'per object'` or `'per timestep'`.

        Raises:
            ValueError: The bound does not fit the data (see the class docstring).
        """
        if not isinstance(bound, (pd.Series, pd.DataFrame)):
            return 'scalar'
        if bound.empty:
            raise ValueError('A Series or DataFrame bound is empty.')
        axes = [bound.index] + ([bound.columns] if isinstance(bound, pd.DataFrame) else [])
        if any(axis.has_duplicates for axis in axes):
            raise ValueError('A bound has duplicate labels.')
        if isinstance(bound, pd.DataFrame):
            if not isinstance(data, pd.DataFrame):
                raise ValueError('A DataFrame bound needs a flag that gives a DataFrame.')
            return 'per value'
        if isinstance(data, pd.Series):
            return 'per value'
        return cls._series_form(bound, data)

    @classmethod
    def _series_form(cls, bound: pd.Series, data: pd.DataFrame) -> str:
        axes = {'per object': data.columns, 'per timestep': data.index}
        names = list(bound.index.names)
        by_name = [form for form, axis in axes.items() if None not in names and names == list(axis.names)]
        if len(by_name) == 1:
            return by_name[0]
        overlaps = {
            form: axis.nlevels == bound.index.nlevels and bool(bound.index.isin(axis).any())
            for form, axis in axes.items()
        }
        by_label = [form for form, overlap in overlaps.items() if overlap]
        if len(by_label) == 1:
            return by_label[0]
        on_one_level = any(cls._on_some_levels(bound.index, axis) for axis in axes.values())
        if on_one_level or bound.index.nlevels not in {axis.nlevels for axis in axes.values()}:
            raise ValueError(
                f'A Series bound has {bound.index.nlevels} index levels; it needs all levels of the columns '
                f'({data.columns.nlevels}) or of the index ({data.index.nlevels}). '
                'A bound on one level of a MultiIndex axis is not supported.'
            )
        raise ValueError(
            'A Series bound must match the columns (per object) or the index (per timestep) of the flag data, '
            'by level names or by labels, and not both.'
        )

    @staticmethod
    def _on_some_levels(labels: pd.Index, axis: pd.Index) -> bool:
        """`True` if `labels` has fewer levels than `axis` and matches some of its levels by name or by labels."""
        if labels.nlevels >= axis.nlevels:
            return False
        if set(labels.names) & set(axis.names) - {None}:
            return True
        return labels.nlevels == 1 and any(
            labels.isin(axis.get_level_values(level)).any() for level in range(axis.nlevels)
        )

    def _rule_text(self, forms: dict[str, str]) -> str:
        texts = {
            label: str(getattr(self, label)) if form == 'scalar' else f'{label} {form}' for label, form in forms.items()
        }
        if forms.get('min_value') == forms.get('max_value') == 'scalar' and self.min_value == self.max_value:
            rules = [f'= {self.min_value}']
        elif self.min_value is not None and self.min_value is self.max_value:
            rules = [f'= value {forms["min_value"]}']
        else:
            rules = [
                f'{op} {texts[label]}' for label, op in (('min_value', '>='), ('max_value', '<=')) if label in texts
            ]
        rules.append('empty allowed' if self.isna_ok else 'not empty')
        return ', '.join(rules)

    def _subset_text(self) -> str:
        return f' for objects {self.object_subset}' if self.object_subset is not None else ''

name property

name: str

The check name: the name given, else ConstraintCheck[<flag>].

flags property

flags: tuple[FlagType, ...]

The flags the check reads: flag.

equal classmethod

equal(flag: FlagType, value: float | Series | DataFrame, *, isna_ok: bool = True, object_subset: list[int | str] | None = None, name: str | None = None) -> ConstraintCheck

Return a check that the values of flag equal value: min_value and max_value are both value.

value is a number, a Series or a DataFrame, aligned as a bound (see the class docstring). The other arguments are those of the constructor. Example: ConstraintCheck.equal('Line.Results.flow_sent_up', capacities, isna_ok=False).

Source code in src/mesqual/validation/constraint_check.py
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
@classmethod
def equal(
    cls,
    flag: FlagType,
    value: float | pd.Series | pd.DataFrame,
    *,
    isna_ok: bool = True,
    object_subset: list[int | str] | None = None,
    name: str | None = None,
) -> ConstraintCheck:
    """Return a check that the values of `flag` equal `value`: `min_value` and `max_value` are both `value`.

    `value` is a number, a `Series` or a `DataFrame`, aligned as a bound
    (see the class docstring). The other arguments are those of the
    constructor. Example: `ConstraintCheck.equal('Line.Results.flow_sent_up', capacities, isna_ok=False)`.
    """
    return cls(flag, value, value, isna_ok=isna_ok, object_subset=object_subset, name=name)

ValidationReport dataclass

The results of the checks of one validation run, in run order.

Attributes:

Name Type Description
dataset str

Name of the dataset.

results Sequence[CheckResult]

One result per check.

Source code in src/mesqual/validation/report.py
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
@dataclass(frozen=True)
class ValidationReport:
    """The results of the checks of one validation run, in run order.

    Attributes:
        dataset: Name of the dataset.
        results: One result per check.
    """

    dataset: str
    results: Sequence[CheckResult]

    def __getitem__(self, check: str) -> CheckResult:
        """Return the result of the check with this name.

        Raises:
            KeyError: No result has the name.
        """
        for result in self.results:
            if result.check == check:
                return result
        raise KeyError(f'No check {check} in the report.')

    @property
    def failed(self) -> tuple[CheckResult, ...]:
        """The results with status `FAIL`."""
        return self._with(CheckStatus.FAIL)

    @property
    def errors(self) -> tuple[CheckResult, ...]:
        """The results with status `ERROR`: the checks that raised an exception."""
        return self._with(CheckStatus.ERROR)

    @property
    def passed(self) -> bool:
        """`True` if no check failed or raised."""
        return not self.failed and not self.errors

    def _with(self, status: CheckStatus) -> tuple[CheckResult, ...]:
        return tuple(result for result in self.results if result.status is status)

    def to_frame(self) -> pd.DataFrame:
        """Return one row per check: `check`, `status`, `summary` and the number of detail rows `details`."""
        rows = [(r.check, str(r.status), r.summary, len(r.details)) for r in self.results]
        return pd.DataFrame(rows, columns=['check', 'status', 'summary', 'details'])

    def to_markdown(self, path: Path | str | None = None, max_rows: int = 20) -> str:
        """Return the report as Markdown and write it to `path` if given.

        The report has one table with the status and summary of every check,
        then one section per check with details: the first `max_rows` detail
        rows and the number of rows left out. A details index other than the
        default range index becomes columns.

        Args:
            path: File to write, or `None`.
            max_rows: Detail rows per check.
        """
        lines = [f'# Validation of {self.dataset}', '', *self._table(self.to_frame().drop(columns='details'))]
        for result in self.results:
            if len(result.details):
                lines += ['', f'## {result.check}', '', f'{result.status}: {result.summary}', '']
                lines += self._table(self._flat(result.details).head(max_rows))
                if len(result.details) > max_rows:
                    lines += ['', f'{len(result.details) - max_rows} more rows.']
        text = '\n'.join(lines) + '\n'
        if path is not None:
            Path(path).write_text(text, encoding='utf-8')
        return text

    @staticmethod
    def _flat(details: pd.DataFrame) -> pd.DataFrame:
        if isinstance(details.index, pd.RangeIndex) and details.index.name is None:
            return details
        return details.reset_index()

    @classmethod
    def _table(cls, frame: pd.DataFrame) -> list[str]:
        header = [cls._cell(column) for column in frame.columns]
        cells = frame.map(cls._cell)
        return [
            f'| {" | ".join(header)} |',
            f'|{"|".join(" --- " for _ in header)}|',
            *(f'| {" | ".join(row)} |' for row in cells.itertuples(index=False)),
        ]

    @staticmethod
    def _cell(value) -> str:
        if pd.api.types.is_scalar(value) and pd.isna(value):
            return ''
        return str(value).replace('|', '\\|').replace('\n', ' ')

failed property

failed: tuple[CheckResult, ...]

The results with status FAIL.

errors property

errors: tuple[CheckResult, ...]

The results with status ERROR: the checks that raised an exception.

passed property

passed: bool

True if no check failed or raised.

__getitem__

__getitem__(check: str) -> CheckResult

Return the result of the check with this name.

Raises:

Type Description
KeyError

No result has the name.

Source code in src/mesqual/validation/report.py
26
27
28
29
30
31
32
33
34
35
def __getitem__(self, check: str) -> CheckResult:
    """Return the result of the check with this name.

    Raises:
        KeyError: No result has the name.
    """
    for result in self.results:
        if result.check == check:
            return result
    raise KeyError(f'No check {check} in the report.')

to_frame

to_frame() -> DataFrame

Return one row per check: check, status, summary and the number of detail rows details.

Source code in src/mesqual/validation/report.py
55
56
57
58
def to_frame(self) -> pd.DataFrame:
    """Return one row per check: `check`, `status`, `summary` and the number of detail rows `details`."""
    rows = [(r.check, str(r.status), r.summary, len(r.details)) for r in self.results]
    return pd.DataFrame(rows, columns=['check', 'status', 'summary', 'details'])

to_markdown

to_markdown(path: Path | str | None = None, max_rows: int = 20) -> str

Return the report as Markdown and write it to path if given.

The report has one table with the status and summary of every check, then one section per check with details: the first max_rows detail rows and the number of rows left out. A details index other than the default range index becomes columns.

Parameters:

Name Type Description Default
path Path | str | None

File to write, or None.

None
max_rows int

Detail rows per check.

20
Source code in src/mesqual/validation/report.py
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def to_markdown(self, path: Path | str | None = None, max_rows: int = 20) -> str:
    """Return the report as Markdown and write it to `path` if given.

    The report has one table with the status and summary of every check,
    then one section per check with details: the first `max_rows` detail
    rows and the number of rows left out. A details index other than the
    default range index becomes columns.

    Args:
        path: File to write, or `None`.
        max_rows: Detail rows per check.
    """
    lines = [f'# Validation of {self.dataset}', '', *self._table(self.to_frame().drop(columns='details'))]
    for result in self.results:
        if len(result.details):
            lines += ['', f'## {result.check}', '', f'{result.status}: {result.summary}', '']
            lines += self._table(self._flat(result.details).head(max_rows))
            if len(result.details) > max_rows:
                lines += ['', f'{len(result.details) - max_rows} more rows.']
    text = '\n'.join(lines) + '\n'
    if path is not None:
        Path(path).write_text(text, encoding='utf-8')
    return text

CheckResult dataclass

The result of one check on one dataset.

Attributes:

Name Type Description
check str

Name of the check.

status CheckStatus

The status.

summary str

One line on the result.

details DataFrame

One row per violation or finding; empty if there is none. Equality of two results does not compare the details.

Source code in src/mesqual/validation/result.py
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
@dataclass(frozen=True)
class CheckResult:
    """The result of one check on one dataset.

    Attributes:
        check: Name of the check.
        status: The status.
        summary: One line on the result.
        details: One row per violation or finding; empty if there is none.
            Equality of two results does not compare the details.
    """

    check: str
    status: CheckStatus
    summary: str
    details: pd.DataFrame = field(default_factory=pd.DataFrame, compare=False)

CheckStatus

Bases: str, Enum

Status of a check.

Attributes:

Name Type Description
PASS

The rule holds.

FAIL

The rule does not hold. The details list the violations.

INFO

A finding to read, not a failure.

SKIP

The check did not run, for example because a flag is not available.

ERROR

The check raised an exception. The summary gives its type and text.

Source code in src/mesqual/validation/result.py
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class CheckStatus(str, Enum):
    """Status of a check.

    Attributes:
        PASS: The rule holds.
        FAIL: The rule does not hold. The details list the violations.
        INFO: A finding to read, not a failure.
        SKIP: The check did not run, for example because a flag is not available.
        ERROR: The check raised an exception. The summary gives its type and text.
    """

    PASS = 'PASS'
    FAIL = 'FAIL'
    INFO = 'INFO'
    SKIP = 'SKIP'
    ERROR = 'ERROR'

    def __str__(self) -> str:
        return self.value

DatasetValidator

Holds checks by name and runs them on a dataset.

Check names are unique. A platform package subclasses the validator to give a default set of checks or extra constructor inputs. It passes the default checks to super().__init__(checks). A subclass that binds a dataset at construction overrides run(self, dataset=None) and falls back to the bound dataset: return super().run(self.dataset if dataset is None else dataset).

Examples:

>>> from mesqual.validation import ConstraintCheck
>>> validator = DatasetValidator([
...     ConstraintCheck('Line.Constraints.capacity_up', min_value=0),
...     ConstraintCheck('Generator.Results.generation', min_value=0, isna_ok=False),
... ])
>>> validator.remove('ConstraintCheck[Line.Constraints.capacity_up]')
>>> report = validator.run(dataset)
>>> report.passed
True

Parameters:

Name Type Description Default
checks Iterable[DatasetCheck]

The checks, in run order.

()

Raises:

Type Description
TypeError

A check is not a DatasetCheck.

ValueError

Two checks have the same name.

Source code in src/mesqual/validation/validator.py
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
class DatasetValidator:
    """Holds checks by name and runs them on a dataset.

    Check names are unique. A platform package subclasses the validator to
    give a default set of checks or extra constructor inputs. It passes the
    default checks to `super().__init__(checks)`. A subclass that binds a
    dataset at construction overrides `run(self, dataset=None)` and falls back
    to the bound dataset: `return super().run(self.dataset if dataset is None else dataset)`.

    Examples:
        >>> from mesqual.validation import ConstraintCheck
        >>> validator = DatasetValidator([
        ...     ConstraintCheck('Line.Constraints.capacity_up', min_value=0),
        ...     ConstraintCheck('Generator.Results.generation', min_value=0, isna_ok=False),
        ... ])
        >>> validator.remove('ConstraintCheck[Line.Constraints.capacity_up]')
        >>> report = validator.run(dataset)
        >>> report.passed
        True

    Args:
        checks: The checks, in run order.

    Raises:
        TypeError: A check is not a `DatasetCheck`.
        ValueError: Two checks have the same name.
    """

    def __init__(self, checks: Iterable[DatasetCheck] = ()):
        self._checks: dict[str, DatasetCheck] = {}
        for check in checks:
            self.add(check)

    @property
    def checks(self) -> tuple[DatasetCheck, ...]:
        """The checks, in run order."""
        return tuple(self._checks.values())

    def add(self, check: DatasetCheck, *, replace: bool = False) -> None:
        """Add a check at the end.

        Args:
            check: The check.
            replace: Replace a check with the same name in place. A check with
                a new name is added at the end.

        Raises:
            TypeError: The check is not a `DatasetCheck`.
            ValueError: The validator has a check with the same name and `replace` is `False`.
        """
        if not isinstance(check, DatasetCheck):
            raise TypeError(f'A validator runs DatasetCheck instances, not {type(check).__name__}.')
        if check.name in self._checks and not replace:
            raise ValueError(f'The validator has a check {check.name}; add it with replace=True to replace it.')
        self._checks[check.name] = check

    def remove(self, name: str) -> None:
        """Remove the check with this name.

        Raises:
            KeyError: No check has the name.
        """
        if name not in self._checks:
            raise KeyError(f'No check {name} in the validator.')
        del self._checks[name]

    def run(self, dataset: Dataset) -> ValidationReport:
        """Run every check on the dataset and return the report.

        A check that raises an exception gets the status `ERROR` with
        `"Type: text"` of the exception as summary, and the other checks still
        run. The traceback goes to the log at level `WARNING`.
        """
        return ValidationReport(dataset.name, tuple(self._run_check(check, dataset) for check in self.checks))

    @staticmethod
    def _run_check(check: DatasetCheck, dataset: Dataset) -> CheckResult:
        try:
            return check.run(dataset)
        except Exception as error:
            logger.warning('Check %s raised on dataset %s.', check.name, dataset.name, exc_info=True)
            return check.result(CheckStatus.ERROR, f'{type(error).__name__}: {error}')

checks property

checks: tuple[DatasetCheck, ...]

The checks, in run order.

add

add(check: DatasetCheck, *, replace: bool = False) -> None

Add a check at the end.

Parameters:

Name Type Description Default
check DatasetCheck

The check.

required
replace bool

Replace a check with the same name in place. A check with a new name is added at the end.

False

Raises:

Type Description
TypeError

The check is not a DatasetCheck.

ValueError

The validator has a check with the same name and replace is False.

Source code in src/mesqual/validation/validator.py
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def add(self, check: DatasetCheck, *, replace: bool = False) -> None:
    """Add a check at the end.

    Args:
        check: The check.
        replace: Replace a check with the same name in place. A check with
            a new name is added at the end.

    Raises:
        TypeError: The check is not a `DatasetCheck`.
        ValueError: The validator has a check with the same name and `replace` is `False`.
    """
    if not isinstance(check, DatasetCheck):
        raise TypeError(f'A validator runs DatasetCheck instances, not {type(check).__name__}.')
    if check.name in self._checks and not replace:
        raise ValueError(f'The validator has a check {check.name}; add it with replace=True to replace it.')
    self._checks[check.name] = check

remove

remove(name: str) -> None

Remove the check with this name.

Raises:

Type Description
KeyError

No check has the name.

Source code in src/mesqual/validation/validator.py
72
73
74
75
76
77
78
79
80
def remove(self, name: str) -> None:
    """Remove the check with this name.

    Raises:
        KeyError: No check has the name.
    """
    if name not in self._checks:
        raise KeyError(f'No check {name} in the validator.')
    del self._checks[name]

run

run(dataset: Dataset) -> ValidationReport

Run every check on the dataset and return the report.

A check that raises an exception gets the status ERROR with "Type: text" of the exception as summary, and the other checks still run. The traceback goes to the log at level WARNING.

Source code in src/mesqual/validation/validator.py
82
83
84
85
86
87
88
89
def run(self, dataset: Dataset) -> ValidationReport:
    """Run every check on the dataset and return the report.

    A check that raises an exception gets the status `ERROR` with
    `"Type: text"` of the exception as summary, and the other checks still
    run. The traceback goes to the log at level `WARNING`.
    """
    return ValidationReport(dataset.name, tuple(self._run_check(check, dataset) for check in self.checks))