Skip to content

Layer Flag References

layer_flags

Aggregations and interpreters of layer flags on area and border layers.

LayerAggregation dataclass

Bases: ABC

Common base of AreaAggregation and BorderAggregation. Subclasses are frozen dataclasses.

An aggregation that fetches flags besides the base flag lists them in input_flags. The layer flag is available only if these inputs are available. It passes the effective_config that aggregate gets to each of these fetches, as an interpreter does, so the config of the call reaches the inputs. fetch passes it, together with the parameters that the spec of the input declares.

Attributes:

Name Type Description
version int

Bump by hand when the logic changes. Part of the version identity of each layer flag.

Source code in src/mesqual/layer_flags/aggregations.py
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
@dataclass(frozen=True, kw_only=True)
class LayerAggregation(ABC):
    """Common base of `AreaAggregation` and `BorderAggregation`. Subclasses are frozen dataclasses.

    An aggregation that fetches flags besides the base flag lists them in
    `input_flags`. The layer flag is available only if these inputs are
    available. It passes the `effective_config` that `aggregate` gets to
    each of these fetches, as an interpreter does, so the config of the call
    reaches the inputs. `fetch` passes it, together with the parameters that
    the spec of the input declares.

    Attributes:
        version: Bump by hand when the logic changes. Part of the version
            identity of each layer flag.
    """

    version: int = 1

    def identity(self) -> tuple[str, dict[str, Any]]:
        """Return the link of the aggregation in the version identity: class name and fields.

        Examples:
            >>> WeightedMean(weight='Node.Results.demand_total').identity()
            ('WeightedMean', {'weight': 'Node.Results.demand_total', 'version': 2})
        """
        fields = {f.name: getattr(self, f.name) for f in dataclasses.fields(self) if f.name != 'version'}
        return type(self).__qualname__, {**fields, 'version': self.version}

    def input_flags(
        self,
        dataset: Dataset,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
    ) -> tuple[str, ...]:
        """Return the flags that `aggregate` fetches besides the base flag, for example a weight.

        The layer flag interpreter checks their availability without
        fetching them. Default: none.

        Args:
            dataset: The platform dataset.
            layer: The layer of the flag.
            variant: The variant values of the flag, as `aggregate` gets them.
        """
        return ()

    def check_spec(self, spec: LayerFlagSpec) -> None:
        """Raise `ValueError` if the layer spec does not give what the aggregation needs. Default: no check.

        `LayerFlagSpec` calls it at declaration, so a misdeclared spec
        fails there and not in `describe_flags` or `fetch`.
        """

    @staticmethod
    def _variant_names(spec: LayerFlagSpec) -> set[str]:
        """The names in the `variant` that `aggregate` gets: the variants of the spec and the layer of a layer base."""
        names = {d.name for d in spec.variants}
        placeholder = spec._base_layer_placeholder
        return names if placeholder is None else names | {placeholder}

    @staticmethod
    def fetch(
        dataset: Dataset,
        flag: str,
        effective_config: DatasetConfig | None,
        params: dict[str, Any],
    ) -> pd.DataFrame:
        """Fetch a flag with `effective_config` and the parameters of `params`.

        Parameters that the spec of `flag` does not declare are left out.
        """
        declared = {d.name for d in dataset.flag_index.get_spec(flag).params}
        kwargs = {name: value for name, value in params.items() if name in declared}
        return dataset.fetch(flag, effective_config, **kwargs)

identity

identity() -> tuple[str, dict[str, Any]]

Return the link of the aggregation in the version identity: class name and fields.

Examples:

>>> WeightedMean(weight='Node.Results.demand_total').identity()
('WeightedMean', {'weight': 'Node.Results.demand_total', 'version': 2})
Source code in src/mesqual/layer_flags/aggregations.py
54
55
56
57
58
59
60
61
62
def identity(self) -> tuple[str, dict[str, Any]]:
    """Return the link of the aggregation in the version identity: class name and fields.

    Examples:
        >>> WeightedMean(weight='Node.Results.demand_total').identity()
        ('WeightedMean', {'weight': 'Node.Results.demand_total', 'version': 2})
    """
    fields = {f.name: getattr(self, f.name) for f in dataclasses.fields(self) if f.name != 'version'}
    return type(self).__qualname__, {**fields, 'version': self.version}

input_flags

input_flags(dataset: Dataset, layer: AreaLayer | BorderLayer, variant: LayerVariantValues) -> tuple[str, ...]

Return the flags that aggregate fetches besides the base flag, for example a weight.

The layer flag interpreter checks their availability without fetching them. Default: none.

Parameters:

Name Type Description Default
dataset Dataset

The platform dataset.

required
layer AreaLayer | BorderLayer

The layer of the flag.

required
variant LayerVariantValues

The variant values of the flag, as aggregate gets them.

required
Source code in src/mesqual/layer_flags/aggregations.py
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def input_flags(
    self,
    dataset: Dataset,
    layer: AreaLayer | BorderLayer,
    variant: LayerVariantValues,
) -> tuple[str, ...]:
    """Return the flags that `aggregate` fetches besides the base flag, for example a weight.

    The layer flag interpreter checks their availability without
    fetching them. Default: none.

    Args:
        dataset: The platform dataset.
        layer: The layer of the flag.
        variant: The variant values of the flag, as `aggregate` gets them.
    """
    return ()

check_spec

check_spec(spec: LayerFlagSpec) -> None

Raise ValueError if the layer spec does not give what the aggregation needs. Default: no check.

LayerFlagSpec calls it at declaration, so a misdeclared spec fails there and not in describe_flags or fetch.

Source code in src/mesqual/layer_flags/aggregations.py
82
83
84
85
86
87
def check_spec(self, spec: LayerFlagSpec) -> None:
    """Raise `ValueError` if the layer spec does not give what the aggregation needs. Default: no check.

    `LayerFlagSpec` calls it at declaration, so a misdeclared spec
    fails there and not in `describe_flags` or `fetch`.
    """

fetch staticmethod

fetch(dataset: Dataset, flag: str, effective_config: DatasetConfig | None, params: dict[str, Any]) -> DataFrame

Fetch a flag with effective_config and the parameters of params.

Parameters that the spec of flag does not declare are left out.

Source code in src/mesqual/layer_flags/aggregations.py
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
@staticmethod
def fetch(
    dataset: Dataset,
    flag: str,
    effective_config: DatasetConfig | None,
    params: dict[str, Any],
) -> pd.DataFrame:
    """Fetch a flag with `effective_config` and the parameters of `params`.

    Parameters that the spec of `flag` does not declare are left out.
    """
    declared = {d.name for d in dataset.flag_index.get_spec(flag).params}
    kwargs = {name: value for name, value in params.items() if name in declared}
    return dataset.fetch(flag, effective_config, **kwargs)

AreaAggregation dataclass

Bases: LayerAggregation

Aggregates the values of base objects to areas. Subclasses are frozen dataclasses.

A platform package or a study subclasses it for special cases.

Source code in src/mesqual/layer_flags/aggregations.py
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@dataclass(frozen=True, kw_only=True)
class AreaAggregation(LayerAggregation):
    """Aggregates the values of base objects to areas. Subclasses are frozen dataclasses.

    A platform package or a study subclasses it for special cases.
    """

    @abstractmethod
    def aggregate(
        self,
        values: pd.DataFrame | None,
        mapping: pd.Series | pd.DataFrame,
        dataset: Dataset,
        layer: AreaLayer,
        variant: LayerVariantValues,
        effective_config: DatasetConfig,
        **params: Any,
    ) -> pd.DataFrame:
        """Return the aggregated result with the area as object level.

        For a model flag, one row per area. For a time series, one column per
        area; an aggregation on a border base may add column levels, for
        example a partner area.

        Args:
            values: The base flag for the same variant values. On a node
                base, one column per base object, or one row per base object
                for a model flag; objects without area are left out. On a
                border base, one column per border. `None` if the spec
                consumes base variant dimensions, except for a model spec on
                a node base; the aggregation then fetches the variants it
                needs with
                `self.fetch(dataset, variant.base_flag(...), effective_config, params)`.
            mapping: For a node base: base object id to area id, from the
                layer membership column, without objects that have no area.
                For a border base: the border model, with `area_from` and
                `area_to` per border.
            dataset: The platform dataset, to fetch more flags, for example
                weights.
            layer: The layer to aggregate to.
            variant: The variant values of the flag, including the layer. For
                a border base, also the `border_layer` that the interpreter
                resolved for the area layer.
            effective_config: The effective config of the layer flag fetch.
                Pass it to each fetch.
            **params: The resolved parameters of the flag.
        """

aggregate abstractmethod

aggregate(values: DataFrame | None, mapping: Series | DataFrame, dataset: Dataset, layer: AreaLayer, variant: LayerVariantValues, effective_config: DatasetConfig, **params: Any) -> DataFrame

Return the aggregated result with the area as object level.

For a model flag, one row per area. For a time series, one column per area; an aggregation on a border base may add column levels, for example a partner area.

Parameters:

Name Type Description Default
values DataFrame | None

The base flag for the same variant values. On a node base, one column per base object, or one row per base object for a model flag; objects without area are left out. On a border base, one column per border. None if the spec consumes base variant dimensions, except for a model spec on a node base; the aggregation then fetches the variants it needs with self.fetch(dataset, variant.base_flag(...), effective_config, params).

required
mapping Series | DataFrame

For a node base: base object id to area id, from the layer membership column, without objects that have no area. For a border base: the border model, with area_from and area_to per border.

required
dataset Dataset

The platform dataset, to fetch more flags, for example weights.

required
layer AreaLayer

The layer to aggregate to.

required
variant LayerVariantValues

The variant values of the flag, including the layer. For a border base, also the border_layer that the interpreter resolved for the area layer.

required
effective_config DatasetConfig

The effective config of the layer flag fetch. Pass it to each fetch.

required
**params Any

The resolved parameters of the flag.

{}
Source code in src/mesqual/layer_flags/aggregations.py
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
@abstractmethod
def aggregate(
    self,
    values: pd.DataFrame | None,
    mapping: pd.Series | pd.DataFrame,
    dataset: Dataset,
    layer: AreaLayer,
    variant: LayerVariantValues,
    effective_config: DatasetConfig,
    **params: Any,
) -> pd.DataFrame:
    """Return the aggregated result with the area as object level.

    For a model flag, one row per area. For a time series, one column per
    area; an aggregation on a border base may add column levels, for
    example a partner area.

    Args:
        values: The base flag for the same variant values. On a node
            base, one column per base object, or one row per base object
            for a model flag; objects without area are left out. On a
            border base, one column per border. `None` if the spec
            consumes base variant dimensions, except for a model spec on
            a node base; the aggregation then fetches the variants it
            needs with
            `self.fetch(dataset, variant.base_flag(...), effective_config, params)`.
        mapping: For a node base: base object id to area id, from the
            layer membership column, without objects that have no area.
            For a border base: the border model, with `area_from` and
            `area_to` per border.
        dataset: The platform dataset, to fetch more flags, for example
            weights.
        layer: The layer to aggregate to.
        variant: The variant values of the flag, including the layer. For
            a border base, also the `border_layer` that the interpreter
            resolved for the area layer.
        effective_config: The effective config of the layer flag fetch.
            Pass it to each fetch.
        **params: The resolved parameters of the flag.
    """

BorderAggregation dataclass

Bases: LayerAggregation

Aggregates edge values or area values to borders. Subclasses are frozen dataclasses.

Source code in src/mesqual/layer_flags/aggregations.py
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
@dataclass(frozen=True, kw_only=True)
class BorderAggregation(LayerAggregation):
    """Aggregates edge values or area values to borders. Subclasses are frozen dataclasses."""

    @abstractmethod
    def aggregate(
        self,
        values: pd.DataFrame | None,
        border_model: pd.DataFrame | None,
        dataset: Dataset,
        layer: BorderLayer,
        variant: LayerVariantValues,
        effective_config: DatasetConfig,
        **params: Any,
    ) -> pd.DataFrame:
        """Return one column per border, or one row per border for a model flag.

        Args:
            values: The base flag for the same variant values: one column per
                edge, or per area for an area base. For an edge model base,
                the edge model. `None` if the spec consumes base variant
                dimensions. An aggregation that needs other variant values
                (for example the swapped direction) fetches them with
                `self.fetch(dataset, variant.base_flag(...), effective_config, params)`.
            border_model: The border model of the layer, with `area_from`
                and `area_to` per border. `None` for the border model flag
                itself.
            dataset: The platform dataset, to fetch more flags.
            layer: The layer to aggregate to.
            variant: The variant values of the flag, including the layer. For
                an area base, also the `area_layer` of the border layer.
            effective_config: The effective config of the layer flag fetch.
                Pass it to each fetch.
            **params: The resolved parameters of the flag.
        """

aggregate abstractmethod

aggregate(values: DataFrame | None, border_model: DataFrame | None, dataset: Dataset, layer: BorderLayer, variant: LayerVariantValues, effective_config: DatasetConfig, **params: Any) -> DataFrame

Return one column per border, or one row per border for a model flag.

Parameters:

Name Type Description Default
values DataFrame | None

The base flag for the same variant values: one column per edge, or per area for an area base. For an edge model base, the edge model. None if the spec consumes base variant dimensions. An aggregation that needs other variant values (for example the swapped direction) fetches them with self.fetch(dataset, variant.base_flag(...), effective_config, params).

required
border_model DataFrame | None

The border model of the layer, with area_from and area_to per border. None for the border model flag itself.

required
dataset Dataset

The platform dataset, to fetch more flags.

required
layer BorderLayer

The layer to aggregate to.

required
variant LayerVariantValues

The variant values of the flag, including the layer. For an area base, also the area_layer of the border layer.

required
effective_config DatasetConfig

The effective config of the layer flag fetch. Pass it to each fetch.

required
**params Any

The resolved parameters of the flag.

{}
Source code in src/mesqual/layer_flags/aggregations.py
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
@abstractmethod
def aggregate(
    self,
    values: pd.DataFrame | None,
    border_model: pd.DataFrame | None,
    dataset: Dataset,
    layer: BorderLayer,
    variant: LayerVariantValues,
    effective_config: DatasetConfig,
    **params: Any,
) -> pd.DataFrame:
    """Return one column per border, or one row per border for a model flag.

    Args:
        values: The base flag for the same variant values: one column per
            edge, or per area for an area base. For an edge model base,
            the edge model. `None` if the spec consumes base variant
            dimensions. An aggregation that needs other variant values
            (for example the swapped direction) fetches them with
            `self.fetch(dataset, variant.base_flag(...), effective_config, params)`.
        border_model: The border model of the layer, with `area_from`
            and `area_to` per border. `None` for the border model flag
            itself.
        dataset: The platform dataset, to fetch more flags.
        layer: The layer to aggregate to.
        variant: The variant values of the flag, including the layer. For
            an area base, also the `area_layer` of the border layer.
        effective_config: The effective config of the layer flag fetch.
            Pass it to each fetch.
        **params: The resolved parameters of the flag.
    """

Sum dataclass

Bases: AreaAggregation

Sum over the objects of each area, with the NaN and index rules of AreaSumCalculator.calculate.

Source code in src/mesqual/layer_flags/aggregations.py
204
205
206
207
208
209
210
211
212
213
@dataclass(frozen=True, kw_only=True)
class Sum(AreaAggregation):
    """Sum over the objects of each area, with the NaN and index rules of `AreaSumCalculator.calculate`."""

    version: int = 2

    def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one column per area with the sum of its objects."""
        values = _require_values(self, values)
        return AreaSumCalculator(mapping.to_frame(layer.index), layer.index).calculate(values)

aggregate

aggregate(values, mapping, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one column per area with the sum of its objects.

Source code in src/mesqual/layer_flags/aggregations.py
210
211
212
213
def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one column per area with the sum of its objects."""
    values = _require_values(self, values)
    return AreaSumCalculator(mapping.to_frame(layer.index), layer.index).calculate(values)

WeightedMean dataclass

Bases: AreaAggregation

Mean over the objects of each area, weighted by the flag weight. Uses AreaPriceCalculator.

Attributes:

Name Type Description
weight str

Weight flag. It may use the placeholders of the base name, for example 'Node.Results.demand_{product}'. A time step where the weights of an area sum to 0 gives NaN for that area.

Examples:

>>> WeightedMean(weight='Node.Results.demand_total')
WeightedMean(version=2, weight='Node.Results.demand_total')
Source code in src/mesqual/layer_flags/aggregations.py
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
@dataclass(frozen=True, kw_only=True)
class WeightedMean(AreaAggregation):
    """Mean over the objects of each area, weighted by the flag `weight`. Uses `AreaPriceCalculator`.

    Attributes:
        weight: Weight flag. It may use the placeholders of the base name,
            for example `'Node.Results.demand_{product}'`. A time step where
            the weights of an area sum to 0 gives NaN for that area.

    Examples:
        >>> WeightedMean(weight='Node.Results.demand_total')
        WeightedMean(version=2, weight='Node.Results.demand_total')
    """

    weight: str
    version: int = 2

    def check_spec(self, spec: LayerFlagSpec) -> None:
        """Raise `ValueError` if `weight` has a placeholder that is not in the variant values of the spec."""
        fields = {name for _, name, _, _ in string.Formatter().parse(self.weight) if name is not None}
        unknown = fields - self._variant_names(spec)
        if unknown:
            raise ValueError(
                f'Spec {spec.name!r}: the weight {self.weight!r} has placeholders {sorted(unknown)} '
                f'that are no variants of the spec.'
            )

    def input_flags(
        self,
        dataset: Dataset,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
    ) -> tuple[str, ...]:
        """Return the weight flag for the variant values."""
        return (self._weight_flag(variant),)

    def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one column per area with the weighted mean of its objects."""
        values = _require_values(self, values)
        weights = self.fetch(dataset, self._weight_flag(variant), effective_config, params)
        weights = weights.reindex(index=values.index, columns=values.columns)
        return AreaPriceCalculator(mapping.to_frame(layer.index), layer.index).calculate(values, weights)

    def _weight_flag(self, variant: LayerVariantValues) -> str:
        return self.weight.format(**variant.as_dict())

check_spec

check_spec(spec: LayerFlagSpec) -> None

Raise ValueError if weight has a placeholder that is not in the variant values of the spec.

Source code in src/mesqual/layer_flags/aggregations.py
233
234
235
236
237
238
239
240
241
def check_spec(self, spec: LayerFlagSpec) -> None:
    """Raise `ValueError` if `weight` has a placeholder that is not in the variant values of the spec."""
    fields = {name for _, name, _, _ in string.Formatter().parse(self.weight) if name is not None}
    unknown = fields - self._variant_names(spec)
    if unknown:
        raise ValueError(
            f'Spec {spec.name!r}: the weight {self.weight!r} has placeholders {sorted(unknown)} '
            f'that are no variants of the spec.'
        )

input_flags

input_flags(dataset: Dataset, layer: AreaLayer | BorderLayer, variant: LayerVariantValues) -> tuple[str, ...]

Return the weight flag for the variant values.

Source code in src/mesqual/layer_flags/aggregations.py
243
244
245
246
247
248
249
250
def input_flags(
    self,
    dataset: Dataset,
    layer: AreaLayer | BorderLayer,
    variant: LayerVariantValues,
) -> tuple[str, ...]:
    """Return the weight flag for the variant values."""
    return (self._weight_flag(variant),)

aggregate

aggregate(values, mapping, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one column per area with the weighted mean of its objects.

Source code in src/mesqual/layer_flags/aggregations.py
252
253
254
255
256
257
def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one column per area with the weighted mean of its objects."""
    values = _require_values(self, values)
    weights = self.fetch(dataset, self._weight_flag(variant), effective_config, params)
    weights = weights.reindex(index=values.index, columns=values.columns)
    return AreaPriceCalculator(mapping.to_frame(layer.index), layer.index).calculate(values, weights)

AreaModel dataclass

Bases: AreaAggregation

The area model: one row per area, with node count and representative point. Uses AreaModelGenerator.

Source code in src/mesqual/layer_flags/aggregations.py
263
264
265
266
267
268
269
270
271
@dataclass(frozen=True, kw_only=True)
class AreaModel(AreaAggregation):
    """The area model: one row per area, with node count and representative point. Uses `AreaModelGenerator`."""

    def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one row per area, indexed by `layer.index`."""
        objects = _require_values(self, values).loc[mapping.index].assign(**{layer.index: mapping})
        area_model = AreaModelGenerator(objects, layer.index).generate_area_model()
        return area_model.rename_axis(layer.index)

aggregate

aggregate(values, mapping, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one row per area, indexed by layer.index.

Source code in src/mesqual/layer_flags/aggregations.py
267
268
269
270
271
def aggregate(self, values, mapping, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one row per area, indexed by `layer.index`."""
    objects = _require_values(self, values).loc[mapping.index].assign(**{layer.index: mapping})
    area_model = AreaModelGenerator(objects, layer.index).generate_area_model()
    return area_model.rename_axis(layer.index)

BorderModel dataclass

Bases: BorderAggregation

The border model: one row per border and direction, with its areas, edge counts and geometry.

A border runs from area_from to area_to. Both directions of an area pair are borders. An edge counts with the border if its first end is in area_from and its second end in area_to, and against the border the other way round. An edge with both ends in one area, or with an end without area (AreaMembership.has_area), is on no border. So a border joins two areas that are both set. The columns n_edges_with_border and n_edges_against_border count the edges per border.

The columns opposite_border, canonical_border and is_canonical come from AreaBorderModelGenerator. canonical_border is the border of the pair with the area names in alphabetical order. is_canonical is True where the border is its canonical border.

The model always has the geometry columns (projection_point, azimuth_angle, geo_line_string, areas_touch). A missing geometry is NaN. The values come from AreaBorderGeometryCalculator. They need polygons in the column geometry of the area layer model. The calculator gets the CRS of these polygons: the CRS of a GeoDataFrame or of a geometry column. It reads polygons without a CRS as lon/lat coordinates in EPSG:4326. A border whose geometry fails gets NaN in the geometry columns, and the fetch logs a warning. If the dataset cannot give these polygons, the geometry columns are NaN and the fetch logs the reason at INFO, once per dataset and area layer. If the availability of the area layer model is unknown, the fetch fetches it.

Attributes:

Name Type Description
geometry bool

Compute the geometry. With False, the geometry columns are NaN.

Source code in src/mesqual/layer_flags/aggregations.py
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
@dataclass(frozen=True, kw_only=True)
class BorderModel(BorderAggregation):
    """The border model: one row per border and direction, with its areas, edge counts and geometry.

    A border runs from `area_from` to `area_to`. Both directions of an area
    pair are borders. An edge counts with the border if its first end is in
    `area_from` and its second end in `area_to`, and against the border the
    other way round. An edge with both ends in one area, or with an end
    without area (`AreaMembership.has_area`), is on no border. So a border
    joins two areas that are both set. The columns `n_edges_with_border` and
    `n_edges_against_border` count the edges per border.

    The columns `opposite_border`, `canonical_border` and `is_canonical` come
    from `AreaBorderModelGenerator`. `canonical_border` is the border of the
    pair with the area names in alphabetical order. `is_canonical` is True
    where the border is its canonical border.

    The model always has the geometry columns (`projection_point`,
    `azimuth_angle`, `geo_line_string`, `areas_touch`). A missing geometry is
    NaN. The values come from `AreaBorderGeometryCalculator`. They need
    polygons in the column `geometry` of the area layer model. The
    calculator gets the CRS of these polygons: the CRS of a `GeoDataFrame`
    or of a geometry column. It reads polygons without a CRS as lon/lat
    coordinates in EPSG:4326. A border whose geometry fails gets NaN in the
    geometry columns, and the fetch logs a warning. If
    the dataset cannot give these polygons, the geometry columns are NaN
    and the fetch logs the reason at `INFO`, once per dataset and area layer.
    If the availability of the area layer model is unknown, the fetch fetches it.

    Attributes:
        geometry: Compute the geometry. With `False`, the geometry columns are NaN.
    """

    N_EDGES_WITH = 'n_edges_with_border'
    N_EDGES_AGAINST = 'n_edges_against_border'
    _LOGGED_WITHOUT_GEOMETRY: ClassVar[weakref.WeakKeyDictionary[Dataset, set[str]]] = weakref.WeakKeyDictionary()
    """Per dataset: the area layers whose border model without geometry is logged at `INFO`."""

    geometry: bool = True
    version: int = 3

    def input_flags(
        self,
        dataset: Dataset,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
    ) -> tuple[str, ...]:
        """Return the base model of the area layer. The optional geometry adds no input."""
        return (dataset.flag_index.layers.area_layer_of(layer).base_model,)

    def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one row per border, indexed by `layer.index`."""
        edges = _require_values(self, values)
        index = dataset.flag_index
        area_layer = index.layers.area_layer_of(layer)
        column = index.get_mapping_column(area_layer)
        nodes = self.fetch(dataset, area_layer.base_model, effective_config, params)
        end_from, end_to = index.get_edge_ends(layer)
        generator = AreaBorderModelGenerator(nodes, edges, column, end_from, end_to, border_identifier=layer.index)
        model = generator.generate_area_border_model()
        model = self._add_edge_counts(model, generator.line_areas)
        if self.geometry:
            model = self._add_geometry(model, generator, dataset, area_layer, effective_config)
        columns = list(dict.fromkeys([*model.columns, *AreaBorderGeometryCalculator.GEOMETRY_COLUMNS]))
        return model.reindex(columns=columns)

    def _add_edge_counts(self, model: pd.DataFrame, edge_areas: pd.DataFrame) -> pd.DataFrame:
        counts = Counter(zip(edge_areas[AreaMembership.AREA_FROM], edge_areas[AreaMembership.AREA_TO]))
        pairs = list(zip(model[AreaMembership.AREA_FROM], model[AreaMembership.AREA_TO]))
        return model.assign(
            **{
                self.N_EDGES_WITH: [counts[(area_from, area_to)] for area_from, area_to in pairs],
                self.N_EDGES_AGAINST: [counts[(area_to, area_from)] for area_from, area_to in pairs],
            }
        )

    def _add_geometry(
        self,
        model: pd.DataFrame,
        generator: AreaBorderModelGenerator,
        dataset: Dataset,
        area_layer: AreaLayer,
        effective_config: DatasetConfig,
    ) -> pd.DataFrame:
        if model.empty:
            return model
        layer_model = area_layer.layer_model
        if not dataset.flag_is_accepted(layer_model):
            return self._without_geometry(model, dataset, area_layer, f'the dataset does not accept {layer_model}')
        if dataset.is_flag_available(layer_model) is False:
            return self._without_geometry(model, dataset, area_layer, f'{layer_model} is unavailable')
        areas = dataset.fetch(layer_model, effective_config)
        if 'geometry' not in areas or not all(isinstance(g, BaseGeometry) for g in areas['geometry']):
            reason = f'{layer_model} has no polygons in the column geometry'
            return self._without_geometry(model, dataset, area_layer, reason)
        calculator = AreaBorderGeometryCalculator(gpd.GeoDataFrame(areas, geometry='geometry'))
        calculator.non_crossing_path_finder.show_progress = False
        return generator.enhance_with_geometry(model, calculator)

    @classmethod
    def _without_geometry(
        cls,
        model: pd.DataFrame,
        dataset: Dataset,
        area_layer: AreaLayer,
        reason: str,
    ) -> pd.DataFrame:
        """Return `model`. Log the reason at `INFO` once per dataset and area layer, then at `DEBUG`."""
        logged = cls._LOGGED_WITHOUT_GEOMETRY.setdefault(dataset, set())
        level = logging.DEBUG if area_layer.name in logged else logging.INFO
        logged.add(area_layer.name)
        logger.log(level, 'Border model of area layer %r without geometry: %s.', area_layer.name, reason)
        return model

input_flags

input_flags(dataset: Dataset, layer: AreaLayer | BorderLayer, variant: LayerVariantValues) -> tuple[str, ...]

Return the base model of the area layer. The optional geometry adds no input.

Source code in src/mesqual/layer_flags/aggregations.py
315
316
317
318
319
320
321
322
def input_flags(
    self,
    dataset: Dataset,
    layer: AreaLayer | BorderLayer,
    variant: LayerVariantValues,
) -> tuple[str, ...]:
    """Return the base model of the area layer. The optional geometry adds no input."""
    return (dataset.flag_index.layers.area_layer_of(layer).base_model,)

aggregate

aggregate(values, border_model, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one row per border, indexed by layer.index.

Source code in src/mesqual/layer_flags/aggregations.py
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one row per border, indexed by `layer.index`."""
    edges = _require_values(self, values)
    index = dataset.flag_index
    area_layer = index.layers.area_layer_of(layer)
    column = index.get_mapping_column(area_layer)
    nodes = self.fetch(dataset, area_layer.base_model, effective_config, params)
    end_from, end_to = index.get_edge_ends(layer)
    generator = AreaBorderModelGenerator(nodes, edges, column, end_from, end_to, border_identifier=layer.index)
    model = generator.generate_area_border_model()
    model = self._add_edge_counts(model, generator.line_areas)
    if self.geometry:
        model = self._add_geometry(model, generator, dataset, area_layer, effective_config)
    columns = list(dict.fromkeys([*model.columns, *AreaBorderGeometryCalculator.GEOMETRY_COLUMNS]))
    return model.reindex(columns=columns)

BorderEdgeSum dataclass

Bases: BorderAggregation

Sum of an edge value over the edges of each border, in both directions.

For edge values without a direction, for example a cost or an income per line. An edge from A to B and an edge from B to A both add their value to the border A - B and to the border B - A. So both borders of a pair have the same value, and a sum over all borders counts each edge twice. To count each edge once, select one direction with the column is_canonical of the border model.

An edge with both ends in one area, or with an end without area, is on no border. A time step where all edges of a border are NaN gives NaN. The base needs no variant. It groups the edges by their areas (layer_edge_areas).

Source code in src/mesqual/layer_flags/aggregations.py
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
@dataclass(frozen=True, kw_only=True)
class BorderEdgeSum(BorderAggregation):
    """Sum of an edge value over the edges of each border, in both directions.

    For edge values without a direction, for example a cost or an income per
    line. An edge from A to B and an edge from B to A both add their value
    to the border A - B and to the border B - A. So both borders of a pair
    have the same value, and a sum over all borders counts each edge twice.
    To count each edge once, select one direction with the column
    `is_canonical` of the border model.

    An edge with both ends in one area, or with an end without area, is on
    no border. A time step where all edges of a border are NaN gives NaN.
    The base needs no variant. It groups the edges by their areas
    (`layer_edge_areas`).
    """

    def input_flags(
        self,
        dataset: Dataset,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
    ) -> tuple[str, ...]:
        """Return the inputs of `layer_edge_areas`."""
        return layer_edge_area_inputs(dataset, layer)

    def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one column per border: its edges in border direction plus its edges against it."""
        values = _require_values(self, values)
        against = self._against_border_values(values, dataset, variant, effective_config, params)
        edge_areas = layer_edge_areas(dataset, layer, effective_config, params)
        return self.sum_by_border(values, edge_areas, border_model, against).rename_axis(columns=layer.index)

    def _against_border_values(self, values, dataset, variant, effective_config, params) -> pd.DataFrame:
        """The values of the edges when they run against the border direction: `values`, without direction."""
        return values

    @classmethod
    def sum_by_border(
        cls,
        values: pd.DataFrame,
        edge_areas: pd.DataFrame,
        border_model: pd.DataFrame,
        against: pd.DataFrame | None = None,
    ) -> pd.DataFrame:
        """Return one column per border: the edges in border direction plus the edges against it.

        Args:
            values: One column per edge. An edge in border direction adds
                its value of `values`.
            edge_areas: Two columns per edge, as `AreaMembership.edge_areas`
                or `AreaBorderModelGenerator.line_areas` give them: the first
                with the area of the from end, the second with the area of
                the to end.
            border_model: The border model, with columns of the same two
                names: the source and the target area of each border.
            against: One column per edge. An edge against the border
                direction adds its value of `against`. Default: `values`.

        Returns:
            One column per border of `border_model`, on the index of
                `values`. NaN where all edges of a border are NaN, and for a
                border without edges in `values`.
        """
        against = values if against is None else against
        column_from, column_to = edge_areas.columns
        border_of_pair = dict(zip(zip(border_model[column_from], border_model[column_to]), border_model.index))
        area_from, area_to = edge_areas[column_from], edge_areas[column_to]
        with_border = cls._sum_in_direction(values, area_from, area_to, border_of_pair)
        against_border = cls._sum_in_direction(against, area_to, area_from, border_of_pair)
        return with_border.add(against_border, fill_value=0).reindex(columns=border_model.index)

    @staticmethod
    def _sum_in_direction(
        values: pd.DataFrame,
        area_from: pd.Series,
        area_to: pd.Series,
        border_of_pair: dict[tuple, Any],
    ) -> pd.DataFrame:
        """Sum the edges whose areas run from `area_from` to `area_to` of a border. One column per border."""
        border_of_edge = pd.Series(
            [border_of_pair.get(pair) for pair in zip(area_from, area_to)],
            index=area_from.index,
        )
        border_of_edge = border_of_edge.dropna()
        edges = values.columns.intersection(border_of_edge.index, sort=False)
        return values[edges].T.groupby(border_of_edge[edges]).sum(min_count=1).T

input_flags

input_flags(dataset: Dataset, layer: AreaLayer | BorderLayer, variant: LayerVariantValues) -> tuple[str, ...]

Return the inputs of layer_edge_areas.

Source code in src/mesqual/layer_flags/aggregations.py
406
407
408
409
410
411
412
413
def input_flags(
    self,
    dataset: Dataset,
    layer: AreaLayer | BorderLayer,
    variant: LayerVariantValues,
) -> tuple[str, ...]:
    """Return the inputs of `layer_edge_areas`."""
    return layer_edge_area_inputs(dataset, layer)

aggregate

aggregate(values, border_model, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one column per border: its edges in border direction plus its edges against it.

Source code in src/mesqual/layer_flags/aggregations.py
415
416
417
418
419
420
def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one column per border: its edges in border direction plus its edges against it."""
    values = _require_values(self, values)
    against = self._against_border_values(values, dataset, variant, effective_config, params)
    edge_areas = layer_edge_areas(dataset, layer, effective_config, params)
    return self.sum_by_border(values, edge_areas, border_model, against).rename_axis(columns=layer.index)

sum_by_border classmethod

sum_by_border(values: DataFrame, edge_areas: DataFrame, border_model: DataFrame, against: DataFrame | None = None) -> DataFrame

Return one column per border: the edges in border direction plus the edges against it.

Parameters:

Name Type Description Default
values DataFrame

One column per edge. An edge in border direction adds its value of values.

required
edge_areas DataFrame

Two columns per edge, as AreaMembership.edge_areas or AreaBorderModelGenerator.line_areas give them: the first with the area of the from end, the second with the area of the to end.

required
border_model DataFrame

The border model, with columns of the same two names: the source and the target area of each border.

required
against DataFrame | None

One column per edge. An edge against the border direction adds its value of against. Default: values.

None

Returns:

Type Description
DataFrame

One column per border of border_model, on the index of values. NaN where all edges of a border are NaN, and for a border without edges in values.

Source code in src/mesqual/layer_flags/aggregations.py
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
@classmethod
def sum_by_border(
    cls,
    values: pd.DataFrame,
    edge_areas: pd.DataFrame,
    border_model: pd.DataFrame,
    against: pd.DataFrame | None = None,
) -> pd.DataFrame:
    """Return one column per border: the edges in border direction plus the edges against it.

    Args:
        values: One column per edge. An edge in border direction adds
            its value of `values`.
        edge_areas: Two columns per edge, as `AreaMembership.edge_areas`
            or `AreaBorderModelGenerator.line_areas` give them: the first
            with the area of the from end, the second with the area of
            the to end.
        border_model: The border model, with columns of the same two
            names: the source and the target area of each border.
        against: One column per edge. An edge against the border
            direction adds its value of `against`. Default: `values`.

    Returns:
        One column per border of `border_model`, on the index of
            `values`. NaN where all edges of a border are NaN, and for a
            border without edges in `values`.
    """
    against = values if against is None else against
    column_from, column_to = edge_areas.columns
    border_of_pair = dict(zip(zip(border_model[column_from], border_model[column_to]), border_model.index))
    area_from, area_to = edge_areas[column_from], edge_areas[column_to]
    with_border = cls._sum_in_direction(values, area_from, area_to, border_of_pair)
    against_border = cls._sum_in_direction(against, area_to, area_from, border_of_pair)
    return with_border.add(against_border, fill_value=0).reindex(columns=border_model.index)

BorderFlowSum dataclass

Bases: BorderEdgeSum

Sum of the line flows across each border, in border direction.

A line against the border direction counts with its directions swapped: its down adds to the border up, its up to the border down, and its net with a minus sign. total (up plus down) is the same in both directions. The base needs a direction variant with the values up, down and, if used, net and total. Any other value raises ValueError. For up and down it fetches the swapped variant through self.fetch. It groups the lines by their areas (layer_edge_areas).

For example, a line from B to A on the border A-B with up 30 and down 50 adds 50 to the border up and 30 to the border down.

Source code in src/mesqual/layer_flags/aggregations.py
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
@dataclass(frozen=True, kw_only=True)
class BorderFlowSum(BorderEdgeSum):
    """Sum of the line flows across each border, in border direction.

    A line against the border direction counts with its directions swapped:
    its `down` adds to the border `up`, its `up` to the border `down`, and
    its `net` with a minus sign. `total` (up plus down) is the same in both
    directions. The base needs a `direction` variant with the values `up`,
    `down` and, if used, `net` and `total`. Any other value raises
    `ValueError`. For `up` and `down` it fetches the swapped variant through
    `self.fetch`. It groups the lines by their areas (`layer_edge_areas`).

    For example, a line from B to A on the border A-B with up 30 and down 50
    adds 50 to the border up and 30 to the border down.
    """

    DIRECTION = 'direction'

    def check_spec(self, spec: LayerFlagSpec) -> None:
        """Raise `ValueError` if the spec has no `direction` variant."""
        if self.DIRECTION not in self._variant_names(spec):
            raise ValueError(f'Spec {spec.name!r}: {type(self).__name__} needs a {self.DIRECTION!r} variant.')

    def input_flags(
        self,
        dataset: Dataset,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
    ) -> tuple[str, ...]:
        """Return the inputs of `layer_edge_areas`, and the swapped base variant for `up` and `down`."""
        swapped = self._swapped_direction(getattr(variant, self.DIRECTION))
        swapped_flag = () if swapped is None else (variant.base_flag(**{self.DIRECTION: swapped}),)
        return layer_edge_area_inputs(dataset, layer) + swapped_flag

    def _against_border_values(self, values, dataset, variant, effective_config, params) -> pd.DataFrame:
        """The values of the lines against the border direction: the swapped direction, or `-net`, or `total`."""
        direction = getattr(variant, self.DIRECTION)
        swapped = self._swapped_direction(direction)
        if swapped is not None:
            return self.fetch(dataset, variant.base_flag(**{self.DIRECTION: swapped}), effective_config, params)
        if direction == 'net':
            return -values
        if direction == 'total':
            return values
        raise ValueError(f'{type(self).__name__}: unknown direction {direction!r}. Use up, down, net or total.')

    @staticmethod
    def _swapped_direction(direction: str) -> str | None:
        """`down` for `up` and `up` for `down`, else `None`."""
        return {'up': 'down', 'down': 'up'}.get(direction)

check_spec

check_spec(spec: LayerFlagSpec) -> None

Raise ValueError if the spec has no direction variant.

Source code in src/mesqual/layer_flags/aggregations.py
496
497
498
499
def check_spec(self, spec: LayerFlagSpec) -> None:
    """Raise `ValueError` if the spec has no `direction` variant."""
    if self.DIRECTION not in self._variant_names(spec):
        raise ValueError(f'Spec {spec.name!r}: {type(self).__name__} needs a {self.DIRECTION!r} variant.')

input_flags

input_flags(dataset: Dataset, layer: AreaLayer | BorderLayer, variant: LayerVariantValues) -> tuple[str, ...]

Return the inputs of layer_edge_areas, and the swapped base variant for up and down.

Source code in src/mesqual/layer_flags/aggregations.py
501
502
503
504
505
506
507
508
509
510
def input_flags(
    self,
    dataset: Dataset,
    layer: AreaLayer | BorderLayer,
    variant: LayerVariantValues,
) -> tuple[str, ...]:
    """Return the inputs of `layer_edge_areas`, and the swapped base variant for `up` and `down`."""
    swapped = self._swapped_direction(getattr(variant, self.DIRECTION))
    swapped_flag = () if swapped is None else (variant.base_flag(**{self.DIRECTION: swapped}),)
    return layer_edge_area_inputs(dataset, layer) + swapped_flag

BorderCapacitySum dataclass

Bases: BorderFlowSum

Same as BorderFlowSum, on line capacities.

Source code in src/mesqual/layer_flags/aggregations.py
558
559
560
@dataclass(frozen=True, kw_only=True)
class BorderCapacitySum(BorderFlowSum):
    """Same as `BorderFlowSum`, on line capacities."""

PriceSpread dataclass

Bases: BorderAggregation

Area value at the end of the border minus the value at its start. Base: an area spec.

Source code in src/mesqual/layer_flags/aggregations.py
563
564
565
566
567
568
569
570
571
572
573
574
575
576
@dataclass(frozen=True, kw_only=True)
class PriceSpread(BorderAggregation):
    """Area value at the end of the border minus the value at its start. Base: an area spec."""

    def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
        """Return one column per border: value of `area_to` minus value of `area_from`."""
        values = _require_values(self, values)
        missing = pd.Series(float('nan'), index=values.index)
        result = {
            border: values.get(row[AreaMembership.AREA_TO], missing)
            - values.get(row[AreaMembership.AREA_FROM], missing)
            for border, row in border_model.iterrows()
        }
        return pd.DataFrame(result, index=values.index).rename_axis(columns=layer.index)

aggregate

aggregate(values, border_model, dataset, layer, variant, effective_config, **params) -> DataFrame

Return one column per border: value of area_to minus value of area_from.

Source code in src/mesqual/layer_flags/aggregations.py
567
568
569
570
571
572
573
574
575
576
def aggregate(self, values, border_model, dataset, layer, variant, effective_config, **params) -> pd.DataFrame:
    """Return one column per border: value of `area_to` minus value of `area_from`."""
    values = _require_values(self, values)
    missing = pd.Series(float('nan'), index=values.index)
    result = {
        border: values.get(row[AreaMembership.AREA_TO], missing)
        - values.get(row[AreaMembership.AREA_FROM], missing)
        for border, row in border_model.iterrows()
    }
    return pd.DataFrame(result, index=values.index).rename_axis(columns=layer.index)

LayerFlagInterpreter

Bases: Dataset

Common base of AreaFlagInterpreter and BorderFlagInterpreter. Not used directly.

A subclass lists layer specs in flag_specs and is registered on a platform dataset with register_interpreter. It fetches the base flags and the layer model through parent_dataset.fetch, so overrides of the base flags reach the layer flags. It passes effective_config to each of these fetches and to aggregate, so the config of the call reaches every input, also the inputs that the aggregation fetches. It passes to each fetch only the parameters that the target spec declares, and the aggregation parameters only to aggregate.

A spec that consumes base dimensions gets values=None in aggregate: the interpreter does not fetch the base flag, and the aggregation fetches what it needs. Only a model spec on a node base always gets the base model as values.

Parameters:

Name Type Description Default
parent_dataset Dataset | None

The platform dataset.

None
**kwargs Any

Takes the interpreter arguments of any platform; not used. The platform dataset rejects unknown names.

{}
Source code in src/mesqual/layer_flags/interpreters.py
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
class LayerFlagInterpreter(Dataset):
    """Common base of `AreaFlagInterpreter` and `BorderFlagInterpreter`. Not used directly.

    A subclass lists layer specs in `flag_specs` and is registered on a
    platform dataset with `register_interpreter`. It fetches the base flags
    and the layer model through `parent_dataset.fetch`, so overrides of the
    base flags reach the layer flags. It passes `effective_config` to each
    of these fetches and to `aggregate`, so the config of the call reaches
    every input, also the inputs that the aggregation fetches. It passes to
    each fetch only the parameters that the target spec declares, and the
    aggregation parameters only to `aggregate`.

    A spec that consumes base dimensions gets `values=None` in `aggregate`:
    the interpreter does not fetch the base flag, and the aggregation fetches
    what it needs. Only a model spec on a node base always gets the base
    model as values.

    Args:
        parent_dataset: The platform dataset.
        **kwargs: Takes the interpreter arguments of any platform; not used.
            The platform dataset rejects unknown names.
    """

    def __init__(self, parent_dataset: Dataset | None = None, **kwargs: Any):
        super().__init__(parent_dataset=parent_dataset)

    @property
    def accepted_flags(self) -> set[str]:
        """One flag per matching layer and variant value of each spec in the flag index of the dataset."""
        index = self.flag_index
        specs = (index.get_spec_by_name(spec.name) for spec in self.flag_specs)
        return {flag for spec in specs if spec is not None for flag in index.static_flags_of(spec) or ()}

    def flag_is_accepted(self, flag: Any) -> bool:
        """`True` if a spec of this interpreter declares the flag on a layer of the dataset."""
        try:
            spec = self.flag_index.get_spec(flag)
        except UnknownFlagError:
            return False
        return any(spec.name == own.name for own in self.flag_specs)

    def _is_available(self, flag: Any) -> Availability | None:
        """Availability of a layer flag: the combined availability of the flags that its fetch reads.

        The inputs are the base flag, the model flags that the interpreter
        fetches (`_model_inputs`) and the `input_flags` of the aggregation.
        `Availability.of_inputs` combines them. Unavailable if the fetch
        needs a border model that the flag index does not have. `None`
        (unknown) without a parent dataset.
        """
        parent = self.parent_dataset
        if parent is None:
            return None
        index = self.flag_index
        spec, layer = index.get_spec(flag), index.get_layer(flag)
        model_inputs = self._model_inputs(spec, layer)
        if model_inputs is None:
            return Availability(False, f'no border model for layer {layer.name!r}')
        variant = index.get_layer_variant(flag)
        inputs = (index.get_base_flag(flag), *model_inputs, *spec.aggregation.input_flags(parent, layer, variant))
        return Availability.of_inputs({f: parent.availability(f) for f in dict.fromkeys(inputs)})

    @abstractmethod
    def _model_inputs(self, spec: LayerFlagSpec, layer: AreaLayer | BorderLayer) -> tuple[str, ...] | None:
        """The model flags that the fetch reads besides the base flag. `None` if a border model it needs is missing."""

    def _border_model_inputs(self, border_layer: BorderLayer | None) -> tuple[str, ...] | None:
        model_flag = None if border_layer is None else self.flag_index.get_layer_model_flag(border_layer)
        return None if model_flag is None else (model_flag,)

    def _fetch(self, flag: Any, effective_config: DatasetConfig, **params) -> pd.DataFrame:
        index = self.flag_index
        spec = index.get_spec(flag)
        if not isinstance(spec, LayerFlagSpec):
            raise ValueError(f'{type(self).__name__}: {flag!r} is not a flag of a layer spec.')
        return self._aggregate(spec, index.get_layer(flag), index.get_layer_variant(flag), effective_config, params)

    @abstractmethod
    def _aggregate(
        self,
        spec: LayerFlagSpec,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
        effective_config: DatasetConfig,
        params: dict[str, Any],
    ) -> pd.DataFrame:
        """Fetch the inputs of the layer flag with `effective_config` and call the aggregation of `spec`."""

    def _base_values(
        self, spec: LayerFlagSpec, variant: LayerVariantValues, effective_config: DatasetConfig, params
    ) -> pd.DataFrame | None:
        """The base flag for the variant values, or `None` if the spec consumes base dimensions."""
        if spec.consumes:
            return None
        return LayerAggregation.fetch(self.parent_dataset, variant.base_flag(), effective_config, params)

    def _aggregate_on_border_model(
        self,
        spec: LayerFlagSpec,
        layer: AreaLayer | BorderLayer,
        variant: LayerVariantValues,
        effective_config: DatasetConfig,
        params: dict[str, Any],
        border_layer: BorderLayer | None,
    ) -> pd.DataFrame:
        """Aggregate the base values with the border model of `border_layer` (`None`: no border model)."""
        border_model = None if border_layer is None else self._layer_model(border_layer, effective_config)
        values = self._base_values(spec, variant, effective_config, params)
        return spec.aggregation.aggregate(
            values, border_model, self.parent_dataset, layer, variant, effective_config, **params
        )

    def _layer_model(self, layer: AreaLayer | BorderLayer, effective_config: DatasetConfig) -> pd.DataFrame:
        model_flag = self.flag_index.get_layer_model_flag(layer)
        if model_flag is None:
            raise ValueError(
                f'Layer {layer.name!r} has no model flag. Declare a BorderFlagSpec with BorderModel() on the edge model.'
            )
        return self.parent_dataset.fetch(model_flag, effective_config)

accepted_flags property

accepted_flags: set[str]

One flag per matching layer and variant value of each spec in the flag index of the dataset.

flag_is_accepted

flag_is_accepted(flag: Any) -> bool

True if a spec of this interpreter declares the flag on a layer of the dataset.

Source code in src/mesqual/layer_flags/interpreters.py
53
54
55
56
57
58
59
def flag_is_accepted(self, flag: Any) -> bool:
    """`True` if a spec of this interpreter declares the flag on a layer of the dataset."""
    try:
        spec = self.flag_index.get_spec(flag)
    except UnknownFlagError:
        return False
    return any(spec.name == own.name for own in self.flag_specs)

AreaFlagInterpreter

Bases: LayerFlagInterpreter

Answers the flags of AreaFlagSpecs.

On a node or node-connected base, it fetches the base flag and the base model, takes the mapping from the layer membership column, leaves out the objects without area, and calls the aggregation. The aggregation maps one object level: each column of the base is one object. A flag with several column levels from node values comes from an interpreter on the layer that builds it from its inputs, not from an aggregation of an object flag.

On a border base, it fetches the border model of the matching border layer as the mapping. The aggregation may build several column levels, for example a trade balance per area and partner.

Source code in src/mesqual/layer_flags/interpreters.py
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
class AreaFlagInterpreter(LayerFlagInterpreter):
    """Answers the flags of `AreaFlagSpec`s.

    On a node or node-connected base, it fetches the base flag and the base
    model, takes the mapping from the layer membership column, leaves out
    the objects without area, and calls the aggregation. The aggregation
    maps one object level: each column of the base is one object. A flag
    with several column levels from node values comes from an interpreter
    on the layer that builds it from its inputs, not from an aggregation of
    an object flag.

    On a border base, it fetches the border model of the matching border
    layer as the mapping. The aggregation may build several column levels,
    for example a trade balance per area and partner.
    """

    def _aggregate(self, spec: AreaFlagSpec, layer, variant, effective_config, params) -> pd.DataFrame:
        if spec._has_layer_base:
            border_layer = self.flag_index.get_base_layer(spec, layer)
            return self._aggregate_on_border_model(spec, layer, variant, effective_config, params, border_layer)
        model = LayerAggregation.fetch(self.parent_dataset, layer.base_model, effective_config, params)
        mapping = self._mapping(model, layer)
        values = model if spec.is_model else self._base_values(spec, variant, effective_config, params)
        if values is not None:
            values = (
                values.loc[values.index.intersection(mapping.index, sort=False)]
                if spec.is_model
                else (values[values.columns.intersection(mapping.index, sort=False)])
            )
        return spec.aggregation.aggregate(
            values, mapping, self.parent_dataset, layer, variant, effective_config, **params
        )

    def _model_inputs(self, spec: AreaFlagSpec, layer: AreaLayer) -> tuple[str, ...] | None:
        """The base model of the layer, or for a border base the border model of the matching border layer."""
        if spec._has_layer_base:
            return self._border_model_inputs(self.flag_index.get_base_layer(spec, layer))
        return (layer.base_model,)

    def _mapping(self, model: pd.DataFrame, layer: AreaLayer) -> pd.Series:
        areas = model[self.flag_index.get_mapping_column(layer)]
        return areas[AreaMembership.has_area(areas)]

BorderFlagInterpreter

Bases: LayerFlagInterpreter

Answers the flags of BorderFlagSpecs.

It fetches the base flag (with the area layer of the border layer for an area base) and the border model of the layer, and calls the aggregation. For the border model flag itself, it passes the edge model as values and no border model.

Source code in src/mesqual/layer_flags/interpreters.py
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
class BorderFlagInterpreter(LayerFlagInterpreter):
    """Answers the flags of `BorderFlagSpec`s.

    It fetches the base flag (with the area layer of the border layer for an
    area base) and the border model of the layer, and calls the aggregation.
    For the border model flag itself, it passes the edge model as values and
    no border model.
    """

    def _aggregate(self, spec: BorderFlagSpec, layer, variant, effective_config, params) -> pd.DataFrame:
        border_layer = None if spec.is_model else layer
        return self._aggregate_on_border_model(spec, layer, variant, effective_config, params, border_layer)

    def _model_inputs(self, spec: BorderFlagSpec, layer: BorderLayer) -> tuple[str, ...] | None:
        """The border model of the layer. Empty for the border model flag itself."""
        return () if spec.is_model else self._border_model_inputs(layer)

layer_edge_area_inputs

layer_edge_area_inputs(dataset: Dataset, layer: BorderLayer) -> tuple[str, str]

The flags that layer_edge_areas fetches: the base model of the area layer and the edge model of layer.

Source code in src/mesqual/layer_flags/aggregations.py
530
531
532
def layer_edge_area_inputs(dataset: Dataset, layer: BorderLayer) -> tuple[str, str]:
    """The flags that `layer_edge_areas` fetches: the base model of the area layer and the edge model of `layer`."""
    return dataset.flag_index.layers.area_layer_of(layer).base_model, layer.edge_model

layer_edge_areas

layer_edge_areas(dataset: Dataset, layer: BorderLayer, effective_config: DatasetConfig | None, params: dict[str, Any]) -> DataFrame

AreaMembership.edge_areas for the edge model of layer, with the areas of its area layer.

It fetches the flags of layer_edge_area_inputs with LayerAggregation.fetch.

Parameters:

Name Type Description Default
effective_config DatasetConfig | None

The config for the fetches: the effective config of the enclosing fetch, or None for the config of dataset (outside a fetch).

required
Source code in src/mesqual/layer_flags/aggregations.py
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
def layer_edge_areas(
    dataset: Dataset,
    layer: BorderLayer,
    effective_config: DatasetConfig | None,
    params: dict[str, Any],
) -> pd.DataFrame:
    """`AreaMembership.edge_areas` for the edge model of `layer`, with the areas of its area layer.

    It fetches the flags of `layer_edge_area_inputs` with `LayerAggregation.fetch`.

    Args:
        effective_config: The config for the fetches: the effective config
            of the enclosing fetch, or `None` for the config of `dataset`
            (outside a fetch).
    """
    index = dataset.flag_index
    area_layer = index.layers.area_layer_of(layer)
    node_model, edge_model = layer_edge_area_inputs(dataset, layer)
    nodes = LayerAggregation.fetch(dataset, node_model, effective_config, params)
    edges = LayerAggregation.fetch(dataset, edge_model, effective_config, params)
    return AreaMembership.edge_areas(edges, nodes[index.get_mapping_column(area_layer)], *index.get_edge_ends(layer))

edges_on_no_border

edges_on_no_border(dataset: Dataset, layer: BorderLayer) -> int

Return the number of edges of the edge model of layer that cross no border.

An edge with both ends in one area, or with an end without area, is on no border. It fetches model flags only.

Parameters:

Name Type Description Default
dataset Dataset

A dataset whose model results have no dataset level, for example a platform dataset or a merge collection.

required
layer BorderLayer

The border layer.

required

Raises:

Type Description
TypeError

The edge model or the base model of the area layer of dataset has a dataset level, for example on a concat collection.

Examples:

>>> from mesqual.testing import DummyScenario, make_dummy_dataset
>>> dataset = make_dummy_dataset(DummyScenario(name='base', node_without_country=True))
>>> edges_on_no_border(dataset, dataset.flag_index.get_layer('CountryBorder.Model'))
2
Source code in src/mesqual/layer_flags/coverage.py
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
def edges_on_no_border(dataset: Dataset, layer: BorderLayer) -> int:
    """Return the number of edges of the edge model of `layer` that cross no border.

    An edge with both ends in one area, or with an end without area, is on
    no border. It fetches model flags only.

    Args:
        dataset: A dataset whose model results have no dataset level, for
            example a platform dataset or a merge collection.
        layer: The border layer.

    Raises:
        TypeError: The edge model or the base model of the area layer of
            `dataset` has a dataset level, for example on a concat collection.

    Examples:
        >>> from mesqual.testing import DummyScenario, make_dummy_dataset
        >>> dataset = make_dummy_dataset(DummyScenario(name='base', node_without_country=True))
        >>> edges_on_no_border(dataset, dataset.flag_index.get_layer('CountryBorder.Model'))
        2
    """
    _check_no_dataset_level(dataset, layer.edge_model)
    _check_no_dataset_level(dataset, dataset.flag_index.layers.area_layer_of(layer).base_model)
    areas = layer_edge_areas(dataset, layer, None, {})
    return int((~AreaMembership.crosses_border(areas)).sum())

objects_without_area

objects_without_area(dataset: Dataset, layer: AreaLayer) -> int

Return the number of objects of the base model of layer that belong to no area.

An object with an empty value in the layer membership column belongs to no area. It fetches the base model only.

Parameters:

Name Type Description Default
dataset Dataset

A dataset whose model results have no dataset level, for example a platform dataset or a merge collection.

required
layer AreaLayer

The area layer.

required

Raises:

Type Description
TypeError

The base model of dataset has a dataset level, for example on a concat collection.

Examples:

>>> from mesqual.testing import DummyScenario, make_dummy_dataset
>>> dataset = make_dummy_dataset(DummyScenario(name='base', node_without_country=True))
>>> objects_without_area(dataset, dataset.flag_index.get_layer('Country.Results.price'))
1
Source code in src/mesqual/layer_flags/coverage.py
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
def objects_without_area(dataset: Dataset, layer: AreaLayer) -> int:
    """Return the number of objects of the base model of `layer` that belong to no area.

    An object with an empty value in the layer membership column belongs to
    no area. It fetches the base model only.

    Args:
        dataset: A dataset whose model results have no dataset level, for
            example a platform dataset or a merge collection.
        layer: The area layer.

    Raises:
        TypeError: The base model of `dataset` has a dataset level, for
            example on a concat collection.

    Examples:
        >>> from mesqual.testing import DummyScenario, make_dummy_dataset
        >>> dataset = make_dummy_dataset(DummyScenario(name='base', node_without_country=True))
        >>> objects_without_area(dataset, dataset.flag_index.get_layer('Country.Results.price'))
        1
    """
    _check_no_dataset_level(dataset, layer.base_model)
    areas = dataset.fetch(layer.base_model)[dataset.flag_index.get_mapping_column(layer)]
    return int((~AreaMembership.has_area(areas)).sum())