Skip to content

Area Variable Accounting

AreaPriceCalculator

Bases: AreaVariableCalculatorBase

Area prices from node prices: the mean or the weighted mean per area.

Weights are for example the node demand or the node generation. To leave out a node, for example a virtual node, drop its column from the prices.

Parameters:

Name Type Description Default
node_model_df DataFrame

DataFrame with node-area mappings

required
area_column str

Column name containing area identifiers

required

Examples:

>>> calculator = AreaPriceCalculator(dataset.fetch('Node.Model'), 'country')
>>> prices = dataset.fetch('Node.Results.price')
>>> calculator.calculate(prices).iloc[:2].round(2)
country                       C1     C2
time
2024-03-28 00:00:00+01:00  43.28  41.71
2024-03-28 01:00:00+01:00  43.40  42.23
>>> demand = dataset.fetch('Node.Results.demand_hourly')
>>> calculator.calculate(prices, weighting_factor_df=demand).iloc[:2].round(2)  # demand-weighted
country                       C1     C2
time
2024-03-28 00:00:00+01:00  44.18  42.32
2024-03-28 01:00:00+01:00  44.34  43.23
Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_price_calculator.py
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
class AreaPriceCalculator(AreaVariableCalculatorBase):
    """Area prices from node prices: the mean or the weighted mean per area.

    Weights are for example the node demand or the node generation. To leave
    out a node, for example a virtual node, drop its column from the prices.

    Args:
        node_model_df: DataFrame with node-area mappings
        area_column: Column name containing area identifiers

    Examples:
        >>> calculator = AreaPriceCalculator(dataset.fetch('Node.Model'), 'country')
        >>> prices = dataset.fetch('Node.Results.price')
        >>> calculator.calculate(prices).iloc[:2].round(2)
        country                       C1     C2
        time
        2024-03-28 00:00:00+01:00  43.28  41.71
        2024-03-28 01:00:00+01:00  43.40  42.23
        >>> demand = dataset.fetch('Node.Results.demand_hourly')
        >>> calculator.calculate(prices, weighting_factor_df=demand).iloc[:2].round(2)  # demand-weighted
        country                       C1     C2
        time
        2024-03-28 00:00:00+01:00  44.18  42.32
        2024-03-28 01:00:00+01:00  44.34  43.23
    """

    def calculate(
        self,
        node_price_df: pd.DataFrame,
        weighting_factor_df: pd.DataFrame = None,
    ) -> pd.DataFrame:
        """Calculate area prices with different weighting options.

        Aggregates node-level prices to area-level using simple averaging (when no weights
        provided) or weighted averaging (when weights provided). A node of the node model
        that is not in `node_price_df` is left out. A column that is not a node of the
        node model is left out, and a warning is logged.

        Args:
            node_price_df: Node-level price time series with datetime index and node columns.
                Values represent electricity prices in €/MWh or similar units.
            weighting_factor_df: Optional weighting factor DataFrame with same structure as
                node_price_df, for example the node demand or supply.
                If None, simple arithmetic average is used. A time step where
                the weights of an area sum to zero gives NaN for that area.

        Returns:
            DataFrame with area-level prices. Index matches input time series, columns
                represent areas with prices in same units as input.

        Raises:
            KeyError: If a node of `node_price_df` is missing from weighting_factor_df
        """
        self._validate_node_data(node_price_df, 'node_price_df')

        area_prices = {}

        for area in self.areas:
            area_nodes = self.get_area_nodes(area)
            area_nodes = [n for n in area_nodes if n in node_price_df.columns]

            if not area_nodes:
                continue

            prices = node_price_df[area_nodes]

            if weighting_factor_df is None:
                area_prices[area] = self._calculate_simple_average(prices)
            else:
                self._validate_node_data(weighting_factor_df, 'weighting_factor_df')
                area_prices[area] = self._calculate_weighted_average(prices, weighting_factor_df[area_nodes])

        result = pd.DataFrame(area_prices)
        result.columns.name = self.area_column
        return result

    def _calculate_simple_average(self, prices: pd.DataFrame) -> pd.Series:
        return prices.mean(axis=1)

    def _calculate_weighted_average(self, prices: pd.DataFrame, weights: pd.DataFrame) -> pd.Series:
        """Calculate weighted average of prices using provided weights.

        Computes volume-weighted or otherwise weighted prices. A time period where
        the weights sum to zero has no weighted price and gives NaN. When all prices
        are NaN for a time period, the result is also NaN.

        Args:
            prices: DataFrame with price time series for nodes in an area
            weights: DataFrame with weighting factors (e.g., demand, supply) with same
                structure as prices. Must have non-negative values.

        Returns:
            Series with weighted average prices over time

        Note:
            This method assumes weights are extensive quantities (like energy volumes)
            while prices are intensive quantities (like €/MWh).
        """
        weighted_sum = (prices * weights).sum(axis=1)
        weight_sum = weights.sum(axis=1).replace(0, np.nan)
        weighted_price = weighted_sum / weight_sum
        weighted_price[prices.isna().all(axis=1)] = np.nan
        return weighted_price

calculate

calculate(node_price_df: DataFrame, weighting_factor_df: DataFrame = None) -> DataFrame

Calculate area prices with different weighting options.

Aggregates node-level prices to area-level using simple averaging (when no weights provided) or weighted averaging (when weights provided). A node of the node model that is not in node_price_df is left out. A column that is not a node of the node model is left out, and a warning is logged.

Parameters:

Name Type Description Default
node_price_df DataFrame

Node-level price time series with datetime index and node columns. Values represent electricity prices in €/MWh or similar units.

required
weighting_factor_df DataFrame

Optional weighting factor DataFrame with same structure as node_price_df, for example the node demand or supply. If None, simple arithmetic average is used. A time step where the weights of an area sum to zero gives NaN for that area.

None

Returns:

Type Description
DataFrame

DataFrame with area-level prices. Index matches input time series, columns represent areas with prices in same units as input.

Raises:

Type Description
KeyError

If a node of node_price_df is missing from weighting_factor_df

Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_price_calculator.py
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
def calculate(
    self,
    node_price_df: pd.DataFrame,
    weighting_factor_df: pd.DataFrame = None,
) -> pd.DataFrame:
    """Calculate area prices with different weighting options.

    Aggregates node-level prices to area-level using simple averaging (when no weights
    provided) or weighted averaging (when weights provided). A node of the node model
    that is not in `node_price_df` is left out. A column that is not a node of the
    node model is left out, and a warning is logged.

    Args:
        node_price_df: Node-level price time series with datetime index and node columns.
            Values represent electricity prices in €/MWh or similar units.
        weighting_factor_df: Optional weighting factor DataFrame with same structure as
            node_price_df, for example the node demand or supply.
            If None, simple arithmetic average is used. A time step where
            the weights of an area sum to zero gives NaN for that area.

    Returns:
        DataFrame with area-level prices. Index matches input time series, columns
            represent areas with prices in same units as input.

    Raises:
        KeyError: If a node of `node_price_df` is missing from weighting_factor_df
    """
    self._validate_node_data(node_price_df, 'node_price_df')

    area_prices = {}

    for area in self.areas:
        area_nodes = self.get_area_nodes(area)
        area_nodes = [n for n in area_nodes if n in node_price_df.columns]

        if not area_nodes:
            continue

        prices = node_price_df[area_nodes]

        if weighting_factor_df is None:
            area_prices[area] = self._calculate_simple_average(prices)
        else:
            self._validate_node_data(weighting_factor_df, 'weighting_factor_df')
            area_prices[area] = self._calculate_weighted_average(prices, weighting_factor_df[area_nodes])

    result = pd.DataFrame(area_prices)
    result.columns.name = self.area_column
    return result

AreaSumCalculator

Bases: AreaVariableCalculatorBase

Area sums of node values, for extensive quantities such as demand or generation.

To leave out a node, for example a virtual node, drop its column from the data.

Parameters:

Name Type Description Default
node_model_df DataFrame

DataFrame mapping nodes to areas

required
area_column str

Column name containing area identifiers

required

Examples:

>>> calculator = AreaSumCalculator(dataset.fetch('Node.Model'), 'country')
>>> calculator.calculate(dataset.fetch('Node.Results.demand_hourly')).iloc[:2].round(2)
country                         C1       C2
time
2024-03-28 00:00:00+01:00  1377.90  1583.60
2024-03-28 01:00:00+01:00  1414.62  1628.73
Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_sum_calculator.py
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
class AreaSumCalculator(AreaVariableCalculatorBase):
    """Area sums of node values, for extensive quantities such as demand or generation.

    To leave out a node, for example a virtual node, drop its column from the
    data.

    Args:
        node_model_df: DataFrame mapping nodes to areas
        area_column: Column name containing area identifiers

    Examples:
        >>> calculator = AreaSumCalculator(dataset.fetch('Node.Model'), 'country')
        >>> calculator.calculate(dataset.fetch('Node.Results.demand_hourly')).iloc[:2].round(2)
        country                         C1       C2
        time
        2024-03-28 00:00:00+01:00  1377.90  1583.60
        2024-03-28 01:00:00+01:00  1414.62  1628.73
    """

    def calculate(self, node_data_df: pd.DataFrame) -> pd.DataFrame:
        """Calculate area sums from node-level extensive quantity data.

        Sums node-level values within each area to create area-level aggregates.
        This method is designed for extensive quantities where summation is the
        appropriate aggregation method (e.g., generation, demand, volumes).

        A node of the node model that is not in the data is left out. A column
        that is not a node of the node model is left out, and a warning is
        logged. An area without nodes in the data has no column. NaN values are left out of the
        sum. A time step where all nodes of an area are NaN gives NaN for that
        area. The result has the index of `node_data_df`, also when
        `node_data_df` has no columns.

        Args:
            node_data_df: DataFrame with node-level time series data. Index should
                be datetime, columns should be node identifiers. Values represent
                extensive quantities (MW, MWh, etc.) that should be summed.

        Returns:
            DataFrame with area-level aggregated data. Index matches input time series,
                columns represent areas. Units are preserved from input data.
        """

        self._validate_node_data(node_data_df, 'node_data_df')

        area_sums = {}
        for area in self.areas:
            area_nodes = self.get_area_nodes(area)
            area_nodes = [n for n in area_nodes if n in node_data_df.columns]
            if area_nodes:
                area_sums[area] = node_data_df[area_nodes].sum(axis=1, min_count=1)

        columns = pd.Index(list(area_sums), name=self.area_column)
        return pd.DataFrame(area_sums, index=node_data_df.index, columns=columns)

calculate

calculate(node_data_df: DataFrame) -> DataFrame

Calculate area sums from node-level extensive quantity data.

Sums node-level values within each area to create area-level aggregates. This method is designed for extensive quantities where summation is the appropriate aggregation method (e.g., generation, demand, volumes).

A node of the node model that is not in the data is left out. A column that is not a node of the node model is left out, and a warning is logged. An area without nodes in the data has no column. NaN values are left out of the sum. A time step where all nodes of an area are NaN gives NaN for that area. The result has the index of node_data_df, also when node_data_df has no columns.

Parameters:

Name Type Description Default
node_data_df DataFrame

DataFrame with node-level time series data. Index should be datetime, columns should be node identifiers. Values represent extensive quantities (MW, MWh, etc.) that should be summed.

required

Returns:

Type Description
DataFrame

DataFrame with area-level aggregated data. Index matches input time series, columns represent areas. Units are preserved from input data.

Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_sum_calculator.py
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
def calculate(self, node_data_df: pd.DataFrame) -> pd.DataFrame:
    """Calculate area sums from node-level extensive quantity data.

    Sums node-level values within each area to create area-level aggregates.
    This method is designed for extensive quantities where summation is the
    appropriate aggregation method (e.g., generation, demand, volumes).

    A node of the node model that is not in the data is left out. A column
    that is not a node of the node model is left out, and a warning is
    logged. An area without nodes in the data has no column. NaN values are left out of the
    sum. A time step where all nodes of an area are NaN gives NaN for that
    area. The result has the index of `node_data_df`, also when
    `node_data_df` has no columns.

    Args:
        node_data_df: DataFrame with node-level time series data. Index should
            be datetime, columns should be node identifiers. Values represent
            extensive quantities (MW, MWh, etc.) that should be summed.

    Returns:
        DataFrame with area-level aggregated data. Index matches input time series,
            columns represent areas. Units are preserved from input data.
    """

    self._validate_node_data(node_data_df, 'node_data_df')

    area_sums = {}
    for area in self.areas:
        area_nodes = self.get_area_nodes(area)
        area_nodes = [n for n in area_nodes if n in node_data_df.columns]
        if area_nodes:
            area_sums[area] = node_data_df[area_nodes].sum(axis=1, min_count=1)

    columns = pd.Index(list(area_sums), name=self.area_column)
    return pd.DataFrame(area_sums, index=node_data_df.index, columns=columns)

AreaVariableCalculatorBase

Bases: ABC, AreaMembership

Base class of the calculators that aggregate node data to areas.

It maps the nodes to their areas and checks the node data. A subclass implements calculate, for example a sum (AreaSumCalculator) or a mean (AreaPriceCalculator).

Parameters:

Name Type Description Default
node_model_df DataFrame

DataFrame containing node information with area assignments. Index should be node identifiers, must contain the specified area_column.

required
area_column str

Name of the column in node_model_df that contains area assignments. Each node has at most one area. A node without an area (AreaMembership.has_area) belongs to no area.

required

Attributes:

Name Type Description
node_model_df

The input node model DataFrame

area_column

Name of the area assignment column

areas

Sorted list of the unique areas of the nodes with an area

Raises:

Type Description
ValueError

If area_column is not found in node_model_df

Examples:

>>> from mesqual.energy_data_handling.area_accounting import AreaSumCalculator
>>> calculator = AreaSumCalculator(dataset.fetch('Node.Model'), 'country')
>>> calculator.areas
['C1', 'C2']
>>> calculator.get_area_nodes('C1')
['N1', 'N3']
Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_base.py
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
class AreaVariableCalculatorBase(ABC, AreaMembership):
    """Base class of the calculators that aggregate node data to areas.

    It maps the nodes to their areas and checks the node data. A subclass
    implements `calculate`, for example a sum (`AreaSumCalculator`) or a
    mean (`AreaPriceCalculator`).

    Args:
        node_model_df: DataFrame containing node information with area assignments.
            Index should be node identifiers, must contain the specified area_column.
        area_column: Name of the column in node_model_df that contains area assignments.
            Each node has at most one area. A node without an area
            (`AreaMembership.has_area`) belongs to no area.

    Attributes:
        node_model_df: The input node model DataFrame
        area_column: Name of the area assignment column
        areas: Sorted list of the unique areas of the nodes with an area

    Raises:
        ValueError: If area_column is not found in node_model_df

    Examples:
        >>> from mesqual.energy_data_handling.area_accounting import AreaSumCalculator
        >>> calculator = AreaSumCalculator(dataset.fetch('Node.Model'), 'country')
        >>> calculator.areas
        ['C1', 'C2']
        >>> calculator.get_area_nodes('C1')
        ['N1', 'N3']
    """

    def __init__(self, node_model_df: pd.DataFrame, area_column: str):
        """Initialize the area variable calculator.

        Args:
            node_model_df: DataFrame with node-to-area mapping
            area_column: Column name containing area assignments

        Raises:
            ValueError: If area_column not found in node_model_df
        """
        self.node_model_df = node_model_df
        self.area_column = area_column
        self._node_areas = self.get_node_areas()
        self.areas = sorted(self._node_areas.unique())

    def get_area_nodes(self, area: str) -> list[str]:
        """Get all nodes belonging to a specific area.

        Args:
            area: Area name to get nodes for

        Returns:
            List of node IDs that belong to the specified area. Empty for a
                value that is no area (`AreaMembership.has_area`).
        """
        return self._node_areas.index[self._node_areas == area].tolist()

    @abstractmethod
    def calculate(self, **kwargs: Any) -> pd.DataFrame:
        """Calculate the area variable. Must be implemented by subclasses.

        This method should contain the specific logic for aggregating node-level
        data to area level for the particular variable type. The implementation
        will vary depending on whether the variable is extensive (additive like
        energy volumes) or intensive (averaged like prices).

        Args:
            **kwargs: Variable-specific parameters for the calculation

        Returns:
            DataFrame with area-level aggregated data. Index should be datetime
                for time series data, columns should be area identifiers.
        """
        pass

    def _validate_node_data(self, node_df: pd.DataFrame, data_name: str):
        """Validate that required nodes are present in node_model_df.

        Logs warnings for any nodes found in the data that are not in the node model.
        This is important for detecting data inconsistencies or model updates.

        Args:
            node_df: DataFrame containing node-level data to validate
            data_name: Descriptive name of the data being validated (for logging)
        """
        missing_nodes = set(node_df.columns) - set(self.node_model_df.index)
        if missing_nodes:
            logger.warning(f'{len(missing_nodes)} nodes missing in node_model_df from {data_name}')

__init__

__init__(node_model_df: DataFrame, area_column: str)

Initialize the area variable calculator.

Parameters:

Name Type Description Default
node_model_df DataFrame

DataFrame with node-to-area mapping

required
area_column str

Column name containing area assignments

required

Raises:

Type Description
ValueError

If area_column not found in node_model_df

Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_base.py
42
43
44
45
46
47
48
49
50
51
52
53
54
55
def __init__(self, node_model_df: pd.DataFrame, area_column: str):
    """Initialize the area variable calculator.

    Args:
        node_model_df: DataFrame with node-to-area mapping
        area_column: Column name containing area assignments

    Raises:
        ValueError: If area_column not found in node_model_df
    """
    self.node_model_df = node_model_df
    self.area_column = area_column
    self._node_areas = self.get_node_areas()
    self.areas = sorted(self._node_areas.unique())

get_area_nodes

get_area_nodes(area: str) -> list[str]

Get all nodes belonging to a specific area.

Parameters:

Name Type Description Default
area str

Area name to get nodes for

required

Returns:

Type Description
list[str]

List of node IDs that belong to the specified area. Empty for a value that is no area (AreaMembership.has_area).

Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_base.py
57
58
59
60
61
62
63
64
65
66
67
def get_area_nodes(self, area: str) -> list[str]:
    """Get all nodes belonging to a specific area.

    Args:
        area: Area name to get nodes for

    Returns:
        List of node IDs that belong to the specified area. Empty for a
            value that is no area (`AreaMembership.has_area`).
    """
    return self._node_areas.index[self._node_areas == area].tolist()

calculate abstractmethod

calculate(**kwargs: Any) -> DataFrame

Calculate the area variable. Must be implemented by subclasses.

This method should contain the specific logic for aggregating node-level data to area level for the particular variable type. The implementation will vary depending on whether the variable is extensive (additive like energy volumes) or intensive (averaged like prices).

Parameters:

Name Type Description Default
**kwargs Any

Variable-specific parameters for the calculation

{}

Returns:

Type Description
DataFrame

DataFrame with area-level aggregated data. Index should be datetime for time series data, columns should be area identifiers.

Source code in src/mesqual/energy_data_handling/area_accounting/area_variable_base.py
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
@abstractmethod
def calculate(self, **kwargs: Any) -> pd.DataFrame:
    """Calculate the area variable. Must be implemented by subclasses.

    This method should contain the specific logic for aggregating node-level
    data to area level for the particular variable type. The implementation
    will vary depending on whether the variable is extensive (additive like
    energy volumes) or intensive (averaged like prices).

    Args:
        **kwargs: Variable-specific parameters for the calculation

    Returns:
        DataFrame with area-level aggregated data. Index should be datetime
            for time series data, columns should be area identifiers.
    """
    pass