Skip to content

MESQUAL Pandas Utils

pandas_utils

MESQUAL pandas utilities for energy systems data analysis.

This package provides specialized pandas utilities designed for energy systems analysis workflows in the MESQUAL framework. These utilities are optimized for working with MultiIndex data structures that are common in multi-scenario energy modeling studies.

The utilities support MESQUAL's core data flow patterns where energy data is typically structured with multiple dimensions (scenarios, time periods, network components, technologies) and requires specialized operations for filtering, transformation, and aggregation.

Modules:

Name Description
prepend_model_prop_levels

Add model properties as MultiIndex levels to time-series data
filter_by_model_query

Filter time-series using model metadata queries
flatten_df

Convert MultiIndex DataFrames to flat format for visualization
sort_multiindex

Sort MultiIndex levels with custom ordering
xs_df

Enhanced cross-section interface for MultiIndex DataFrames
merge_multi_index_levels

Combine multiple MultiIndex levels into single level
add_index_as_column

Convert index levels to DataFrame columns

Examples:

Basic usage for energy systems analysis:
>>> from mesqual.utils.pandas_utils import prepend_model_prop_levels, filter_by_model_query
>>>
>>> # Load generator model and time-series data
>>> generators = study.scen.fetch('generators')  # Model metadata
>>> generation = study.scen.fetch('generators_t.p')  # Time-series data
>>>
>>> # Add technology and zone properties to time-series
>>> gen_with_props = prepend_model_prop_levels(
...     generation, generators, 'technology', 'zone'
... )
>>>
>>> # Filter for renewable generators only
>>> renewable_gen = filter_by_model_query(
...     gen_with_props, generators, 'technology.isin(["solar", "wind"])'
... )
>>>
>>> # Aggregate by technology and zone
>>> tech_zone_totals = renewable_gen.T.groupby(level=['technology', 'zone']).sum().T
Architecture Integration

These utilities integrate seamlessly with MESQUAL's three-tier architecture:

  • General utilities (this package): Platform-agnostic data transformations
  • Platform-specific: Used by platform interfaces (mesqual-pypsa, etc.)
  • Study-specific: Extended in individual studies for custom analysis

They preserve MESQUAL's MultiIndex data flow patterns while enabling flexible data manipulation and transformation for energy systems analysis workflows.

Performance Notes
  • Operations maintain MultiIndex structures for memory efficiency
  • Bulk operations are preferred over iterative transformations
  • Query-based filtering minimizes data copying
  • Lazy evaluation patterns supported where possible