Granularity Modules¶
TimeSeriesGranularityAnalyzer
¶
Analyzes and validates time granularity in DatetimeIndex sequences.
It detects the granularity of a time series, also per day. A series can have mixed granularities; strict mode (the default) raises an error for them, the other mode warns.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict_mode
|
bool
|
If True, raises GranularityError for multiple granularities, duplicate timestamps and days with one row. If False, issues warnings, and a day with one row gives NaT. Default is True. |
True
|
Examples:
>>> analyzer = TimeSeriesGranularityAnalyzer()
>>> index = pd.date_range('2024-01-01', periods=24, freq='h')
>>> analyzer.get_granularity_as_timedelta(index)
Timedelta('0 days 01:00:00')
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 | |
strict_mode
property
writable
¶
strict_mode: bool
If True, the analyzer raises GranularityError where it else issues a warning. Can be set.
get_granularity_as_series_of_timedeltas
¶
get_granularity_as_series_of_timedeltas(dt_index: DatetimeIndex) -> Series
Return the step length of each row, detected per day.
The step of a row is the gap to the next row of the same day; the
last row of a day takes the step before it. A day is the calendar
date in the timezone of the index. The index may be unsorted: the
steps are computed on the sorted unique timestamps and returned in
the order of dt_index.
Without strict mode, duplicate timestamps and mixed step lengths
issue a warning. The first occurrence of a timestamp in dt_index
gets its step, and each later occurrence gets 0, so a sum of the
steps counts each timestamp once. A day with one row gives NaT.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dt_index
|
DatetimeIndex
|
Timestamps, or values that convert to a |
required |
Returns:
| Type | Description |
|---|---|
Series
|
Series of |
Raises:
| Type | Description |
|---|---|
ValueError
|
The index contains |
GranularityError
|
Strict mode, and the index has duplicate timestamps, a day has mixed step lengths, or a day has one row. |
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | |
get_granularity_as_series_of_minutes
¶
get_granularity_as_series_of_minutes(dt_index: DatetimeIndex) -> Series
Return the step length of each row in minutes, NaN for a step of NaT.
The rules of get_granularity_as_series_of_timedeltas apply.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
102 103 104 105 106 107 108 | |
get_granularity_as_series_of_hours
¶
get_granularity_as_series_of_hours(dt_index: DatetimeIndex) -> Series
Return the step length of each row in hours, NaN for a step of NaT.
The rules of get_granularity_as_series_of_timedeltas apply.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
110 111 112 113 114 115 116 | |
get_granularity_as_timedelta
¶
get_granularity_as_timedelta(dt_index: DatetimeIndex) -> Timedelta
Return the one step length of the index.
The reference step is the step of the earliest timestamp, so the
result does not depend on the row order. Repeated timestamps do not
count as another step length. Without strict mode, a reference
step of NaT (the earliest day has one row) issues a warning and
the result is NaT. The rules of
get_granularity_as_series_of_timedeltas apply.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dt_index
|
DatetimeIndex
|
Timestamps, or values that convert to a |
required |
Returns:
| Type | Description |
|---|---|
Timedelta
|
The step length. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The index contains |
GranularityError
|
Strict mode, and the index has more than one
step length, or a rule of |
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | |
get_granularity_as_hours
¶
get_granularity_as_hours(dt_index: DatetimeIndex) -> float
Return the one step length of the index in hours, NaN for NaT.
The rules of get_granularity_as_timedelta apply.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
167 168 169 170 171 172 173 | |
get_granularity_as_minutes
¶
get_granularity_as_minutes(dt_index: DatetimeIndex) -> float
Return the one step length of the index in minutes, NaN for NaT.
The rules of get_granularity_as_timedelta apply.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
175 176 177 178 179 180 181 | |
validate_constant_granularity
¶
validate_constant_granularity(dt_index: DatetimeIndex, expected_hours: float) -> bool
Return True if the step length of the index is expected_hours, within 1e-10 hours.
The step length is get_granularity_as_hours. In strict mode, an
index with more than one step length raises GranularityError.
Without strict mode, it issues a warning, and only the step of the
earliest timestamp is compared.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
183 184 185 186 187 188 189 190 191 192 193 | |
TimeSeriesGranularityConverter
¶
Converts time series between granularities with the quantity type of the data.
- Upsampling (coarser to finer) repeats each row
ntimes, withn = native step / target step.INTENSIVE(prices, power) keeps the value.EXTENSIVE(volumes, welfare) divides it byn, so the sum stays. - Downsampling (finer to coarser) takes the mean (
INTENSIVE) or the sum (EXTENSIVE) per target step. A target step without input rows gives no row.
A gap is a missing timestamp. It stays a gap: the converter never fills forward. A NaN value is a value: upsampling repeats it as NaN, and a target step whose rows are all NaN gives NaN.
The native step is known per calendar day. A day with two or more rows takes the most frequent step between its rows, the smallest of equally frequent steps. A day with one row (for example daily data) takes the step to the next input row. So weekly data spreads each row over 7 days, and monthly data each row over its own month. The last row of the input, alone on its day, takes the step of the nearest day before it, else the most frequent step of the whole input: for monthly data the last row can be up to three days too long or too short, and it can reach into the next month. A single row before a gap looks the same as coarser data, so it spreads up to the next row. The day step picks the direction per day, so a change of step at a day boundary (for example hourly before a date, 15 minutes after it) converts each day with its own step.
A step of whole days spans calendar days on tz-aware data: daily data
upsamples to 23 or 25 hours on DST days. Other steps are absolute, so
hourly data has a step of one hour also across the DST change. A
repeated row never reaches past the next input row. An EXTENSIVE
value splits evenly over its rows, also where the last of them is
shorter (a target that does not divide the step).
The native step comes from the row index, for all columns together. So a frame with an hourly and a 15-minute column on a common index converts on the 15-minute grid, and the NaN rows of the hourly column stay NaN.
The input is sorted by time first. Duplicate timestamps and NaT raise.
The target is a pandas offset (parse_frequency). Downsampling to 'D'
on tz-aware data gives calendar days, also on DST days (23 or 25 hours).
Source code in src/mesqual/energy_data_handling/granularity_converter.py
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 | |
parse_frequency
staticmethod
¶
parse_frequency(freq: str | Timedelta | BaseOffset) -> BaseOffset
Return freq as a pandas offset of a positive fixed frequency.
Strings are pandas frequencies such as '15min', '1h', 'h' or
'D', or Timedelta strings such as '15 minutes'. A step of whole
days ('24h', '1 day', pd.Timedelta('1D')) is a calendar day,
pd.offsets.Day(n), the same as 'D': on tz-aware data it has 23 or
25 hours on DST days.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Source code in src/mesqual/energy_data_handling/granularity_converter.py
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | |
convert
¶
convert(data: DataFrame | Series, target_freq: str | Timedelta | BaseOffset, quantity_type: QuantityTypeEnum, time_level: int | str | None = None) -> DataFrame | Series
Convert time series data to a target frequency.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series
|
Input time series with DatetimeIndex. |
required |
target_freq
|
str | Timedelta | BaseOffset
|
Target frequency, see |
required |
quantity_type
|
QuantityTypeEnum
|
INTENSIVE (prices, power) or EXTENSIVE (volumes, welfare). UNKNOWN and AMBIGUOUS raise: the caller states the type. |
required |
time_level
|
int | str | None
|
Name or position of the datetime level of a row
|
None
|
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Converted data, same type as input (Series or DataFrame). Empty
input gives a copy, whatever its index type. A series of one row
is returned unchanged: it has no step. With |
Raises:
| Type | Description |
|---|---|
ValueError
|
|
TypeError
|
The index of non-empty data is not a DatetimeIndex. |
Source code in src/mesqual/energy_data_handling/granularity_converter.py
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | |
convert_to_target_index
¶
convert_to_target_index(data: DataFrame | Series, target_index: DatetimeIndex, quantity_type: QuantityTypeEnum) -> DataFrame | Series
Convert data to match a specific target DatetimeIndex.
The target frequency is the most frequent step of target_index, by
the rule of the native step; a step of whole days is a calendar day. The result is reindexed to target_index.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Source code in src/mesqual/energy_data_handling/granularity_converter.py
190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 | |
GranularityError
¶
Bases: Exception
Raised in strict mode when an index has no single step length.
TimeSeriesGranularityAnalyzer raises it for duplicate timestamps, mixed
step lengths within a day or across the index, or a day with one row.
CongestionRentCalculator raises it again with a hint to pass
granularity_hrs.
Source code in src/mesqual/energy_data_handling/granularity_analyzer.py
8 9 10 11 12 13 14 15 | |