timeseries-peakbench / working note synthetic building · illustrative MA tariff · 2025-01-01 to 2025-12-31

Choosing the dispatch policy is worth $20k a year. Choosing the forecaster is worth $2.8k.

When a commercial battery uses a day-ahead load forecast to cut demand charges, how much money does forecast error cost? A 500 kWh / 250 kW battery, an office building that peaks at 797 kW, a $22.00/kW monthly demand charge, six day-ahead forecasters and three dispatch policies, simulated at 15-minute resolution over 365 days.

Synthetic dataThe building load is generated (weekday occupancy, Boston-like weather, a summer cooling peak, random equipment spikes), not read from a meter. The tariff is illustrative, not a filed rate.

In short

Policy beats forecaster. Holding the forecaster fixed (Ridge), the three dispatch policies are $20k a year apart. Holding the policy fixed (threshold + ratchet), the six forecasters are $2.8k apart. In every case the spread across policies exceeds the spread across forecasters, by 2.3x to 5.3x.
The obvious threshold policy is the worst one. Setting each day's target as low as the forecast says the battery can hold captures only 16-24% of perfect-foresight savings. It spends the whole battery on the forecast's peak, so when the real peak is higher the battery is empty before it arrives: it ran out with the meter still above target on 47-72% of business days (the ratchet: 5-7%).
Reading the meter only pays if the policy respects the bill. Adding one rule (never shave below a peak already set this month) lifts the same closed-loop policy to 62-67%, against 32-35% for a forecast-scheduled window.
Timing error is what kills schedules. A perfect forecast running 30 minutes late costs the window policy $38k a year; the meter-driven policies don't move.
But no policy wins everywhere. Remove the short equipment spikes from the same building and the ranking flips: the window schedule captures 67-79%, the ratchet 43-54%. Spikes are what a forecast can't place and a meter can catch; on a smooth load, the risk is under-forecasting the level, and a schedule bounded by its plan degrades more gently.
Real meters back this up. On metered 15-minute data from six buildings in Portugal (UCI, 2014, picked by a fixed rule), threshold + ratchet was the best policy on six of six with the Ridge forecast, capturing 47-93% of the best perfect-foresight savings. A perfect forecast 30 minutes off cost the window schedule 35-70% of its savings. Real loads are harder to forecast (Ridge MAPE 7-21% on business days). TimesFM 3.0 vs ridge: lower MAPE on six of six buildings, lower peak error on two, more threshold + ratchet savings on three (-$1.5k to +$1.2k a year per building). Timestral 209M vs ridge: lower MAPE on six of six buildings, lower peak error on five, more threshold + ratchet savings on three (-$1.3k to +$1.7k a year per building).
Foundation models did not change the answer. On the synthetic building, Timestral 209M beat ridge on day-ahead peak error; the others did not (mean absolute error: TimesFM 3.0 71 kW, TimesFM 2.5 66 kW, Timestral 209M 62 kW, ridge 65 kW). Under threshold + ratchet they moved annual savings by +$893 (TimesFM 3.0), +$2.8k (TimesFM 2.5), +$1.3k (Timestral 209M), against $20k between policies. TimeGPT is implemented (timegpt_zeroshot()) but did not run for this report: no NIXTLA_API_KEY was available, so it is left out of every comparison rather than estimated.

The bill, one yearRidge forecast · threshold + ratchet

	Without battery	With battery	Change
Energy charges	$401,708	$400,786	-$922
Demand charges (12 monthly peaks × $22.00/kW)	$177,705	$147,603	-$30,102
Total bill	$579,413	$548,389	-$31,024 (-5.4%)
Average monthly peak	673 kW	559 kW	-114 kW

Saved $31,024 (5.4% of the bill) Left on the table vs perfect foresight: $19,147 (62% of the possible savings captured)

Timing error

Run a perfect forecast 30 minutes late and the window schedule loses $38k a year; the meter-driven policies don't move.

Annual savings when the forecast is shifted earlier or later in time. The window policy discharges on the forecast's clock; threshold and ratchet discharge when the meter crosses a kW level, so only the forecast's magnitude reaches them.

Forecaster vs policy

With equipment spikes: threshold + ratchet wins with every forecaster (62-67% of perfect foresight); within a policy, forecasters differ by a few points.

Annual savings for every forecaster under each policy; bar labels give the share of perfect-foresight savings captured.

The day that set the peak

Tuesday July 22, the year's peak day. The Ridge forecast said 572 kW, the meter hit 797 kW, and with threshold + ratchet the building drew at most 661 kW.

Top: actual load, the day-ahead forecast, the kW level the policy defends, and net load after the battery (the shaded band is energy supplied or absorbed). Bottom: energy stored.

Month by month

Demand charges fall $30k over the year (Ridge, threshold + ratchet), but only $117 in February: one day the battery couldn't cover sets the whole month's charge.

Each month's demand charge, set by its single highest 15-minute interval, without and with the battery.

Error vs dollars

Each kW of day-ahead peak error costs $5 a day under threshold + ratchet, $12 a day under threshold, $13 a day under window (Ridge). The ratchet's line is the flattest: it degrades most gracefully.

Each dot is a business day: how much less peak the battery shaved than perfect foresight, priced at the demand charge (what the day was worth had it set the month's peak). Lines are least-squares fits per policy.

Did the foundation models help? (synthetic building)

Better average accuracy did not reliably buy better peaks, and the dollars moved by at most $3.0k, against $20k for the choice of policy.

TimesFM 3.0 (zero-shot) did not beat ridge on day-ahead peak error (71 vs 65 kW mean absolute error; its peaks run 66 kW low on average), while its business-day MAPE was better (4.7% vs 5.7%). Dollars vs ridge: +$893 under threshold + ratchet, -$3.0k under threshold, -$14 under window. TimesFM 2.5 (zero-shot) did not beat ridge on day-ahead peak error (66 vs 65 kW mean absolute error; its peaks run 58 kW low on average), while its business-day MAPE was better (4.7% vs 5.7%). Dollars vs ridge: +$2.8k under threshold + ratchet, -$411 under threshold, -$1.5k under window. Timestral 209M (zero-shot) beat ridge on day-ahead peak error (62 vs 65 kW mean absolute error; its peaks run 56 kW low on average), and its business-day MAPE was better too (4.0% vs 5.7%). Dollars vs ridge: +$1.3k under threshold + ratchet, +$962 under threshold, -$61 under window. TimeGPT is implemented (timegpt_zeroshot()) but did not run for this report: no NIXTLA_API_KEY was available, so it is left out of every comparison rather than estimated.

Forecaster	MAPE, business days	Peak error (MAE)	Peak bias	Peak timing error	Threshold + ratchet	Threshold	Window
Seasonal naive	6.1%	75 kW	-5 kW	136 min	$31,278 62%	$9,655 18%	$16,569 32%
28-day profile	5.0%	61 kW	-44 kW	120 min	$33,085 66%	$10,689 20%	$18,050 35%
Ridge	5.7%	65 kW	-47 kW	124 min	$31,024 62%	$11,314 22%	$18,046 35%
TimesFM 3.0 (zero-shot)	4.7%	71 kW	-66 kW	109 min	$31,917 64%	$8,271 16%	$18,032 35%
TimesFM 2.5 (zero-shot)	4.7%	66 kW	-58 kW	105 min	$33,863 67%	$10,903 21%	$16,562 32%
Timestral 209M (zero-shot)	4.0%	62 kW	-56 kW	111 min	$32,296 64%	$12,276 24%	$17,984 34%

Peak columns are over business days. Policy columns: annual savings and share of perfect-foresight savings captured.

Robustness

Battery size shifts the numbers, not the ranking. The building's load shape changes the ranking.

Share of perfect-foresight savings captured, range across the real forecasters.

Case	Threshold + ratchet	Threshold	Window
Half-size battery (250 kWh / 125 kW)	76-80%	14-25%	21-28%
Base battery (500 kWh / 250 kW)	62-67%	16-24%	32-35%
Double-size battery (1,000 kWh / 500 kW)	57-68%	25-32%	47-50%
Smooth (no equipment spikes), base battery	43-54%	20-25%	67-79%

Does the policy still matter more than the forecaster? In every case the spread across policies exceeds the spread across forecasters, by 2.3x to 5.3x. The two best policies come closer than that, and in one case (double-size battery, with equipment spikes) closer to each other than the best and worst forecasters are.

Case	Spread across policies	Best vs second-best policy	Spread across forecasters	Ratio
Half-size battery, with equipment spikes	$19,203	$15,975	$3,643	5.3x
Base battery, with equipment spikes	$19,710	$12,978	$4,005	4.9x
Double-size battery, with equipment spikes	$15,227	$3,874	$6,761	2.3x
Base battery, smooth (no equipment spikes)	$11,916	$6,878	$2,930	4.1x

Annual savings. Policy columns: the smallest gap over all forecasters. Forecaster column: the largest gap over all policies.

Real meters

On all six real buildings, threshold + ratchet is the best policy, and the policy spread beats the forecaster spread on six of six.

Metered 15-minute load from six buildings in UCI's ElectricityLoadDiagrams20112014 (Portugal, 2014; CC BY 4.0), picked by a rule fixed before any results: complete data, a 2013 peak between 100 and 2,000 kW, and an office-like weekly pattern. Each battery is sized at 30% of the building's 2013 peak with two hours of storage. Bars use the Ridge forecast and show savings as a share of the best perfect-foresight result across the three policies.

Building	2014 peak	Battery	MAPE	Threshold + ratchet	Threshold	Window	Policy spread	Forecaster spread	Peak MAE, kW
MT_059	239 kW	70 kW / 140 kWh	12.0%	72%	6%	10%	$5,909	$1,098	23 / 19 / 17
MT_061	159 kW	55 kW / 110 kWh	11.2%	90%	23%	32%	$4,494	$1,375	9 / 9 / 8
MT_128	772 kW	235 kW / 470 kWh	16.8%	47%	19%	39%	$10,802	$8,383	80 / 70 / 70
MT_151	289 kW	85 kW / 170 kWh	6.9%	48%	33%	40%	$2,201	$916	15 / 16 / 14
MT_158	215 kW	65 kW / 130 kWh	21.0%	93%	26%	29%	$5,996	$1,460	15 / 16 / 16
MT_164	262 kW	80 kW / 160 kWh	18.6%	66%	15%	17%	$3,975	$1,790	12 / 13 / 11

Policy columns: share of the best perfect-foresight savings, Ridge forecast. Spreads are annual dollars. Peak MAE: ridge / TimesFM 3.0 / Timestral 209M.

Method and limits

How the numbers were made

Each forecaster issues a 96-interval forecast for day D using only data up to midnight before D; a test poisons the future and checks that no forecast changes. Foundation models see a 28-day trailing context, zero-shot, and their forecasts are cached per (model, data fingerprint, date). The battery has 88% round-trip efficiency (square root on each leg) and a state of charge kept between 10% to 95%. It charges only off-peak, never above the day's target, and paced to refill before the next on-peak window (8:00 to 21:00 on business days, $0.19/kWh; $0.13/kWh otherwise). Savings are full-bill: demand plus time-of-use energy.

Limitations

The main study is one generated building. The six real meters are a check, not a census: anonymous Portuguese clients, one year each, with the illustrative MA tariff applied to them.
Illustrative tariff. No customer charges, ratchets, coincident-peak or capacity charges.
No battery degradation, so cycling looks free and the threshold policy's extra cycles are not penalised.
No grid-services or wholesale revenue stacked on top of bill savings.
A single meter and a single battery; no portfolio effects.
Day-ahead only: no intraday re-forecasting, which a production controller would do.

Reproduce

make install
make run      # evaluation, about 14.1s on a laptop CPU (foundation models served from cache)
make test
make report   # this page
streamlit run app.py