Business Intelligence · 6 min read
Forecast accuracy is not enough: evaluate the planning decision
Read forecast bias, error, uncertainty, and business exposure together, with realistic baselines and a practical planning review grid.
Updated
A forecast can look accurate in a summary and still create difficult planning decisions. Overprediction for one product can cancel underprediction for another. A model can fit historical sales closely while struggling with the next promotion. A reliable forecast at category level can conceal repeated shortages at particular locations.
The useful question is not simply, "How accurate is the model?" It is, "Does this forecast improve the decision at the time and level where the business acts?" That requires a clear target, an honest evaluation, and a view of the consequences of being wrong.
For leaders, this means connecting forecast quality with service, inventory, margin, and planning effort. For technical teams, it means testing realistic horizons, avoiding information leakage, and reporting more than one aggregate score. Neither perspective is enough on its own.
Define what is being forecast
Sales, demand, shipments, orders, and receipts are different targets. Observed sales can be limited by stock availability. Shipments can be constrained by supply or fulfillment capacity. A forecast of recorded sales should not be presented as unconstrained customer demand without an appropriate method and supporting evidence.
Specify the entity, unit, time interval, and horizon. A SKU-location weekly forecast used for replenishment has different requirements from a monthly category-revenue forecast used for a leadership review. Include the decision date and lead time. A forecast that becomes accurate only after the purchase order must be placed may be of limited planning value.
Agree on what information is available at that decision point. Promotion plans, prices, and supplier commitments may change later. The evaluation should reflect what planners could reasonably have known, not an idealized historical dataset containing final answers.
Evaluate on genuinely unseen periods
Forecasting: Principles and Practice distinguishes errors on new observations from residuals on data used to fit a model. A close historical fit is not proof of future performance. Keep a suitable test period and compare methods at the horizon the business needs.
Rolling-origin evaluation repeats the exercise at successive historical decision dates, using only earlier information for each forecast. This is useful when one unusually easy or difficult test period would otherwise dominate the conclusion. Include ordinary weeks and relevant seasonal conditions rather than selecting a convenient window.
For an operational evaluation, also preserve the forecast version that was actually available before the decision. Replacing it later with a revised forecast makes performance appear better without improving the original decision. Label retrospective model comparisons separately from live operating results.
Establish a credible baseline
A new method should earn its complexity. Compare it with a practical baseline, such as the recent run rate, the corresponding prior-season period, or the current planning method. Choose a baseline that fits the business; last week's sales may be a poor comparison for a strongly seasonal product.
Use the same periods, products, exclusions, and horizons for each method. If one method has no forecast for difficult cases, do not quietly exclude those cases only from its score. Report coverage alongside quality. A model that performs well for a small subset may still be useful, but its scope should be clear.
Separate new products and intermittent-demand items when appropriate. They have different evidence from established high-volume products. The evaluation should reveal those differences so planners know where judgment or another method remains necessary.
Read bias and absolute error together
Bias describes a tendency to forecast high or low. Define the sign convention explicitly. In this article, error = forecast - actual, so a positive average indicates overforecasting. Absolute error measures the size of misses without allowing positive and negative values to cancel.
Consider an illustrative two-item review. Actual sales are 100 units for each item. The forecasts are 140 and 60. Total forecast equals total actual, so aggregate bias is zero. Yet each item missed by forty units, potentially creating excess inventory in one place and a shortage in the other. The planning problem is visible only below the total.
Mean absolute error reports average error in the original units. Percentage measures can support comparisons, but denominators matter. Mean absolute percentage error is undefined when an actual value is zero and unstable near zero. Do not replace those cases with zero error merely to complete a scorecard.
Use weighted measures carefully
For a set of nonnegative actual unit sales, one commonly used WAPE definition is sum of absolute errors / sum of actual units. In the two-item example, it is 80 / 200 = 40%. This is not the same as measuring the error of the aggregate total, which would be zero. State the evaluation grain before quoting the percentage.
WAPE gives greater influence to high-volume items and is undefined when the total actual denominator is zero. It can hide the experience of smaller but strategically important items. Review segments, absolute units, and business exposure alongside the portfolio measure. Do not compare changing product mixes without noting the difference.
No single error metric answers every planning question. A high-volume commodity and a low-volume, high-margin item may deserve different service decisions. Keep the quality score separate from the business rule that decides how much inventory or capacity to hold.
Show uncertainty and decision consequences
A point forecast gives one estimate. A prediction interval expresses a range under the model's assumptions. Review whether the stated intervals have reasonable coverage over time, and recognize that structural changes can weaken those assumptions. A wide interval is information about uncertainty, not necessarily a reason to hide the forecast.
Underforecasting and overforecasting can have different costs. A missed sale, an expedited shipment, a markdown, and excess storage are not interchangeable outcomes. Use an agreed planning policy to translate uncertainty into a replenishment or capacity decision. Do not claim that minimizing a statistical error automatically maximizes profit.
Keep demand estimation separate from supply feasibility. Available inventory, lead times, minimum order quantities, and budget constrain the plan. Showing both the demand view and the feasible plan helps the business see unmet opportunity rather than treating a supply limitation as an absence of demand.
Give planners a useful review grid
A practical grid can show product-location, forecast version, horizon, actual, forecast, signed error, absolute error, availability, promotion status, and exposure. Include the baseline comparison and the reason an item needs review. Sort by material business risk rather than only the largest percentage miss.
Flag periods affected by stockouts or incomplete data. Low recorded sales during a stockout do not establish low underlying demand. Likewise, a newly corrected source record can change an evaluation result without any change to the forecast. Preserve the distinction so the team investigates the right cause.
Record planner overrides and their reasons before the outcome is known. Compare the original forecast and the adjusted plan on consistent cases. Human adjustments can add valuable context, but they should be evaluated rather than assumed to improve every prediction. The purpose is to learn which interventions help.
Turn evaluation into an operating habit
Start with one planning workflow and agree on its target, horizon, baseline, metrics, and review cadence. Keep the measures understandable to the people making the decision. Review recurring sources of error, such as missed event information or a location-level data gap, before assuming the answer is a more complex model.
UJAS Business Intelligence connects governed retail reporting, forecasts, and planning views so teams can examine performance in context. Skynar Commerce Intelligence brings forecasts together with marketplace signals and recommendations. The available history and decision requirements guide the analytical setup; neither product removes the need for accountable planning judgment.
VeriVal helps teams build forecast-ready data, evaluate methods, and connect results to practical decisions. If forecast accuracy is reported but its planning value remains unclear, start a conversation.