Forecast Value Added (FVA) is a metric that evaluates each participant’s performance in the forecasting process. It helps decide which adds value and which doesn’t.
We should not take for granted that all forecasters (both persons and algorithms) add value to a prediction.
To streamline our process, we can eliminate predicting elements that return poor results. This way we get more accurate forecasts without any extra effort involved.
FVA is a very rational approach. It pitches the results of doing something against doing nothing.
The key to FVA analysis is to compare any forecast to what is called a naïve forecast. Although other forms of naive forecast exist, we are safe here to define naive forecast as an estimating method that takes the last period’s actuals as this period’s forecast.
A simple example of naive forecast in Pharma
If a pharmaceutical company sold 1000 units of medicine in March last year and 2000 units in April, your naive forecast for March and April of this year would also be 1000 and 2000 respectively.
Since we are in the field of healthcare, we could compare the naive forecast to the placebo in a controlled experiment. The experimental group in this FVA analysis receives a treatment of forecasting. The control group gets the placebo, the naive forecast. In this way, the control group (the one with the naive forecast) provides a baseline that lets us see if the treatment certainly improves the prediction.
A better forecaster?
On the other hand, whenever we compare the performance of different forecasters we shouldn’t rely on forecast errors. In the pharmaceutical industry, we often have several demand forecasters for a company in a given region, say in the U.S for instance. Each of them is in charge of a portfolio of medicines for which they forecast.
You should reward the one with the lowest error, right?
Not necessarily! What if accuracy is dependent on the difficulty to forecast the demand of a drug? A good criterion for bonuses should include a comparison with a naïve model.
Let us go back to the example above. The naive model for March forecasts 1000 units for March-21 (because that was the figure for March-20).
A machine learning algorithm analyzes historical demand data of the pharmaceutical company and returns a forecast of 1800 units for March-21.
A demand forecaster in charge of the product then reviews the prediction and overrides it: the expert’s forecast is 200 units.
March ends, and the demand turns out to be 1400.
In this case, the naïve model was able to achieve an error of 31 percent. The algorithm added value by reducing the error by seven points to 24 percent. However, the analyst override made the forecast worse, increasing the error to 38 percent. The override’s FVA was negative-seven (-7) percentage points compared to the naïve model and was negative-ten (-14) percentage points compared to the statistical forecast.
One period is not enough to draw a conclusion. Like any other statistical evidence, it needs to be evaluated over time.
Tracking the efficiency of each step to eliminate waste
FVA helps you do away with waste in a forecasting process. By eliminating the activities that do not add value, you can optimize the forecasting method.
Forecast value added measures the efficiency of each participant in the forecasting process. FVA gauges efficient steps (i.e., decreasing forecast error and not consuming too much time) and those that both waste resources and do not produce any extra accuracy.
For each participant working on the forecasting process, you will need to track:
- their FVA compared to that of the previous team
- the time spent working on the forecast
In many forecasting processes, planners make several minor adjustments to the forecasts, producing no added value while wasting time. In such cases, FVA would tell us that we are better off getting rid of those adjustments.
Finally, the business side of FVA
Increasing forecast accuracy is not an end. Reducing forecast error and variability via FVA analysis can significantly impact backorders, inventory, and customer service. Each time we add a percentage of forecast value-added, that improvement means something in dollars.
Reference:
Gilliland, M. (2010). The Business Forecasting Deal: Exposing Myths, Eliminating Bad Practices, Providing Practical Solutions. John Wiley & Sons, Hoboken, N.J.
Recent Comments