If an organization is sitting on historical data from 3 years ago, the data quality could risk degradation. Why is that so? 

‍

#1  Outdated Data:

The forecast is based on all available historical data, including data that may be outdated and might no longer be representative of current conditions. This isn’t the most efficient use of the data as the model(s) would not be able to take advantage of the most recent data to inform its forecasts, potentially leading to less accurate predictions.

‍

#2 Overfitting: We might run the risk of overfitting the model to the historical data, making it less accurate when applied to new data.

‍

#3 Higher variance: There might be higher variance in your forecast results. 

This isn’t ideal when the goal is to tighten the confidence interval and minimize the margin of error. To compound this issue, the model may not be able to capture the underlying patterns in the data.

‍

What’s the solution?

‍

A recommended (and common) way to work through these challenges is by doing backtesting of the forecasting model. Backtesting is a technique used to evaluate the accuracy of a particular forecasting model by testing it on historical data.  It essentially involves generating forecasts as if we were at previous points in time and evaluating them against the actual outcomes. 

By doing so, we can determine how well our forecasting model(s) would have performed if we were to use it in the past.

Let’s start with some clear advantages:

• Helps to evaluate the accuracy of forecasting models: Backtesting provides an opportunity to evaluate the accuracy of forecasting models by comparing actual outcomes with forecasted values. This evaluation helps in identifying the strengths and weaknesses of the models and gives us room to improve future forecasts.
  
  
• Improves the reliability of forecasts: By testing a forecasting model on historical data, it becomes possible to estimate the model's reliability in predicting future events. By producing a clear record of past performance, backtesting can help to build trust and credibility in these models.
  
  
• Provides a measure of the model's robustness:  Backtesting provides a measure of the model's robustness by testing the model's performance against different historical datasets. This helps to identify if the model's performance is consistent across different datasets or if it is specific to one dataset.

That said, backtesting should be approached with caution. In plenty of cases, the backtesting conducted includes data that is not available in the previously forecasted periods. This is recognized as the look-ahead bias, something that can arise when “sneak peeking” at data that was not available at the point in time the backtesting is simulating. 

‍
Enter “expanding window”

Here’s where “Expanding window”, a variant of the “rolling window” approach comes into play. 

‍

How exactly does this functionality work?

‍

This statistical forecasting technique, often known as rolling windows. hinges on gradually increasing the size of the training data set used. By dividing the available historical data into a series of windows, starting with a small initial window, and gradually increasing its size to include more data points. At each step, a new forecast is generated based on the expanded window, and the accuracy of the forecast is evaluated against the actual outcomes.

Think of it like a puzzle: You started with a few pieces and slowly add more until you have a complete picture. Each time you add more data, you make a new prediction based on what you’re learned so far.

The much-requested capability in Indicio aims to optimize prediction accuracy.

This helps you continuously update a model as new data becomes available, by using a fixed number of historical observations to make a forecast for a fixed future time horizon. Plus, you are more ready to adapt to changes or trends in the data overtime when including more recent observations.

‍

Think of it like a puzzle. You start with a few pieces and slowly add more until you have a complete picture. Each time you add more data, you make a new prediction based on what you’re learned so far.

‍

What’s possible with this:

#1 You can ensure that your models are continually updated with the most recent data. This can help to capture any short-term changes or trends in the data that may not be reflected in a longer historical period. By constantly updating the model with the latest data, the forecast accuracy can be improved, particularly in situations where the underlying factors driving the data may be changing rapidly. Pretty common in today’s political and economic climate.

#2 Easily compare the accuracy of your forecasts over time. This approach can help to identify any patterns in the forecast accuracy and any areas where improvements can be made. Comparing the accuracy of forecasts over time can also help to identify any factors that may be causing forecast errors, such as changes in the data, seasonal patterns, or unexpected events.

‍

Real-life cases

Utilizing the expanding window approach to forecasting is advantageous for companies - to better manage their operations by forecasting demand, managing supply chain inventory, and plan production schedules efficiently .

Consider an electronic components manufacturer that would need to forecast demand for their products over the next two quarters. By using this approach, they can continually update their demand forecasting model with the latest sales data and identify any short-term changes or trends in demand that may not be captured by a longer historical period.

This also is relevant to the supply side of planning, where a manufacturer needs to forecast the amount of raw materials required to meet production targets. Again, using this approach, they can capture any short-term changes in demand for their products and adjust their raw material orders accordingly, avoiding overstocking or shortages.

‍

Let’s take another example.

‍

Automotive manufacturers can better predict expected sales volumes and production levels, leading to another level of preparedness. They can adjust their production schedules accordingly to meet changing demand levels and avoid costly inventory shortages or overproduction. With strong seasonal patterns, relatively stable and minimal historical points (relative to a more volatile industry), the utility of the expanding window approach is most well-suited for these three given contexts. 

‍

‍

Benefit

The goal is to get as close to ensuring that the models used are the best-performing, which would consequently generate the most accurate forecast. 

‍

What has this “expanding window” capability meant for our users:

- Since comprehensive backtesting is now done automatically in Indicio, they’ve been able to test models quicker and identify models that work best at the forecasted point.

- When updating their model with the latest data, they’re able to capture any short-term trends that might not be obvious just by looking at a longer history. 

‍

‍

 Extract the full value from your data. Book a quick demo and see how it can boost your forecasting.

‍

‍