InMemory ETL task: AI Forecasting

[Cloud release: 2025.03.26]
[On-prem release: 2025 March]
[On-prem build: 25.03.19002]

The 'AI Forecasting' task will use advanced time series forecasting models to make qualified predictions of your future data.

Users in the cloud or on-prem can forecast all data (values, measures) that exist in a table together with a time dimension.

AI Forecasting task properties:

Input table name
The name of the table that contains data to be forecasted.

Input time column
The name of the column - in the input table - that contains dates.

Input value column
The name of the column - in the input table - that contains the data to be forecasted.

Input ID column (optional)
If an ID column is given, the model will produce a forecast for each ID. In technical details, the model interprets each ID as its own timeseries, from which it can learn separate patterns but also use information from each timeseries to better learn patterns for the other IDs as well.
For example, if you want to forecast on revenue based on different product groups, you can use the select the product groups column to represent the IDs.

Additional variables (optional)
The additional variables are columns that can be used to influence the forecast (known as regressors), i.e. they have some sort of influence on the value column that are forecasted.

Forecast
Here you will select from the dropdown in what time units you want to produce forecasts, and how many of these units (Length of the forecast). Currently supported periods are days, months and years.

Forecasting any period into the future requires at least twice the amount of historic data.

  • Example: Forecasting 2 months into the future requires 4 months of data to learn from
  • Forecasting 4 years into the future requires 8 years of data to learn from
  • And so forth

When providing IDs, the model considers each ID as it separate time series. Therefore, the data requirement is also present for each ID here. If one ID does not live up to the data requirements, it will be discarded before forecasting.

Currently available forecast length options:

  • From 1 year up to 5 years
  • From 1 month up to 60 months
  • From 1 day up to 730 days

Output table
The name of the resulting table that will contain forecast data.

No. of models
Setting this will ensure that multiple different model configurations are tried, such that the most fitting model based on your input data is used to produce the forecast estimates. We recommend setting this setting to something like 20 models. You can experiment with this to see if better results are produced with more models. Optionally, read the section “How can i see how accurate my model is?” to see how you can compare results from different models.

Note, that the number of models chosen significantly changes how long the script will take.

Forecast history
If this box is checked, the Output Table will not only contain forecasts for future values based on the specified forecast length, but also forecast values from the historic data based on the model training procedure.

This functionality can be useful as a model sanity check for consumers. For example in a TARGIT chart, you can plot the historic forecast values to compare with the actual values, to see how well the model performed on data it have seen before.

Example:

Include bounds
If this box is checked, the Output Table will also contain two additional columns “upper” and “lower”. These values represent forecast bounds or predictions intervals. We use a confidence level of 0.9, meaning that the model should be 90% sure that the true future value will lie in the interval between the upper and lower bounds. Like enabling the forecast history, this is another way to sanity check the forecast estimates, and produce additional certainty measures.

Installation

The InMemory ETL AI Forecasting task requires installation of the TARGIT InMemory Scheduler Service AI for Forecasting.

This component can be downloaded from the TARGIT Download Center; 2025 March version and forward: TARGIT Download Center

Go to InMemory ETL task: AI Forecasting and download the zip file.

Extract the content of the zip file to "C:\Program Files\TARGIT\TARGIT InMemory Scheduler Service AI" on your TARGIT server.

Make sure that your ETL Studio is running the latest version as well.

Further notes on TARGIT AI Forecasting

What does it cost?
In accordance with keeping your data safe and local, using the forecasting model through ETL Studio means running it on your own local machine. Therefore, there are no additional costs associated with using this feature in ETL Studio.

What happens to my data used for forecasting?
Each forecasting model is created, trained, and produces values locally on your own TARGIT environment or in TARGIT’s hosted environment, ensuring your data stays secure throughout the forecasting process.

How can i know how accurate the model is?
Running the model in ETL Studio, there will be two (potentially three) model accuracy related outputs in the log.

These metrics are:

  • SMAPE inspired metric
  • Training RMSE
  • Validation RMSE if “No. of Models” property was set.

The validation RMSE is the best way to actually test the performance of the model on unseen data, as the two other metrics are calculated on the training data, and thus are overly optimistic on the model performance.

What can I do to get as accurate results as possible?
As mentioned in the guidelines above, setting the “No of Models” property in the forecasting component is the best way to obtain as accurate results as possible. Setting this ensures that a number of different NeuralProphet model configurations is created and trained. This process, called model-tuning, is essentially the process of trying out many different forecasting models to find the best one one your exact data.

Furthermore, if you know your measure is influenced by other variables, it can be a good idea to experiment with adding these when selecting the columns to use. Just remember to make sure they are the last columns selected - after the time column, aggregable measure column and optionally the ID column.

A final note on accuracy is on the topic of data quality. In the end, the quality of the produced forecast also entirely depends on the quality of the given data. With better quality and larger quantities comes better forecasts. Furthermore, richer datasets that show clear trends or variations enhance the model’s ability to learn and predict future outcomes more accurately. Less detailed data may result in less precise forecasts.

Thus, it can be a good idea to spend some time looking at the data you want to produce a forecast on, and consider if there is enough data in the first place, if the data is somewhat predictable, if there are any considerable outliers that should be handled first, or if any other statistical data preparation steps could be relevant.

What kind of model is used?
AI Forecasting in Targit is based on the the open-source model NeuralProphet (NeuralProphet: Explainable Forecasting at Scale (2021)).

NeuralProphet is an advanced time series forecasting model that extends Meta's Prophet model by adding a neural network component, enabling it to capture complex patterns in data. Like Prophet, NeuralProphet breaks down data into trends (long-term shifts), seasonal patterns (like daily, weekly, or yearly cycles), and special events (such as holidays or promotions). Thus, it uses historical values to identify recurring patterns and capture both regular and irregular fluctuations in data.

NeuralProphet also supports adding extra variables, like economic indicators or weather activity, to enhance forecasting accuracy. This model is well-suited for users who need to forecast time series data with multiple influencing factors, as it handles both simple and complex patterns while remaining easy to use and flexible for various applications.

Lastly, NeuralProphet is also multivariate, meaning it can handle multiple different timeseries at once. This makes it powerful, since it can make forecasts on different groups (for example different product groups), but still use the knowledge from each group to produce more accurate forecasts.

While Prophet has shown to compare or even outperform traditional timeseries forecasting models, such as ARIMA (Comparative Analysis of ARIMA, SARIMA and Prophet Model in Forecasting , Research & Development, Science Publishing Group (2024)), NeuralProphet has shown to significantly outperform Prophet.
This fact, coupled with NeuralProphet's other capabilities described above, is the reason Targit currently uses this model as its AI Forecasting Model.

Comparing Prophet and Deep Learning to ARIMA in Forecasting Wholesale Food Prices (2021)

Comparison Analysis of Facebook’s Prophet, Amazon’s Deepar+ And CNN-QR Algorithms for Successful Real-World Sales Forecasting (2021)

Was this article helpful?
1 out of 1 found this helpful

Comments

1 comment
  • Looking forward to test this!

    0

Please sign in to leave a comment.