Prediction Versioning

Important

This product feature is currently only available in our Beta environment

Overview

Optimeering is continuously enhancing the accuracy of our predictions. Whenever a prediction series is updated, we release a new version.

For most users, this doesn’t affect how you interact with our product. If you access predictions via the dashboard or use the /predictions/latest or /predictions/retrieve routes in our API, you will automatically receive the latest version.

However, if you integrate our predictions into your own machine learning models, you may need to retrain them to align with our updated predictions. To support this, we provide a simulated predictions dataset for retraining purposes.

Simulated Predictions

To create a simulated dataset, we train our models using only the data that would have been available at a specific point in time, then generate predictions for the corresponding event times.

Our API provides a /predictions/retrieve_versioned route, which accepts a prediction series ID along with a version. This route can return both actual predictions and simulated data. Simulated predictions are marked with an is_simulated flag set to true.

The diagram below shows how we manage the process for releasing a new version:

Predictions Timeline

• Version 1.0.1 is the current version, and has been producing real predictions for some time
• Version 1.0.2 is released, and starts producing real predictions
• Both versions real predictions are available through the /predictions/retrieve_versioned route
• Simulated predictions are generated for version 1.0.2, and also made available through /predictions/retrieve_versioned
• Customers can now update their internal models, and make the switch to using the real predictions from version 1.0.2
• After a deprecation period, version 1.0.1 is discontinued, and no longer producing real predictions

Versioning

We use semantic versioning for our prediction series - MAJOR.MINOR.PATCH

• The MAJOR version is incremented when the output of a prediction series completely changes. For example going from 24hrs to 72hrs prediction window or changing from 1hr to 15min intervals
• The MINOR version is incremented when we add functionality, such as new features to the underlying models
• The PATCH version is incremented when we make small changes that do not impact on the value of the actual prediction, such as improving computational performance

Announcements

We will communicate ahead of time about an upcoming release, and the deprecation period of the previous version.