Google just entered the race of foundation models for time-series forecasting.
In August 2023, the time-series community was disrupted by the discharge of TimeGPT, Nixtla’s first foundation model for time series forecasting.
Following TimeGPT, multiple foundation forecasting models were released, but there was one which stood out. Recently, Google unveiled TimesFM[1], a groundbreaking time-series model with phenomenal results.
Time series are ubiquitous, utilized in many domains like retail, energy demand, economics, healthcare and more. A foundation TS model could be readily applied to any TS case with great accuracy, like GPT-4 for text.
In this text, we discuss:
- The challenges of foundation models in time series in comparison with NLP.
- How TimesFM overcomes these challenges.
- How TimesFM works and why it’s a robust model.
- TimesFM benchmark results.
- Prospects for the long run of foundation models in time-series forecasting
Let’s start.
I’ve launched AI Horizon Forecast, a newsletter specializing in time-series and modern AI research. Subscribe here to broaden your horizons!
The concept of a promising foundation model in NLP was already evident with the discharge of GPT-2 in Language Models are Unsupervised Multitask Learners [2].
But in time series, constructing a foundation model will not be straightforward. There are several challenges:
- Dataset Scarcity: In NLP, finding text data is straightforward. Nonetheless, public time-series datasets should not available.
- Unpredictable Format: Language models are based on well-defined grammars and vocabularies. Time-series data may belong to domains with different characteristics — e.g., highly sparse sales or volatile financial data.
- Different Granularities: Every time series model works for a selected granularity — e.g., hourly, weekly, monthly…