This post was co-authored with Rafael Guedes.
Time series forecasting has been evolving towards foundation models attributable to their success in other artificial intelligence (AI) areas. Particularly, now we have been witnessing the success of such approaches in natural language processing (NLP). The cadence of the event of foundational models has been accelerating over time. A recent, more powerful Large Language Model (LLM) is released every month. This isn’t restricted to NLP. We see an identical growing pattern in computer vision as well. Segmentation models like Meta’s Segment Anything Model (SAM) [1] can discover and accurately segment objects in unseen images. Multimodal models resembling LLaVa [2] or Qwen-VL [3] can handle text and pictures to reply any user query. The common characteristic between these models is that they’ll perform accurate zero-shot inference, meaning that they don’t must be trained in your data to have a wonderful performance.
Defining what a foundational model is and what makes it different from traditional approaches might be useful at this point. First, a foundational model is large-scale (namely its training), which provides a broad understanding of the predominant patterns and necessary nuances we will find in the info. Secondly, it’s general-purpose, i.e., the foundational model can perform various tasks without requiring task-specific training. Regardless that they don’t need task-specific training, they might be fine-tuned (also often called transfer learning). They’re adaptable with relatively small datasets to perform higher at that specific task.
Why is applying it to time series forecasting so tempting based on the above? Foremost, we design foundational models in NLP to know and generate text sequences. Luckily, time series data are also sequential. The previous point also aligns with the undeniable fact that each problems require the model to routinely extract and learn relevant features from the sequence of the info (temporal dynamics in time series data). Moreover, the general-purpose nature of foundational models means we will adapt them to different forecasting tasks. This flexibility allows for applying a single, powerful model across various domains and…