Time series forecasting plays a significant role in crucial decision-making processes across various industries resembling retail, finance, manufacturing, and healthcare. Nevertheless, in comparison with domains like natural language processing and image recognition, the combination of advanced artificial intelligence (AI) techniques into time series forecasting has been relatively slow. Although foundational AI has made significant progress in areas like natural language processing and image recognition, its impact on time series forecasting has been limited until recently. Nevertheless, there’s now an increasing momentum in the event of foundational models specifically tailored for time series forecasting. This text goals to delve into the evolving landscape of foundational AI for time series forecasting, exploring the recent advancements on this domain. Nevertheless, before delving into these advancements, let’s briefly introduce time series forecasting and its applications in various industries.

Time Series Forecasting and Applications

Time series data refers to a sequence of information points collected or recorded at regular time intervals. The sort of data is prevalent across various domains, resembling economics, weather, health, and more. Each data point in a time series is time-stamped, and the sequence is usually used to investigate trends, patterns, and seasonal differences over time.

Time series forecasting involves using historical data to predict future values within the series. It’s a critical method in statistics and machine learning that helps in making informed decisions based on past patterns. Forecasting will be so simple as projecting the identical growth rate into the longer term or as complex as using AI models to predict future trends based on intricate patterns and external aspects.

Some applications of time series forecasting are as follows:

• Financial Markets: In finance, time series forecasting is used to predict stock prices, exchange rates, and market trends. Investors and analysts use historical data to forecast future movements and make trading decisions.
• Weather Forecasting: Meteorological departments use time series data to predict weather conditions. By analyzing past weather data, they forecast future weather patterns, helping in planning and decision-making for agriculture, travel, and disaster management.
• Sales and Marketing: Businesses utilize time series forecasting to predict future sales, demand, and consumer behavior. This helps in inventory management, setting sales targets, and developing marketing strategies.
• Energy Sector: Energy corporations forecast demand and provide to optimize production and distribution. Time series forecasting helps in predicting energy consumption patterns, enabling efficient energy management and planning.
• Healthcare: Within the healthcare sector, time series forecasting is used to predict disease outbreaks, patient admissions, and medical inventory requirements. This assists in healthcare planning, resource allocation, and policy making.

Foundation Time Series Models

Foundational AI models are extensive, pre-trained models that form the premise for various artificial intelligence applications. They’re trained on large and diverse datasets, enabling them to discern patterns, connections, and structures throughout the data. The term “foundational” refers to their capability for being fine-tuned or modified for tasks or domains with minimal additional training. Within the context of time-series forecasting, these models are constructed similarly to large language models (LLMs), utilizing transformer architectures. Like LLMs, they’re trained to predict the next or missing element in a knowledge sequence. Nevertheless, unlike LLMs, which process text as subwords through transformer layers, foundational time-series models treat sequences of continuous time points as tokens, allowing them to sequentially process time-series data.

Recently, various foundational models have been developed for time series data. With higher understanding and selecting the suitable foundational model, we are able to more effectively and efficiently leverage their capabilities. In the next sections, we’ll explore the various foundational models available for time series data evaluation.

• TimesFM: Developed by Google Research, TimesFM is a decoder-only foundational model with 200 million parameters. The model is trained on a dataset of 100 billion real-world time points, encompassing each synthetic and actual data from varied sources resembling Google Trends and Wikipedia Pageviews. TimesFM is able to zero-shot forecasting in multiple sectors, including retail, finance, manufacturing, healthcare, and the natural sciences, across different time granularities. Google intends to release TimesFM on its Google Cloud Vertex AI platform, providing its sophisticated forecasting features to external clients.
• Lag-Llama: Created by researchers from the Université de Montréal, Mila-Québec AI Institute, and McGill University, Lag-Llama is a foundational model designed for univariate probabilistic time series forecasting. Construct on the muse of Llama, the model employs a decoder-only transformer architecture which uses variable sizes time lags and time resolutions for forecasting. The model is trained on diverse time series datasets from several sources across six different groups including energy, transportation, economics, nature, air quality and cloud operations. The model is conveniently accessible through the Huggingface library.
• Moirai: Developed by Salesforce AI Research, Moirai is a foundational time series model designed for universal forecasting. Moirai is trained on the Large-scale Open Time Series Archive (LOTSA) dataset, which incorporates 27 billion observations from nine distinct domains, making it the biggest collection of open time series datasets. This diverse dataset allows Moirai to learn from a wide selection of time series data, enabling it to handle different forecasting tasks. Moirai uses multiple patch size projection layers to capture temporal patterns across various frequencies. A crucial aspect of Moirai is to make use of any-variate attention mechanism, allowing forecasts across any variety of variables. The code, model weights, and data related to Moirai can be found within the GitHub repository called “uni2ts“

• Chronos: Developed by Amazon, Chronos is a set of pre-trained probabilistic models for time series forecasting. Built on the T5 transformer architecture, the models use a vocabulary of 4096 tokens and have various parameters, starting from 8 million to 710 million. Chronos is pretrained on an unlimited array of public and artificial data generated from Gaussian processes. Chronos differs from TimesFM in that it’s an encoder-decoder model, which enables the extraction of encoder embeddings from time series data. Chronos will be easily integrated right into a Python environment and accessed via its API.

• Moment: Developed collaboratively by Carnegie Mellon University and the University of Pennsylvania, Moment is a family of open-source foundational time series models. It utilizes variations of T5 architectures, including small, base, and huge versions, with the bottom model incorporating roughly 125 million parameters. The model undergoes pre-training on the extensive “Time-series Pile,” a various collection of public time-series data spanning various domains. Unlike many other foundational models, MOMENT is pre-trained on a large spectrum of tasks, enhancing its effectiveness in applications resembling forecasting, classification, anomaly detection, and imputation. The whole Python repository and Jupyter notebook code are publicly accessible for utilizing the model.

The Bottom Line

Time series forecasting is an important tool across various domains, from finance to healthcare, enabling informed decision-making based on historical patterns. Advanced foundational models like TimesFM, Chronos, Moment, Lag-Llama, and Moirai offer sophisticated capabilities, leveraging transformer architectures and diverse training datasets for accurate forecasting and evaluation. These models provide a glimpse into the longer term of time series evaluation, empowering businesses and researchers with powerful tools to navigate complex data landscapes effectively.