Home Artificial Intelligence Transform Time Series for Deep Learning Supervised Learning with Time Series Auto-Regression with Deep Learning Hands-On Key Takeaways

Transform Time Series for Deep Learning Supervised Learning with Time Series Auto-Regression with Deep Learning Hands-On Key Takeaways

2
Transform Time Series for Deep Learning
Supervised Learning with Time Series
Auto-Regression with Deep Learning
Hands-On
Key Takeaways

Photo by Claudio Testa on Unsplash

Supervised learning involves training a machine learning model with an input data set. This data set is often a matrix: A two-dimensional data structure composed of rows (samples) and columns (features).

A time series is a sequence of values ordered in time. So, it must be transformed for supervised learning.

In a previous article, we learned the right way to transform a univariate time series from a sequence right into a matrix. This is completed with a sliding window. Each commentary of the series is modeled based on past recent values, also called lags.

Here’s an example of this transformation using the sequence from 1 to 10:

Transforming a sequence right into a matrix with a sliding window. Image by creator.

This transformation enables a sort of modeling called auto-regression. In auto-regression, a model is built using the past recent values (lags) of a time series as explanatory variables. These are used to predict future observations (goal variable). The intuition for the name auto-regression is that the time series is regressed with itself.

In the instance above, the lags are the initial 5 columns. The goal variable is the last column (the worth of the series in the following time step).

While most methods work with matrices, deep neural networks need a unique structure.

The input to deep neural networks equivalent to LSTMs or CNNs is a three-dimensional array. The actual data is identical because the one you’d put in a matrix. But, it’s structured otherwise.

Besides rows (samples) and columns (lags), the additional dimension refers back to the variety of variables within the series. In a matrix, you concatenate all attributes together no matter their source. Neural networks are a bit tidier. The input is organized by each variable within the series using a 3rd dimension.

Let’s do a practical example to make this clear.

Photo by Quino Al on Unsplash

On this tutorial, you’ll learn the right way to transform a time series for supervised learning with an LSTM (Long Short-Term Memory). An LSTM is a sort of neural network that is very useful to model time series.

We’ll split the time series transformation process into two steps:

  1. From a sequence of values right into a matrix;
  2. From a matrix right into a three-D array for deep learning.

First, we’ll do an example with a univariate time series. Multivariate time series are covered next.

Univariate Time Series

Let’s start by reading the info. We’ll use a time series related to the sales of various sorts of wine. You possibly can check the source in reference [1].

import pandas as pd

# https://github.com/vcerqueira/blog/tree/predominant/data
data = pd.read_csv('data/wine_sales.csv', parse_dates=['date'])
data.set_index('date', inplace=True)

series = data['Sparkling']

We concentrate on the sales of sparkling wine to do an example for the univariate case. This time series looks like this:

From a sequence of values right into a matrix

We apply a sliding window to remodel this series for supervised learning. You possibly can learn more about this process in a previous article.

# src module here: https://github.com/vcerqueira/blog/tree/predominant/src
from src.tde import time_delay_embedding

# using 3 lags as explanatory variables
N_LAGS = 3
# forecasting the following 2 values
HORIZON = 2

# using a sliding window method called time delay embedding
X, Y = time_delay_embedding(series, n_lags=N_LAGS, horizon=HORIZON, return_Xy=True)

Here’s a sample of the explanatory variables (X) and corresponding goal variables (Y):

Sample of the explanatory (left) and goal (right) variables after transforming the time series right into a 2-d matrix structure. Image by creator.

This data set is the premise for training traditional machine learning methods. For instance, a linear regression or an xgboost.

from sklearn.linear_model import RidgeCV

# training a ridge regression model
model = RidgeCV()
model.fit(X, Y)

From a matrix right into a three-D structure for deep learning

You have to reshape this data set to coach a neural network like an LSTM. The next function may be used to do that:

import re
import pandas as pd
import numpy as np

def from_matrix_to_3d(df: pd.DataFrame) -> np.ndarray:
"""
Transforming a time series from matrix into three-D structure for deep learning

:param df: (pd.DataFrame) Time series within the matrix format after embedding

:return: Reshaped time series into three-D structure
"""

cols = df.columns

# getting unique variables within the time series
# this list has a single element for univariate time series
var_names = np.unique([re.sub(r'([^)]*)', '', c) for c in cols]).tolist()

# getting commentary for every variable
arr_by_var = [df.loc[:, cols.str.contains(v)].values for v in var_names]
# reshaping the info of every variable right into a three-D format
arr_by_var = [x.reshape(x.shape[0], x.shape[1], 1) for x in arr_by_var]

# concatenating the arrays of every variable right into a single array
ts_arr = np.concatenate(arr_by_var, axis=2)

return ts_arr

# transforming the matrices
X_3d = from_matrix_to_3d(X)
Y_3d = from_matrix_to_3d(Y)

Finally, you’ll be able to train an LSTM using the resulting data set:

from sklearn.model_selection import train_test_split

from keras.models import Sequential
from keras.layers import (Dense,
LSTM,
TimeDistributed,
RepeatVector)

# variety of variables within the time series
# 1 since the series is univariate
N_FEATURES = 1

# creating an easy stacked LSTM
model = Sequential()
model.add(LSTM(8, activation='relu', input_shape=(N_LAGS, N_FEATURES)))
model.add(RepeatVector(HORIZON))
model.add(LSTM(4, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(N_FEATURES)))

model.compile(optimizer='adam', loss='mse')

# compiling the model
model.compile(optimizer='adam', loss='mse')

# basic train/validation split
X_train, X_valid, Y_train, Y_valid = train_test_split(X_3d, Y_3d, test_size=.2, shuffle=False)

# training the model
model.fit(X_train, Y_train, epochs=100, validation_data=(X_valid, Y_valid))

# making predictions
preds = model.predict_on_batch(X_valid)

Multivariate Time Series

Now, let’s take a look at a multivariate time series example. On this case, the goal is to forecast the longer term values of several variables, not only one. So, you wish a model for multivariate and multi-step forecasting.

Multivariate time series concerning the sales of various kinds of wine. Image by creator.

The transformation process is like before.

To rework the multivariate time series right into a matrix format, you’ll be able to apply the sliding window approach to every variable. Then, you mix all resulting matrices right into a single one.

Here’s an example:

# transforming each variable right into a matrix format
mat_by_variable = []
for col in data:
col_df = time_delay_embedding(data[col], n_lags=N_LAGS, horizon=HORIZON)
mat_by_variable.append(col_df)

# concatenating all variables
mat_df = pd.concat(mat_by_variable, axis=1).dropna()

# defining goal (Y) and explanatory variables (X)
predictor_variables = mat_df.columns.str.comprises('(t-|(t)')
target_variables = mat_df.columns.str.comprises('(t+')
X = mat_df.iloc[:, predictor_variables]
Y = mat_df.iloc[:, target_variables]

The explanatory variables appear like this for 2 of the variables (others are omitted for conciseness):

You should utilize the identical function to remodel the info into three dimensions:

X_3d = from_matrix_to_3d(X)
Y_3d = from_matrix_to_3d(Y)

The training part can also be like before. The knowledge concerning the variety of variables within the series is provided within the N_FEATURES constant. Because the name implied, this constant is the variety of variables within the time series.

model = Sequential()
model.add(LSTM(8, activation='relu', input_shape=(N_LAGS, N_FEATURES)))
model.add(Dropout(.2))
model.add(RepeatVector(HORIZON))
model.add(LSTM(4, activation='relu', return_sequences=True))
model.add(Dropout(.2))
model.add(TimeDistributed(Dense(N_FEATURES)))

model.compile(optimizer='adam', loss='mse')

X_train, X_valid, Y_train, Y_valid = train_test_split(X_3d, Y_3d, test_size=.2, shuffle=False)

model.fit(X_train, Y_train, epochs=500, validation_data=(X_valid, Y_valid))

preds = model.predict_on_batch(X_valid)

The next plot shows a sample of one-step ahead forecasts.

One step ahead forecasts by an LSTM. Image by creator.

The forecasts aren’t that good. The time series is small and we didn’t optimize the model in any way. Deep learning methods are known to be data-hungry. So, in case you go for this type of approach, ensure you will have enough data.

2 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here