Home Artificial Intelligence Transform Time Series for Deep Learning Supervised Learning with Time Series Auto-Regression with Deep Learning Hands-On Key Takeaways

Transform Time Series for Deep Learning Supervised Learning with Time Series Auto-Regression with Deep Learning Hands-On Key Takeaways

admin
-
117
Transform Time Series for Deep Learning Supervised Learning with Time Series Auto-Regression with Deep Learning Hands-On Key Takeaways

Photo by Claudio Testa on Unsplash

Supervised learning involves training a machine learning model with an input data set. This data set is normally a matrix: A two-dimensional data structure composed of rows (samples) and columns (features).

A time series is a sequence of values ordered in time. So, it must be transformed for supervised learning.

In a previous article, we learned the way to transform a univariate time series from a sequence right into a matrix. This is finished with a sliding window. Each commentary of the series is modeled based on past recent values, also called lags.

Here’s an example of this transformation using the sequence from 1 to 10:

Transforming a sequence right into a matrix with a sliding window. Image by writer.

This transformation enables a style of modeling called auto-regression. In auto-regression, a model is built using the past recent values (lags) of a time series as explanatory variables. These are used to predict future observations (goal variable). The intuition for the name auto-regression is that the time series is regressed with itself.

In the instance above, the lags are the initial 5 columns. The goal variable is the last column (the worth of the series in the subsequent time step).

While most methods work with matrices, deep neural networks need a special structure.

The input to deep neural networks similar to LSTMs or CNNs is a three-dimensional array. The actual data is identical because the one you’d put in a matrix. But, it’s structured in a different way.

Besides rows (samples) and columns (lags), the additional dimension refers back to the variety of variables within the series. In a matrix, you concatenate all attributes together no matter their source. Neural networks are a bit tidier. The input is organized by each variable within the series using a 3rd dimension.

Let’s do a practical example to make this clear.

Photo by Quino Al on Unsplash

On this tutorial, you’ll learn the way to transform a time series for supervised learning with an LSTM (Long Short-Term Memory). An LSTM is a style of neural network that is particularly useful to model time series.

We’ll split the time series transformation process into two steps:

  1. From a sequence of values right into a matrix;
  2. From a matrix right into a three-D array for deep learning.

First, we’ll do an example with a univariate time series. Multivariate time series are covered next.

Univariate Time Series

Let’s start by reading the info. We’ll use a time series related to the sales of various sorts of wine. You possibly can check the source in reference [1].

import pandas as pd
# https://github.com/vcerqueira/blog/tree/important/data
data = pd.read_csv('data/wine_sales.csv', parse_dates=['date'])
data.set_index('date', inplace=True)
series = data['Sparkling']

We give attention to the sales of sparkling wine to do an example for the univariate case. This time series looks like this:

From a sequence of values right into a matrix

We apply a sliding window to rework this series for supervised learning. You possibly can learn more about this process in a previous article.

# src module here: https://github.com/vcerqueira/blog/tree/important/src
from src.tde import time_delay_embedding
# using 3 lags as explanatory variables
N_LAGS = 3
# forecasting the subsequent 2 values
HORIZON = 2
# using a sliding window method called time delay embedding
X, Y = time_delay_embedding(series, n_lags=N_LAGS, horizon=HORIZON, return_Xy=True)

Here’s a sample of the explanatory variables (X) and corresponding goal variables (Y):

Sample of the explanatory (left) and goal (right) variables after transforming the time series right into a 2-d matrix structure. Image by writer.

This data set is the idea for training traditional machine learning methods. For instance, a linear regression or an xgboost.

from sklearn.linear_model import RidgeCV
# training a ridge regression model
model = RidgeCV()
model.fit(X, Y)

From a matrix right into a three-D structure for deep learning

You should reshape this data set to coach a neural network like an LSTM. The next function may be used to do that:

import re
import pandas as pd
import numpy as np
def from_matrix_to_3d(df: pd.DataFrame) -> np.ndarray:
"""
Transforming a time series from matrix into three-D structure for deep learning
:param df: (pd.DataFrame) Time series within the matrix format after embedding
:return: Reshaped time series into three-D structure 
"""    
cols = df.columns
# getting unique variables within the time series
# this list has a single element for univariate time series
var_names = np.unique([re.sub(r'([^)]*)', '', c) for c in cols]).tolist()
# getting commentary for every variable
arr_by_var = [df.loc[:, cols.str.contains(v)].values for v in var_names]
# reshaping the info of every variable right into a three-D format
arr_by_var = [x.reshape(x.shape[0], x.shape[1], 1) for x in arr_by_var]
# concatenating the arrays of every variable right into a single array
ts_arr = np.concatenate(arr_by_var, axis=2)
return ts_arr
# transforming the matrices
X_3d = from_matrix_to_3d(X)
Y_3d = from_matrix_to_3d(Y)

Finally, you’ll be able to train an LSTM using the resulting data set:

from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import (Dense,
LSTM,
TimeDistributed,
RepeatVector)
# variety of variables within the time series
# 1 since the series is univariate
N_FEATURES = 1
# creating a straightforward stacked LSTM
model = Sequential()
model.add(LSTM(8, activation='relu', input_shape=(N_LAGS, N_FEATURES)))
model.add(RepeatVector(HORIZON))
model.add(LSTM(4, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(N_FEATURES)))
model.compile(optimizer='adam', loss='mse')
# compiling the model
model.compile(optimizer='adam', loss='mse')
# basic train/validation split 
X_train, X_valid, Y_train, Y_valid = train_test_split(X_3d, Y_3d, test_size=.2, shuffle=False)
# training the model
model.fit(X_train, Y_train, epochs=100, validation_data=(X_valid, Y_valid))
# making predictions
preds = model.predict_on_batch(X_valid)

Multivariate Time Series

Now, let’s take a look at a multivariate time series example. On this case, the goal is to forecast the long run values of several variables, not only one. So, you wish a model for multivariate and multi-step forecasting.

Multivariate time series in regards to the sales of various kinds of wine. Image by writer.

The transformation process is like before.

To rework the multivariate time series right into a matrix format, you’ll be able to apply the sliding window approach to every variable. Then, you mix all resulting matrices right into a single one.

Here’s an example:

# transforming each variable right into a matrix format
mat_by_variable = []
for col in data:
col_df = time_delay_embedding(data[col], n_lags=N_LAGS, horizon=HORIZON)
mat_by_variable.append(col_df)
# concatenating all variables
mat_df = pd.concat(mat_by_variable, axis=1).dropna()
# defining goal (Y) and explanatory variables (X)
predictor_variables = mat_df.columns.str.accommodates('(t-|(t)')
target_variables = mat_df.columns.str.accommodates('(t+')
X = mat_df.iloc[:, predictor_variables]
Y = mat_df.iloc[:, target_variables]

The explanatory variables appear to be this for 2 of the variables (others are omitted for conciseness):

You need to use the identical function to rework the info into three dimensions:

X_3d = from_matrix_to_3d(X)
Y_3d = from_matrix_to_3d(Y)

The training part can be like before. The knowledge in regards to the variety of variables within the series is provided within the N_FEATURES constant. Because the name implied, this constant is the variety of variables within the time series.

model = Sequential()
model.add(LSTM(8, activation='relu', input_shape=(N_LAGS, N_FEATURES)))
model.add(Dropout(.2))
model.add(RepeatVector(HORIZON))
model.add(LSTM(4, activation='relu', return_sequences=True))
model.add(Dropout(.2))
model.add(TimeDistributed(Dense(N_FEATURES)))
model.compile(optimizer='adam', loss='mse')
X_train, X_valid, Y_train, Y_valid = train_test_split(X_3d, Y_3d, test_size=.2, shuffle=False)
model.fit(X_train, Y_train, epochs=500, validation_data=(X_valid, Y_valid))
preds = model.predict_on_batch(X_valid)

The next plot shows a sample of one-step ahead forecasts.

One step ahead forecasts by an LSTM. Image by writer.

The forecasts should not that good. The time series is small and we didn’t optimize the model in any way. Deep learning methods are known to be data-hungry. So, for those who go for this sort of approach, ensure you’ve enough data.

117 COMMENTS

  1. … [Trackback]

    […] Read More Infos here: bardai.ai/artificial-intelligence/transform-time-series-for-deep-learningsupervised-learning-with-time-seriesauto-regression-with-deep-learninghands-onkey-takeaways/ […]

  6. … [Trackback]

    […] Find More on that Topic: bardai.ai/artificial-intelligence/transform-time-series-for-deep-learningsupervised-learning-with-time-seriesauto-regression-with-deep-learninghands-onkey-takeaways/ […]

  7. … [Trackback]

    […] Information to that Topic: bardai.ai/artificial-intelligence/transform-time-series-for-deep-learningsupervised-learning-with-time-seriesauto-regression-with-deep-learninghands-onkey-takeaways/ […]

  9. What’s Taking place i’m new to this, I stumbled upon this
    I’ve discovered It absolutely useful and it has aided me out loads.

    I’m hoping to contribute & aid other customers like its helped me.
    Good job.

  15. Interesting blog! Is your theme custom made or did you download it from somewhere?
    A design like yours with a few simple adjustements would really make my blog jump out.
    Please let me know where you got your theme.
    With thanks

  20. Hello! Someone in my Facebook group shared this website with us so I came to look it over.
    I’m definitely enjoying the information. I’m bookmarking and will be tweeting this to my followers!
    Superb blog and great design and style.

  22. Hey there I am so glad I found your web site, I really found you by
    mistake, while I was researching on Google for something
    else, Anyhow I am here now and would just like to say kudos for a marvelous post and a
    all round exciting blog (I also love the theme/design), I don’t have time to go through it all
    at the minute but I have book-marked it
    and also added in your RSS feeds, so when I have time
    I will be back to read more, Please do keep up the superb work.

  24. Hello this is kind of of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding experience so I wanted to get advice from someone with experience.
    Any help would be greatly appreciated!

  29. Greetings, There’s no doubt that your website may be having web browser compatibility problems.

    Whenever I take a look at your web site in Safari, it looks fine however,
    if opening in Internet Explorer, it has some overlapping
    issues. I simply wanted to give you a quick heads up!
    Besides that, wonderful site!

  30. I do agree with all of the ideas you have offered to your post.
    They are very convincing and will definitely work. Nonetheless, the posts are very brief
    for starters. Could you please extend them a bit from next
    time? Thank you for the post.

  33. My relatives every time say that I am killing
    my time here at web, except I know I am getting experience all the time by reading thes
    pleasant posts.

  34. Hey, I think your blog might be having browser compatibility issues.
    When I look at your blog site in Opera, it looks fine but when opening in Internet Explorer, it has some overlapping.
    I just wanted to give you a quick heads up!
    Other then that, very good blog!

  36. Simply want to say your article is as surprising.
    The clearness in your post is simply nice and i could assume you’re an expert on this
    subject. Fine with your permission allow me to grab your feed to keep
    updated with forthcoming post. Thanks a million and please continue
    the enjoyable work.

  39. I believe that is one of the such a lot vital info for me.
    And i’m happy reading your article. However should observation on few common things, The website taste is
    great, the articles is actually great : D.

    Excellent process, cheers

  40. Somebody necessarily help to make significantly articles I would state.

    This is the very first time I frequented your web page and up to now?
    I amazed with the research you made to make this particular post extraordinary.
    Fantastic activity!

  41. It’s a pity you don’t have a donate button! I’d definitely
    donate to this outstanding blog! I guess for now i’ll settle for bookmarking and adding your
    RSS feed to my Google account. I look forward to brand new updates and will talk about
    this website with my Facebook group. Chat soon!

  43. I think this is one of the most important info for me. And
    i am glad reading your article. But wanna remark on some general things, The web
    site style is ideal, the articles is really great : D.
    Good job, cheers

  44. Have you ever considered about including a little bit more than just your articles?
    I mean, what you say is fundamental and everything. Nevertheless think about if you added
    some great graphics or video clips to give your posts
    more, “pop”! Your content is excellent but with pics and clips, this website could certainly be one of
    the very best in its field. Great blog!

  45. I was curious if you ever considered changing the page layout of your site?

    Its very well written; I love what youve got to say.
    But maybe you could a little more in the way of content so people could connect with
    it better. Youve got an awful lot of text for only having 1 or two pictures.
    Maybe you could space it out better?

  47. I believe this is one of the so much significant info for me.
    And i’m satisfied studying your article. However should remark on some general issues, The site taste is perfect, the articles is actually nice :
    D. Just right process, cheers

  48. I’m really enjoying the design and layout of your website. It’s a very easy on the eyes which makes it much more pleasant
    for me to come here and visit more often. Did you hire out
    a designer to create your theme? Exceptional work!

  50. naturally like your web-site however you have to test the spelling on quite
    a few of your posts. Many of them are rife with spelling
    issues and I to find it very troublesome to tell the truth however I will surely come again again.

  51. I was curious if you ever thought of changing the structure
    of your blog? Its very well written; I love what youve got to say.
    But maybe you could a litle more in the way of content so people coul connect with it better.

    Youvge got an awful lot of text for only having 1 or two pictures.
    Maybe you could space it out better?

    Also visit my site … 카지노사이트

  52. It’s a shame you don’t have a donate button! I’d most certainly donate to this superb blog!
    I guess for now i’ll settle for book-marking
    and adding your RSS feed to my Google account. I look forward to new updates and
    will share this site with my Facebook group. Chat
    soon!

  55. Does your website have a contact page? I’m having problems locating it but, I’d like
    to send you an e-mail. I’ve got some suggestions for your blog you
    might be interested in hearing. Either way, great blog and I
    look forward to seeing it improve over time.

  59. Great beat ! I would like to apprentice whilst you amend your site,
    how can i subscribe for a blog web site? The account helped me a acceptable
    deal. I had been tiny bit acquainted of this your broadcast offered brilliant clear concept

  64. I’m curious to find out what blog system you’re working with?
    I’m having some minor security problems with my latest blog and I’d like to find something more risk-free.
    Do you have any solutions?

    Also visit my webpage Erotik Forum

  65. You really make it seem so easy with your presentation but I find this topic to be really something that I think I
    would never understand. It seems too complicated and extremely broad for me.
    I am looking forward for your next post, I’ll try to get the hang of it!

  67. Good day! I know this is kind of off topic but I was wondering if you
    knew where I could find a captcha plugin for my comment form?
    I’m using the same blog platform as yours and
    I’m having trouble finding one? Thanks a lot!

  70. Magnificent beat ! I would like to apprentice while you amend your site, how could i subscribe for
    a blog web site? The account helped me a acceptable deal.
    I have been a little bit familiar of this your broadcast offered brilliant clear concept

  72. You can definitely see your expertise in the article you write.
    The sector hopes for more passionate writers like you who are
    not afraid to mention how they believe. Always go after your heart.

  74. Generally I do not learn article on blogs, however I would like to say
    that this write-up very forced me to take a look at and do it!

    Your writing taste has been amazed me. Thanks, quite great post.

  76. Hi! I know this is somewhat off topic but I was wondering which blog platform are you using for this site?
    I’m getting fed up of WordPress because I’ve had problems with hackers and I’m looking at options for another platform.
    I would be great if you could point me in the direction of a good platform.

  77. My spouse and I stumbled over here coming from a different page and thought I might as well check things out.
    I like what I see so now i’m following you. Look forward to finding
    out about your web page repeatedly.

  88. Hey great blog! Does running a blog like this require a lot of work?
    I have no expertise in computer programming however I had been hoping to start my own blog in the near future.
    Anyhow, should you have any recommendations or techniques for new blog owners please share.
    I know this is off subject but I just wanted to ask.

    Thanks a lot!

  89. Hi there! I know this is somewhat off topic but I was wondering which blog
    platform are you using for this site? I’m getting fed up of WordPress because I’ve had problems with hackers and I’m
    looking at alternatives for another platform. I would be awesome if you could point me in the direction of a good platform.

  90. I’m really inspired along with your writing talents and also with the layout for your blog.
    Is that this a paid subject or did you customize it your self?
    Anyway stay up the excellent quality writing, it’s rare to see a great blog
    like this one nowadays..

  91. Hi just wanted to give you a quick heads up and let you know a few of the pictures aren’t loading correctly.

    I’m not sure why but I think its a linking issue.
    I’ve tried it in two different browsers and both show the same outcome.

  93. A motivating discussion is definitely worth comment.
    There’s no doubt that that you ought to write more about this subject matter, it
    may not be a taboo matter but usually people don’t discuss such topics.
    To the next! Many thanks!!

  94. Thank you for some other wonderful article.
    The place else could anyone get that kind of information in such
    an ideal approach of writing? I’ve a presentation next week, and I’m
    on the search for such info.

  95. I blog often and I seriously appreciate your information. This great article has really peaked my interest.
    I’m going to book mark your blog and keep checking for new details about once per week.
    I subscribed to your RSS feed too.

  100. That is very interesting, You are an overly professional blogger.
    I’ve joined your rss feed and sit up for looking for extra of your
    wonderful post. Additionally, I have shared your website in my social networks

  104. Thanks for the marvelous posting! I genuinely enjoyed reading it,
    you might be a great author. I will remember to bookmark your blog and definitely will come
    back someday. I want to encourage that you continue your
    great work, have a nice afternoon!

  106. A person necessarily assist to make critically articles I might
    state. That is the very first time I frequented your website page and
    so far? I amazed with the analysis you made to create this particular put up extraordinary.

    Magnificent activity!

  107. hey there and thank you for your info – I’ve definitely
    picked up something new from right here. I did
    however expertise several technical points using this web site, since
    I experienced to reload the website many times previous to I
    could get it to load properly. I had been wondering if
    your web hosting is OK? Not that I’m complaining, but sluggish
    loading instances times will often affect your placement in google and could damage your high-quality score if advertising and marketing with Adwords.
    Well I’m adding this RSS to my email and can look out for a lot more of
    your respective exciting content. Make sure you update this again very soon.

  113. We absolutely love your blog and find a lot of your post’s to be just what I’m looking for.
    can you offer guest writers to write content in your case?
    I wouldn’t mind creating a post or elaborating on some of the subjects you write regarding here.
    Again, awesome web log!

  115. Hey There. I found your blog using msn. This is a very well written article.
    I will make sure to bookmark it and come back to read more of your
    useful information. Thanks for the post. I will definitely return.

  116. Las Vegas Tankless Water Heaters is the premier
    choice for every water heating solution in Las Vegas.
    With a commitment to offering energy-efficient and reliable instantaneous water heaters, we promise utmost satisfaction for all
    our clients.

    Our Edge in Las Vegas Plumbing Services

    Expertise in Tankless Water Heaters
    Our experts specializes in compact water heaters, offering endless hot water without the limitations of traditional units.
    Our focus makes us unique in the Las Vegas plumbing sector.

    Energy Efficiency
    We focus on eco-friendly water heating, helping you save on utility bills while experiencing reliable hot water.
    The systems we install seek to improve efficiency and sustainability.

    Customized Solutions
    Understanding that each household has different
    needs, which is why we deliver tailored water heating
    solutions. Whether you need a point-of-use heater for a single bathroom or a large-capacity system, we can handle it.

    Professional Installation and Maintenance
    Not only do we install your system, we make sure the longevity of your water
    heater with expert maintenance services. Our team is trained in up-to-date maintenance
    practices for a variety of systems.

    Customer Satisfaction
    The happiness of our clients is our foremost concern. We
    work tirelessly to ensure satisfaction with every service call, offering attentive customer service
    and professional recommendations.

    Wrapping Up, we is not just another plumbing service in Las Vegas for anyone in need of advanced water heating solutions.
    Through our dedication to energy efficiency, personalized systems, and unmatched customer service, we promise
    optimal hot water experience without the hassle.

    Choose Las Vegas Tankless Water Heaters for your next hot water
    system upgrade and join the community of happy homeowners enjoying endless
    hot water in Las Vegas.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

© Newspaper WordPress Theme by TagDiv