Gaussian Processes (GPs) are an incredible class of models. There are only a few Machine Learning algorithms that offer you an accurate measure of uncertainty without spending a dime while still being super flexible. The issue is, GPs are conceptually really obscure. Most explanations use some complex algebra and probability, which is commonly not useful to get an intuition for the way these models work.
There are also many great guides that skip the maths and offer you the intuition for the way these models work, but with regards to using GPs yourself, in the fitting context, my personal belief is that surface knowledge won’t cut it. Because of this I desired to walk through a bare-bones implementation, from scratch, so that you simply get a clearer picture of what’s occurring under the hood of all of the libraries that implement these models for you.
I also link my GitHub repo, where you’ll find the implementation of GPs using only NumPy. I’ve tried to abstract from the maths as much as possible, but obviously there remains to be some which can be required…
Step one is all the time to have a have a look at the information. We’re going to use the monthly CO2 atmospheric concentration over time, measured on the Mauna Loa observatory, a typical dataset for GPs [1]. That is intentionally the identical dataset that sklearn use of their GP tutorial, which teaches methods to use their API and never what is happening under the hood of the model.
This can be a quite simple dataset, which can make it easier to clarify the maths that can follow. The notable features are the linear upwards trend in addition to the seasonal trend, with a period of 1 yr.
What we are going to do is separate the seasonal component and linear components of the information. To do that, we fit a linear model to the information.