If I asked you the right way to evaluate a regression problem, you’ll probably name quite a couple of evaluation metrics, reminiscent of MSE, MAE, RMSE, MAPE, etc. What these metrics have in common is that they concentrate on point predictions.
The situation changes a bit when we would like to coach our models to concentrate on predicting distributions as an alternative of a single point. In that case, we’d like to make use of different metrics, which are usually not as commonly covered in data science blog posts.
Last time, I looked into quantile loss (a.k.a. pinball loss). This time, I’ll walk you thru one other metric used to guage probabilistic forecasts — the Continuous Ranked Probability Rating (CRPS).
The primary concept is a simple one, but it surely continues to be necessary to ensure we’re on the identical page. Probabilistic forecasts provide a distribution of possible outcomes. For instance, while point forecasts would predict tomorrow’s temperature as exactly 23°C, a probabilistic model might predict a 70% probability the temperature shall be between 20°C and 25°C.
