This second module focuses on the concept of models scores, including the test rating and train rating. Those scores are then used to define overfitting and underfitting, in addition to the concepts of bias and variance.

We’ll also see how you can inspect model’s performance with respect to their complexity and the variety of input samples.

All images by writer.

In the event you didn’t catch it, I strongly recommend my first post of this series — it’ll be way easier to follow along:

The primary concept I would like to discuss are train rating and test rating. The rating is a approach to numericaly express the performance of a model. To compute such performance, we use a rating function, that aggregates the “distance” or “error” between what the model predicted versus what the bottom truth is. For instance:

model = LinearRegressor()
model.fit(X_train, y_train)
y_predicted = model.predict(X_test)
test_score = some_score_function(y_predicted, y_test)

In sklearn, all models (also called estimators) provide a fair quicker approach to compute a rating using the model:

# the model will computed the anticipated y-value from X_test, 
# and compare it to y_test with a rating function
test_score = model.rating(X_test, y_test)
train_score = model.rating(X_train, y_train)

The actual rating function of the model relies on the model and the sort of problem it’s designed to resolve. For instance a linear regressor is the R² coefficient (numerical regression) while a support-verctor classifier (classication) will use the accuracy which is basicaly the number of excellent class-prediction.