Pandas Compatibility and More in Scikit-Learn Version ≥1.2.0

Around last 12 months December, Scikit-Learn released a serious stable update (v. 1.2.0–1) and at last I get to try among the highlighted recent features. It’s now more compatible with Pandas and a number of other features may also help us in regression in addition to classification tasks. Below, I am going through among the recent updates with examples of methods to use them. Let’s begin!

Compatibility with Pandas:

Applying some data standardization before using them for training an ML model like regression or neural net is a typical technique to be sure that different features with various ranges get equal importance (if or when obligatory) for predictions. Scikit-Learn provides various pre-processing APIs like StandardScaler , MaxAbsScaler etc. With the newer version, it is feasible to maintain the dataframe format even after the pre-processing, let’s see below:

from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
########################
X, y = load_wine(as_frame=True, return_X_y=True) 
# available from version >=0.23; as_frame
########################
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, 
random_state=0)
X_train.head(3)

The newer version includes an option to maintain this dataframe format even after the standardization:

############
# v1.2.0
############
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().set_output(transform="pandas") 
## change here
scaler.fit(X_train)
X_test_scaled = scaler.transform(X_test)
X_test_scaled.head(3)

Before, it could have modified the format to a Numpy array:

###########
# v 0.24
########### 
scaler.fit(X_train)
X_test_scaled = scaler.transform(X_test)
print (type(X_test_scaled))
>>> 

With the dataframe format remaining intact, we don’t have to keep tabs on the columns, like we wanted to do with the Numpy array format. Evaluation and plotting grow to be easier:

fig = plt.figure(figsize=(8, 5))
fig.add_subplot(121)
plt.scatter(X_test['proline'], X_test['hue'], 
c=X_test['alcohol'], alpha=0.8, cmap='bwr')
clb = plt.colorbar()
plt.xlabel('Proline', fontsize=11)
plt.ylabel('Hue', fontsize=11)
fig.add_subplot(122)
plt.scatter(X_test_scaled['proline'], X_test_scaled['hue'], 
c=X_test_scaled['alcohol'], alpha=0.8, cmap='bwr')
# pretty easy now within the newer version to see the effect
plt.xlabel('Proline (Standardized)', fontsize=11)
plt.ylabel('Hue (Standardized)', fontsize=11)
clb = plt.colorbar()
clb.ax.set_title('Alcohol', fontsize=8)
plt.tight_layout()
plt.show()

Even after we construct a pipeline, each transformer within the pipeline might be configured to return dataframes as below:

from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC
clf = make_pipeline(StandardScaler(), SVC())
clf.set_output(transform="pandas") # change here 
svm_fit = clf.fit(X_train, y_train)
print (clf[:-1]) # StandardScaler 
print ('check that set_output format indeed stays even after we construct a pipleline: ', 'n')
X_test_transformed = clf[:-1].transform(X_test)
X_test_transformed.head(3)

Fetching DataSet is Faster and More Efficient:

OpenML is an open platform for sharing datasets and the Dataset API in Sklearn offers fetch_openml function to fetch data; With the updated Sklearn, this step is more efficient in memory and time.

from sklearn.datasets import fetch_openml
start_t = time.time()
X, y = fetch_openml("titanic", version=1, as_frame=True, 
return_X_y=True, parser="pandas")
# # parser pandas is the addition within the version 1.2.0
X = X.select_dtypes(["number", "category"]).drop(columns=["body"])
print ('check types: ', type(X), 'n',  X.head(3))
print ('check shapes: ', X.shape)
end_t = time.time()
print ('time taken: ', end_t-start_t)

Using parser='pandas' makes a drastic improvement in runtime and memory consumption. One can easily check the memory consumption using psutil library as:

print(psutil.cpu_percent())

Partial Dependency Plots: Categorical Features

Partial dependency plots existed before too, but just for numerical features, now this has been prolonged for categorical features.

As described within the Sklearn documentation:

Partial dependence plots show the dependence between the targets and a set of input feature(s) of interest, marginalizing over the values of all other input features (the ‘complement’ features). Intuitively, we are able to interpret the partial dependence because the expected goal response as a function of the input features of interest.

Using the ‘titanic’ dataset from above, we are able to easily plot the partial dependence of categorical features:

With the code block above, we are able to get partial dependency plots as below:

With version 0.24, we could be getting a worth error for categorical variables:

>>> ValueError: couldn't convert string to drift: ‘female’

Directly Plot Residuals (Regression Models):

For analyzing the performance of a classification model, inside Sklearn metrics API, plotting routines like PrecisionRecallDisplay , RocCurveDisplay existed in older versions (0.24); In the brand new update, it is feasible to do similar for regression models. Let’s see an example below:

While it’s at all times possible to plot the fitted line and residuals using matplotlib or seaborn, after we’ve settled down with the very best model, it’s great to find a way to quickly check the outcomes directly inside Sklearn environment.

There are a number of more improvements/additions available in the brand new Sklearn, but I discovered these 4 major improvements to be particularly useful for normal data evaluation most of the time.

References:

[1] Sklearn Release Highlights: V 1.2.0

[2] Sklearn Release Highlights: Video

[3]All of the plots and codes: My GitHub

In the event you’re eager about further fundamental machine learning concepts and more, you may consider joining Medium using My Link. You won’t pay anything extra but I’ll get a tiny commission. Appreciate you all!!