Linear Regression is a statistical technique used to model the connection between a dependent variable and a number of independent variables. It’s widely utilized in various fields comparable to finance, economics, social sciences, and engineering to make predictions or forecast outcomes.

Within the context of machine learning, Linear Regression is a sort of supervised learning algorithm used for regression tasks. It tries to suit a linear function to a given set of knowledge points by minimizing the difference between the anticipated output and the actual output.

The target of linear regression is to seek out one of the best line of fit (often called the regression line) that describes the connection between the independent and dependent variables. The regression line is a line that minimizes the sum of the squared differences between the observed values and the values predicted by the road.

On this blog, we are going to explore the Kinds of Linear Regression, Assumptions, Mathematical intuition of Easy Linear Regression and its implementation in machine learning.

There are two fundamental varieties of Linear Regression: Easy Linear Regression and Multiple Linear Regression.

Easy Linear Regression

Easy Linear Regression is a linear approach to modelling the connection between a dependent variable and a single independent variable. It’s represented by the equation:

Where Y is the dependent variable, X is the independent variable, β0 is the intercept, β1 is the slope

Multiple Linear Regression

Multiple Linear Regression is an extension of Easy Linear Regression, where the connection between a dependent variable and two or more independent variables is modelled. It’s represented by the equation:

Where Y is the dependent variable, X1, X2, …, Xn are the independent variables, β0 is the intercept, β1, β2, …, βn are the slopes

There are particular assumptions that have to be met for the outcomes of a linear regression model to be valid and reliable. Listed below are the five key assumptions of Linear Regression:

1. Linearity: The connection between the dependent variable and the independent variables have to be linear. Which means that the changes within the dependent variable are directly proportional to the changes within the independent variables.
2. Independence: The observations within the dataset ought to be independent of one another. In other words, the worth of the dependent variable for one commentary mustn’t be influenced by the values of the dependent variable for other observations.
3. Homoscedasticity: The variance of the errors (the differences between the actual and predicted values) ought to be constant across all levels of the independent variables. Which means that the scatter of the residuals ought to be roughly equal across the range of predicted values.
4. Normality: The residuals ought to be normally distributed. Which means that the distribution of the residuals ought to be symmetric around zero, with most residuals near zero and fewer residuals further away.
5. No Multicollinearity: The independent variables mustn’t be highly correlated with one another. If two or more independent variables are highly correlated, it becomes difficult to differentiate the effect of every individual independent variable on the dependent variable.

Now that we all know what Linear Regressions are let’s delve into mathematical Intuition and understand how the slope β1 and intercepts β0 are calculated.

In Linear regression, the fundamental aim is to seek out the optimal values of model parameters (i.e., the slope and intercept) that minimize the sum of squared errors between the anticipated values and the actual values of the dependent variable.

There are several ways to seek out the model parameters in linear regression, including:

1. Closed-form solution: The closed-form solution (also often called the ) is a mathematical formula that directly computes the values of the model parameters that minimize the sum of squared errors between the anticipated values and the actual values of the dependent variable.
2. Non-closed form solution: The non-closed form solution for linear regression refers back to the use of optimization algorithms to estimate the parameters of the model. This is commonly needed when the closed-form solution is either not feasible or not desirable as a consequence of the scale of the dataset or the complexity of the model. One commonly used optimization algorithm is .

Let’s revise in a nutshell suppose we’ve got data points that are linearly related and we’ve got to attract a best-fit line which minimizes the error between the anticipated values and the actual values. The mathematical expression for errors will be found as follows:

The errors are literally the distances between actual and predicted values denoted by d1, d2, d3, d4…dn. Due to this fact, the full error E will be calculated as :

However the distances as also negative so squares of distances are taken and the equation becomes:

But di is given by,

Our aim is to attenuate error E. The values of yi and xi from the error equation are constant so only the values of slope(βi) and intercept(β0) will be used to attenuate the error E. Since E is a function of β0 and βi to seek out the minimum value of E we are going to partially differentiate E each with respect to β0 and βi and make them equal to zero.

Now partially differentiate E with respect to βi,

Within the closed form solution method (odd least square) the values of intercept β0 and slope βi are calculated using the above-derived formulas. The sklearn’s Linear Regression uses the above-derived formulas in its backend to calculate the optimal model parameters.

First import the needed libraries

import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

Next, we’d like to load the dataset and split it into training and testing sets

dataset = pd.read_csv('dataset.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=2)

Then, we create an instance of the LinearRegression class and fit the training data:

regressor = LinearRegression()
regressor.fit(X_train, y_train)

Finally, we are able to make predictions on the test data and evaluate the model using metrics comparable to Mean Squared Error (MSE) and R-squared (R2) rating:

y_pred = regressor.predict(X_test)
mse = np.mean((y_pred - y_test)**2)
r2_score = regressor.rating(X_test, y_test)

Linear Regression is a widely used statistical technique for predictive evaluation in various fields. It will be significant to grasp the varieties of Linear Regression, the assumptions, and the implementation to make sure the accuracy of the predictions. Moreover, it is strongly recommended to guage the model performance using appropriate metrics to make sure its effectiveness.