In this guide, we will learn how to build a multiple linear regression model with Sci-kit learn. Unlike the Simple Linear Regression model that uses a single feature to make predictions, the Multiple Linear Regression model uses more than one feature to make predictions. It shows the relationship between multiple independent variables and a dependent variable. You can read the guide on how to build a Simple Linear Regression model with Sci-kit Learn by clicking here.

## Prerequisite

- A PC with Jupyter Notebook IDE
- Advertising dataset from Kaggle
- Basic Knowledge of Multiple Linear Regression

## Multiple linear regression

The formula of multiple linear regression is:**y = Î¸0 + Î¸1 x1 + Î¸2 x2 + â€¦ + Î¸n xn**where n represents the number of features in our dataset

For our dataset with 3 features, we will have the formula**Sales = Î¸0 + Î¸1 * TV + Î¸2 * radio + Î¸3 * newspaper**

**IMPORTING DATASET**Let us import the libraries we will be using.

We will now load our data.

We have 5 columns in our dataset. We will remove the â€˜Unnamed: 0â€™ since we donâ€™t need the index column.

**Exploratory Data Analysis**

We have 3 features and a target variable in our dataset. Letâ€™s explore the relationship between the features and the target variable.

Using the seaborn heatmap, we can see the correlation of each column to the other column. The TV column has the highest correlation to the Sales column, which is 0.78.

**Feature Selection Using R2**

Feature Selection involves choosing features that will help us build a good model. In most cases, not all the features in our dataset will help us get a good model. We have to remove irrelevant features that can reduce the performance of our model. In this guide, we will be using R squared.

R squared (R2) is a goodness of fit measure for linear regression models. It measures the strength of the relationship between your model and the dependent variable. It ranges from 0 â€“ 1. The larger the R2, the better the regression model fits the observations. You can read more about R2 from here.

We will import the r2 score function from the metrics module of the sklearn library. We will import the *LinearRegression*() class from the *linear_model* module of the *sklearn* library.

We will also extract our features and the target variable.

We will now fit the linear regression model to our dataset. We are using just the TV and newspaper features and we are training on and evaluating the entire dataset. Our goal here is to see the R2 score.

Our R2 score is approximately 0.6458. Now we will use just the TV and radio features this time and see what our R2 score will be.

We get an R2 score of 0.8972, which is higher than what we had when using the *TV* and *newspaper* features.

So by adding the *newspaper* feature, our R2 score did not improve by much. We can drop the newspaper feature from the model. Keeping the newspaper feature will likely lead to poor results on the test set.

**Evaluating the Model**

We will split our dataset into the training and test set, and we will use the Root Mean Squared Error (RMSE) to evaluate our model. We will see if the *newspaper* feature should be kept in the model. Letâ€™s use all three features and see the RMSE and R2 on the test set.

Our RMSE and R2 are approximately 1.4047 and 0.91562 respectively. Letâ€™s use just the *TV* and *radio* features and see what our RMSE and R2 will be.

Our RMSE and R2 are approximately 1.3879 and 0.9176 respectively. Our RMSE has reduced and the R2 has improved. This shows that the lm5 model is better than the lm4 model. So without the newspaper feature, we get a better model. Now letâ€™s try to see what our RMSE and R2 will be if we should use the *TV* and *newspaper* column.

So with just the *TV* and *newspaper* features, we get a really bad model with an RMSE of approximately 3.3520 which is high and the R2 score is low.

The best model so far is the lm5 model. It has the highest R2 score and the lowest RMSE. So we will use the lm5 model.

**Conclusion**

In this guide, we learned how to build a multiple linear regression model with sci-kit learn. We also learned how to do feature selection using R2. Feature Selection helps us to choose the features that will give us a good model. We used R2 to measure the strength of the relationship between our model and the dependent variable.

The Github repo is *here*.