9.28: Penalized Regression

Learning objectives

  • describe why penalized regression methods are useful

  • fit and interpret a LASSO regression model

Classwise videos

If you have questions as you watch the videos, feel free to send me an email or slack message! I will address common questions at the beginning of class.

The first video introduces the challenges of high-dimensional data.

The second video discusses penalized regression with a focus on LASSO.

Textbook

Shrinkage methods: ISLR 6.2

Considerations in high dimensions: ISLR 6.4

Application exercise

Follow this tutorial to fit a lasso model with the Boston housing data we used last week (the dataset is in the ISLR2 package). The tutorial does a nice job of building the code piece by piece to connect the concepts we’ve discussed: lasso, standardizing predictors, cross validation, and tuning the lambda parameter. You can skip the “Basic Lasso Regression in R” section and go straight to using the caret package that we’ve used before, though you will also need to install and load the elasticnet package.

Go further

  • Compare your results to the RMSE you get from least squares estimation

  • Compare your results to Ridge regression using method='ridge'