STAT 413 - Intro to Statistical Learning

Regularized OLS

1. Be the Optimizer (why we need gradient descent)
2. Gradient Descent in 1D (step size & initialization)
3. Gradient Descent in 2D (partial derivatives)
4. Loss Surfaces: OLS vs Ridge vs Lasso (3D + contours, real data)
5. Regularization Paths (OLS vs Ridge vs Lasso)
6. Bias-Variance Decomposition: OLS, Ridge, Lasso (Monte Carlo, train vs test)
7. Why Lasso Zeroes Coefficients (diamond vs disk)
8. Polynomial Regression
9. K-fold Cross Validation
10. Underfitting and overfitting

Classification

1. Logistic Regression
2. Confusion matrix for 2 classes
3. ROC Curves and Bayes optimal classifier

Trees and Ensembles

1. Decision tree inference
2. Fitting a decision tree
3. Bagging
4. Fitting a random forest tree