lasso

Lasso is a regression analysis technique to perform both variable selection as well as model regularization.

Lasso tries to fit a linear model as follows: $y' = \beta_0 + \sum_{i=1}^N x_i . \beta_i$ Where:

• N = input dimensionality
• $x_i$ = input variables i = 1 … N
• $\beta_i$ = model parameters i = 0 … N
• $y'$ = the output

Criterion used to find model paramters is minimizing $\frac{1}{M} \sum_{j=1}^M (y_j - y'_j)^2$ subject to $\sum_{i=0}^N |\beta_i| \leq s$ Where:

• M = number of input samples
• s = a hyper-parameter to control regularization, which is often derived through cross-validation