Lasso and Ridge Regression (L1 and L2 Regularization):

 Lasso and Ridge Regression 

(L1 and L2 Regularization):

When the regression is under fitting then the accuracy of the test data may be low. When the regression is over fitting then the algorithm won’t be flexible enough to predict the value. 

Accuracy for the train data will be high but for the test data accuracy will be low. So, Lasso and Ridge regressions try to find balanced best fit line in between under fitting and over fitting lines.

Under Fitting in regression

LASSO(L2 Regularization):

LASSO stands for Least Absolute Shrinkage and Selection operator.

The linear regression equation is,

To reduce the complexity and to make the equation simple,we shrink the parameters, that is we make Ө3 ,Ө4 …..Өn to zero and the equation becomes

λ is the penalization factor. We can control the values of λ. When Өi2 value is high the mean square value also increases. So by adding the penalty it controls the Ө value and doesn’t allow going too high.

Ridge (L1 Regularization):

The difference between Lasso and Ridge regularization is in Lasso Өi2 is used and in Ridge |Өi| absolute value is used.