Building Lasso Regression from Scratch in Python
සාමාන්‍ය Linear Regression, ඇත්තටම වැදගත් නැති features වලටත් weight දීලා overfit වෙන්න පුළුවන්. Lasso එකෙන් වැදගත් නැති features zero දෙසට force කරන penalty එකක් එකතු කරනවා, model එක automatic ම simplify කරනවා.
Regular Linear Regression can overfit by giving weight to features that don't really matter. Lasso adds a penalty that forces unimportant features toward zero, automatically simplifying the model.
Lesson 41 of 64 · Part 3 · Building Models from Scratch · Algorithms from Scratch · bilingual notes (English + සිංහල)
Why this matters. Regular Linear Regression can overfit by giving weight to features that don't really matter. Lasso adds a penalty that forces unimportant features toward zero, automatically simplifying the model.
සිංහලෙන්: සාමාන්‍ය Linear Regression, ඇත්තටම වැදගත් නැති features වලටත් weight දීලා overfit වෙන්න පුළුවන්. Lasso එකෙන් වැදගත් නැති features zero දෙසට force කරන penalty එකක් එකතු කරනවා, model එක automatic ම simplify කරනවා.
What you'll use this for. Whenever you have many candidate features and suspect only a few actually matter (a common situation in real datasets), Lasso is the tool for automatic feature selection.
සිංහලෙන්: Candidate features ගොඩක් තියෙනකොට, ඇත්තටම වැදගත් ටික ගණනක් විතරයි කියලා ඔයාට හිතෙනකොට (සැබෑ datasets වල නිතර වෙන දෙයක්), automatic feature selection එකට tool එක Lasso.
Key principles
- The penalty term is a dial between simplicity and fit. Turn the penalty up and the model gets simpler (fewer meaningful features) but may fit the training data less tightly — this trade-off is the entire idea behind regularization.
Penalty term එක කියන්නෙ simplicity සහ fit අතර dial එකක්
Walkthrough
Lasso Regression:
Y = wX + b
Y --> Dependent Variable
X --> Independent Variable
w --> weight
b --> bias
Gradient Descent:
Gradient Descent is an optimization algorithm used for minimizing the loss function in various machine learning algorithms. It is used for updating the parameters of the learning model.
w = w - α*dw
b = b - α*db
Learning Rate:
Learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function.
Gradient for Weights
Gradient for Bias
Importing the Dependencies
import numpy as npLasso Regression
# creating a class for Lasso Regression
class Lasso_Regression():
#initiating the hyperparameters
def __init__(self, learning_rate, no_of_iterations, lambda_parameter):
self.learning_rate = learning_rate
self.no_of_iterations = no_of_iterations
self.lambda_parameter = lambda_parameter
# fitting the dataset to the Lasso Regression model
def fit(self, X, Y):
# m --> number of Data points --> number of rows
# n --> number of input features --> number of columns
self.m, self.n = X.shape
self.w = np.zeros(self.n)
self.b = 0
self.X = X
self.Y = Y
# implementing Gradient Descent algorithm for Optimization
for i in range(no_of_iterations):
self.upadte_weights()
# function for updating the weight & bias value
def upadte_weights(self):
# linear equation of the model
Y_prediction = self.predict(self.X)
# gradients (dw, db)
# gradient for weight
dw = np.zeros(self.n)
for i in range(self.n):
if self.w[i]>0:
dw[i] = (-(2*(self.X[:,i]).dot(self.Y - Y_prediction)) + self.lambda_parameter) / self.m
else :
dw[i] = (-(2*(self.X[:,i]).dot(self.Y - Y_prediction)) - self.lambda_parameter) / self.m
# gradient for bias
db = - 2 * np.sum(self.Y - Y_prediction) / self.m
# updating the weights & bias
self.w = self.w - self.learning_rate*dw
self.b = self.b - self.learning_rate*db
# Predicting the Target variable
def predict(self,X):
return X.dot(self.w) + self.bKeep going: ← Lesson 40 · Lesson 42 → · Track it in the Self Guide
Finished this one?
Part of Self Guide · Stage 15: ML 3 — Building Models from Scratch. Progress is saved in this browser.
