Building Logistic Regression from scratch in Python
නමින් වුනත්, මේක classification algorithm එකක්, regression එකක් නෙවෙයි — දැනටමත් හදපු Rock vs Mine සහ Diabetes projects වල "yes/no" predictions වලට power දෙන්නෙ මේකෙන්.
Despite the name, this is a classification algorithm, not regression — it's what powers the "yes/no" predictions in the Rock vs Mine and Diabetes projects you already built.
Lesson 35 of 64 · Part 3 · Building Models from Scratch · Algorithms from Scratch · bilingual notes (English + සිංහල)
Why this matters. Despite the name, this is a classification algorithm, not regression — it's what powers the "yes/no" predictions in the Rock vs Mine and Diabetes projects you already built.
සිංහලෙන්: නමින් වුනත්, මේක classification algorithm එකක්, regression එකක් නෙවෙයි — දැනටමත් හදපු Rock vs Mine සහ Diabetes projects වල "yes/no" predictions වලට power දෙන්නෙ මේකෙන්.
What you'll use this for. Anywhere you need a probability, not just a label — "70% chance of default" instead of just "default" — this is the algorithm underneath.
සිංහලෙන්: Label එකක් විතරක් නෙවෙයි, probability එකක් ඕන ඕනම තැනක — "70% chance of default" කියන එක, "default" කියන එක විතරක් නෙවෙයි — යටින් තියෙන algorithm එකයි මේක.
Key principles
- The sigmoid function is what turns any number into a clean 0-to-1 probability. Linear Regression's raw output can be any number (-500 to +500). Squeezing that through the sigmoid function is the one trick that makes it usable as a probability.
Sigmoid function එකෙන් තමයි ඕනම number එකක්, 0 සිට 1 දක්වා පිරිසිදු probability එකක් බවට හරවන්නෙ
Walkthrough
Logistic Regression:
Y_hat --> predicted value
X --> Input Variable
w --> weight
b --> bias
Gradient Descent:
Gradient Descent is an optimization algorithm used for minimizing the loss function in various machine learning algorithms. It is used for updating the parameters of the learning model.
w = w - α*dw
b = b - α*db
Learning Rate:
Learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function.
Derivatives:
Importing the Dependencies
# importing numpy library
import numpy as npLogistic Regression
class Logistic_Regression():
# declaring learning rate & number of iterations (Hyperparametes)
def __init__(self, learning_rate, no_of_iterations):
self.learning_rate = learning_rate
self.no_of_iterations = no_of_iterations
# fit function to train the model with dataset
def fit(self, X, Y):
# number of data points in the dataset (number of rows) --> m
# number of input features in the dataset (number of columns) --> n
self.m, self.n = X.shape
#initiating weight & bias value
self.w = np.zeros(self.n)
self.b = 0
self.X = X
self.Y = Y
# implementing Gradient Descent for Optimization
for i in range(self.no_of_iterations):
self.update_weights()
def update_weights(self):
# Y_hat formula (sigmoid function)
Y_hat = 1 / (1 + np.exp( - (self.X.dot(self.w) + self.b ) ))
# derivaties
dw = (1/self.m)*np.dot(self.X.T, (Y_hat - self.Y))
db = (1/self.m)*np.sum(Y_hat - self.Y)
# updating the weights & bias using gradient descent
self.w = self.w - self.learning_rate * dw
self.b = self.b - self.learning_rate * db
# Sigmoid Equation & Decision Boundary
def predict(self, X):
Y_pred = 1 / (1 + np.exp( - (X.dot(self.w) + self.b ) ))
Y_pred = np.where( Y_pred > 0.5, 1, 0)
return Y_predKeep going: ← Lesson 34 · Lesson 36 → · Track it in the Self Guide
Finished this one?
Part of Self Guide · Stage 15: ML 3 — Building Models from Scratch. Progress is saved in this browser.
