Building Linear Regression from scratch in Python

`model.fit()` කියලා call කරොත් ඇත්තටම වෙන දේ හංගිලා තියෙනවා. එකපාරක් ඔයාම හදන එක තමයි, ඉස්සරහට එන හැම scikit-learn call එකක්ම magic එකක් වෙනුවට තේරෙන දෙයක් කරන්නෙ.

Calling `model.fit()` hides what's actually happening. Building it yourself, once, is what makes every future scikit-learn call make sense instead of feeling like magic.

11 May 20263 min readSelf Guide · Stage 15#Building Models from Scratch#Algorithms from Scratch#Machine learning

Lesson 32 of 64 · Part 3 · Building Models from Scratch · Algorithms from Scratch · bilingual notes (English + සිංහල)

Why this matters. Calling model.fit() hides what's actually happening. Building it yourself, once, is what makes every future scikit-learn call make sense instead of feeling like magic.

සිංහලෙන්: model.fit() කියලා call කරොත් ඇත්තටම වෙන දේ හංගිලා තියෙනවා. එකපාරක් ඔයාම හදන එක තමයි, ඉස්සරහට එන හැම scikit-learn call එකක්ම magic එකක් වෙනුවට තේරෙන දෙයක් කරන්නෙ.

What you'll use this for. This is the single most-used algorithm in all of ML for continuous predictions — from house prices to sales forecasting — so it's worth understanding to its foundation.

සිංහලෙන්: Continuous predictions වලට මුළු ML එකේම වැඩිපුරම පාවිච්චි වෙන algorithm එකයි මේක — house prices ඉඳන් sales forecasting දක්වා — ඒ නිසා root එකෙන්ම තේරුම් ගැනීම වටිනවා.

Key principles

A "model" is just numbers (weights) it's trying to get right. Linear Regression is literally searching for the slope and intercept that make a line fit the data best — there's no hidden intelligence beyond that search.
"Model" එකක් කියන්නෙ, හරි කරගන්න හදන numbers (weights) විතරයි
Gradient descent is just "try, check the error, adjust, repeat". The model nudges its weights a little in the direction that reduces error, over and over, until the error stops shrinking meaningfully.
Gradient descent කියන්නෙ "try කරන්න, error එක check කරන්න, adjust කරන්න, repeat කරන්න" විතරයි

Walkthrough

Linear Regression:

Y = wX + b

Y --> Dependent Variable

X --> Independent Variable

w --> weight

b --> bias

Gradient Descent:

Gradient Descent is an optimization algorithm used for minimizing the loss function in various machine learning algorithms. It is used for updating the parameters of the learning model.

w = w - α*dw

b = b - α*db

Learning Rate:

Learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function.

ExamplePython

# importing numpy library
import numpy as np

Linear Regression

ExamplePython

class Linear_Regression():

  # initiating the parameters (learning rate & no. of iterations)
  def __init__(self, learning_rate, no_of_iterations):

    self.learning_rate = learning_rate
    self.no_of_iterations = no_of_iterations

  def fit(self, X, Y ):

    # number of training examples & number of features

    self.m, self.n = X.shape  # number of rows & columns

    # initiating the weight and bias 

    self.w = np.zeros(self.n)
    self.b = 0
    self.X = X
    self.Y = Y

    # implementing Gradient Descent
    
    for i in range(self.no_of_iterations):
      self.update_weights()

  def update_weights(self):

    Y_prediction = self.predict(self.X)

    # calculate gradients

    dw = - (2 * (self.X.T).dot(self.Y - Y_prediction)) / self.m

    db = - 2 * np.sum(self.Y - Y_prediction)/self.m

    # upadating the weights
    
    self.w = self.w - self.learning_rate*dw
    self.b = selb.b - self.learning_rate*db
 

  def predict(self, X):

    return X.dot(self.w) + self.b