Implementing SVM from Scratch
දෙවෙනි implementation pass එකක්, බොහොම විට වෙනස් dataset එකකින් — "මම මේක එක පාරක් දැක්කා" කියන එකයි "මට මතකෙන්ම ආයෙත් හදන්න පුළුවන්" කියන එකයි වෙන් කරන්නෙ මේ repeated exposure එකෙන්.
A second implementation pass, likely on a different dataset — the repeated exposure is what separates "I saw this once" from "I could rebuild this from memory."
Lesson 40 of 64 · Part 3 · Building Models from Scratch · Algorithms from Scratch · bilingual notes (English + සිංහල)
Why this matters. A second implementation pass, likely on a different dataset — the repeated exposure is what separates "I saw this once" from "I could rebuild this from memory."
සිංහලෙන්: දෙවෙනි implementation pass එකක්, බොහොම විට වෙනස් dataset එකකින් — "මම මේක එක පාරක් දැක්කා" කියන එකයි "මට මතකෙන්ම ආයෙත් හදන්න පුළුවන්" කියන එකයි වෙන් කරන්නෙ මේ repeated exposure එකෙන්.
What you'll use this for. By this point you have three from-scratch algorithms under your belt (Linear Regression, Logistic Regression, SVM) — genuinely rare among people who only ever call .fit().
සිංහලෙන්: මේ වෙනකොට ඔයාට from-scratch algorithms තුනක් තියෙනවා (Linear Regression, Logistic Regression, SVM) — .fit() විතරක් call කරන අය අතරෙ ඇත්තටම දුර්ලභ දෙයක් තමයි මේක.
Key principles
- Comparing your from-scratch result to scikit-learn's is the real test. If your accuracy is close to scikit-learn's implementation, you've genuinely understood the algorithm — not just copied a formula.
ඔයාගෙ from-scratch result එක scikit-learn එකේ එකට compare කිරීමයි ඇත්තම test එක
Walkthrough
Importing the Dependencies
import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from SVM import SVM_classifierData Collection and Analysis
PIMA Diabetes Dataset
# loading the diabetes dataset to a pandas DataFrame
diabetes_dataset = pd.read_csv('/content/diabetes.csv')# printing the first 5 rows of the dataset
diabetes_dataset.head()| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | … | |
|---|---|---|---|---|---|---|---|---|
| 0 | 6 | 148 | 72 | 35 | 0 | 33.6 | 0.627 | … |
| 1 | 1 | 85 | 66 | 29 | 0 | 26.6 | 0.351 | … |
| 2 | 8 | 183 | 64 | 0 | 0 | 23.3 | 0.672 | … |
| 3 | 1 | 89 | 66 | 23 | 94 | 28.1 | 0.167 | … |
| 4 | 0 | 137 | 40 | 35 | 168 | 43.1 | 2.288 | … |
# number of rows and Columns in this dataset
diabetes_dataset.shapeOutput
(768, 9)# getting the statistical measures of the data
diabetes_dataset.describe()| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | … | |
|---|---|---|---|---|---|---|---|---|
| count | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | … |
| mean | 3.845052 | 120.894531 | 69.105469 | 20.536458 | 79.799479 | 31.992578 | 0.471876 | … |
| std | 3.369578 | 31.972618 | 19.355807 | 15.952218 | 115.244002 | 7.884160 | 0.331329 | … |
| min | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.078000 | … |
| 25% | 1.000000 | 99.000000 | 62.000000 | 0.000000 | 0.000000 | 27.300000 | 0.243750 | … |
| 50% | 3.000000 | 117.000000 | 72.000000 | 23.000000 | 30.500000 | 32.000000 | 0.372500 | … |
… 2 more rows
diabetes_dataset['Outcome'].value_counts()Output
0 500
1 268
Name: Outcome, dtype: int640 --> Non-Diabetic
1 --> Diabetic
diabetes_dataset.groupby('Outcome').mean()| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | Age | |
|---|---|---|---|---|---|---|---|---|
| Outcome | ||||||||
| 0 | 3.298000 | 109.980000 | 68.184000 | 19.664000 | 68.792000 | 30.304200 | 0.429734 | 31.190000 |
| 1 | 4.865672 | 141.257463 | 70.824627 | 22.164179 | 100.335821 | 35.142537 | 0.550500 | 37.067164 |
# separating the data and labels
features = diabetes_dataset.drop(columns = 'Outcome', axis=1)
target = diabetes_dataset['Outcome']print(features)Output
Pregnancies Glucose BloodPressure ... BMI DiabetesPedigreeFunction Age
0 6 148 72 ... 33.6 0.627 50
1 1 85 66 ... 26.6 0.351 31
2 8 183 64 ... 23.3 0.672 32
3 1 89 66 ... 28.1 0.167 21
4 0 137 40 ... 43.1 2.288 33
.. ... ... ... ... ... ... ...
763 10 101 76 ... 32.9 0.171 63
764 2 122 70 ... 36.8 0.340 27
765 5 121 72 ... 26.2 0.245 30
766 1 126 60 ... 30.1 0.349 47
767 1 93 70 ... 30.4 0.315 23
[768 rows x 8 columns]print(target)Output
0 1
1 0
2 1
3 0
4 1
..
763 0
764 0
765 0
766 1
767 0
Name: Outcome, Length: 768, dtype: int64Data Standardization
scaler = StandardScaler()scaler.fit(features)Output
StandardScaler(copy=True, with_mean=True, with_std=True)standardized_data = scaler.transform(features)print(standardized_data)Output
[[ 0.63994726 0.84832379 0.14964075 ... 0.20401277 0.46849198
1.4259954 ]
[-0.84488505 -1.12339636 -0.16054575 ... -0.68442195 -0.36506078
-0.19067191]
[ 1.23388019 1.94372388 -0.26394125 ... -1.10325546 0.60439732
-0.10558415]
...
[ 0.3429808 0.00330087 0.14964075 ... -0.73518964 -0.68519336
-0.27575966]
[-0.84488505 0.1597866 -0.47073225 ... -0.24020459 -0.37110101
1.17073215]
[-0.84488505 -0.8730192 0.04624525 ... -0.20212881 -0.47378505
-0.87137393]]features = standardized_data
target = diabetes_dataset['Outcome']print(features)
print(target)Output
[[ 0.63994726 0.84832379 0.14964075 ... 0.20401277 0.46849198
1.4259954 ]
[-0.84488505 -1.12339636 -0.16054575 ... -0.68442195 -0.36506078
-0.19067191]
[ 1.23388019 1.94372388 -0.26394125 ... -1.10325546 0.60439732
-0.10558415]
...
[ 0.3429808 0.00330087 0.14964075 ... -0.73518964 -0.68519336
-0.27575966]
[-0.84488505 0.1597866 -0.47073225 ... -0.24020459 -0.37110101
1.17073215]
[-0.84488505 -0.8730192 0.04624525 ... -0.20212881 -0.47378505
-0.87137393]]
0 1
1 0
2 1
3 0
4 1
… (7 more lines)Train Test Split
X_train, X_test, Y_train, Y_test = train_test_split(features,target, test_size = 0.2, random_state=2)print(features.shape, X_train.shape, X_test.shape)Output
(768, 8) (614, 8) (154, 8)Training the Model
classifier = SVM_classifier(learning_rate=0.001, no_of_iterations=1000, lambda_parameter = 0.01)#training the support vector Machine Classifier
classifier.fit(X_train, Y_train)Model Evaluation
Accuracy Score
# accuracy score on the training data
X_train_prediction = classifier.predict(X_train)
training_data_accuracy = accuracy_score( Y_train, X_train_prediction)print('Accuracy score of the training data : ', training_data_accuracy)Output
Accuracy score of the training data : 0.7768729641693811# accuracy score on the test data
X_test_prediction = classifier.predict(X_test)
test_data_accuracy = accuracy_score( Y_test, X_test_prediction)print('Accuracy score of the test data : ', test_data_accuracy)Output
Accuracy score of the test data : 0.7532467532467533Building a Predictive System
input_data = (5,166,72,19,175,25.8,0.587,51)
# changing the input_data to numpy array
input_data_as_numpy_array = np.asarray(input_data)
# reshape the array as we are predicting for one instance
input_data_reshaped = input_data_as_numpy_array.reshape(1,-1)
# standardize the input data
std_data = scaler.transform(input_data_reshaped)
print(std_data)
prediction = classifier.predict(std_data)
print(prediction)
if (prediction[0] == 0):
print('The person is not diabetic')
else:
print('The person is diabetic')Output
[[ 0.3429808 1.41167241 0.14964075 -0.09637905 0.82661621 -0.78595734
0.34768723 1.51108316]]
[1]
The person is diabeticKeep going: ← Lesson 39 · Lesson 41 → · Track it in the Self Guide
Finished this one?
Part of Self Guide · Stage 15: ML 3 — Building Models from Scratch. Progress is saved in this browser.
