Cross Validation Python Implementation

Train/test split එකක් chance එකකින් lucky හෝ unlucky වෙන්න පුළුවන්. Cross-validation එකෙන් ඒ split එක කිහිප වතාවක්ම run කරලා average එකක් ගන්නවා, ඒකෙන් ලැබෙන්නෙ ගොඩක් වඩා විශ්වාස කරන්න පුළුවන් performance estimate එකක්.

A single train/test split can get lucky or unlucky by chance. Cross-validation runs that split multiple times and averages the result, giving you a far more trustworthy performance estimate.

15 May 20265 min readSelf Guide · Stage 16#Evaluating & Choosing Models#Section 8#Machine learning

Lesson 43 of 64 · Part 4 · Evaluating & Choosing Models · Section 8 · bilingual notes (English + සිංහල)

Why this matters. A single train/test split can get lucky or unlucky by chance. Cross-validation runs that split multiple times and averages the result, giving you a far more trustworthy performance estimate.

සිංහලෙන්: Train/test split එකක් chance එකකින් lucky හෝ unlucky වෙන්න පුළුවන්. Cross-validation එකෙන් ඒ split එක කිහිප වතාවක්ම run කරලා average එකක් ගන්නවා, ඒකෙන් ලැබෙන්නෙ ගොඩක් වඩා විශ්වාස කරන්න පුළුවන් performance estimate එකක්.

What you'll use this for. Whenever you're deciding between two models or two sets of hyperparameters, cross-validation is what gives you a fair, reliable comparison instead of a lucky guess.

සිංහලෙන්: Model දෙකක් අතර හෝ hyperparameters set දෙකක් අතර තෝරගන්නකොට, lucky guess එකක් නෙවෙයි, සාධාරණ, විශ්වාස කරන්න පුළුවන් comparison එකක් දෙන්නෙ cross-validation එකෙන්.

Key principles

K-fold means every data point gets to be in the test set exactly once. With 5-fold cross-validation, the model is trained and tested 5 times, each time on a different slice — so the final score reflects the whole dataset, not one lucky split.
K-fold කියන්නෙ data point එකකට exactly එක වතාවක් test set එකේ ඉන්න ලැබෙන එක

Walkthrough

Video Link:

K-Fold cross validation - concept ideo: https://youtu.be/dkd1eeaXbJA

Importing the Dependencies

ExamplePython

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
from sklearn.metrics import accuracy_score

Importing the models

ExamplePython

from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier

Data Collection and Processing

ExamplePython

# loading the csv data to a Pandas DataFrame
heart_data = pd.read_csv('/content/heart.csv')

ExamplePython

# print first 5 rows of the dataset
heart_data.head()

	age	sex	cp	trestbps	chol	fbs	restecg	…
0	63	1	3	145	233	1	0	…
1	37	1	2	130	250	0	1	…
2	41	0	1	130	204	0	0	…
3	56	1	1	120	236	0	1	…
4	57	0	0	120	354	0	1	…

ExamplePython

# number of rows and columns in the dataset
heart_data.shape

Output

Output

(303, 14)

ExamplePython

# checking for missing values
heart_data.isnull().sum()

Output

Output

age         0
sex         0
cp          0
trestbps    0
chol        0
fbs         0
restecg     0
thalach     0
exang       0
oldpeak     0
slope       0
ca          0
thal        0
target      0
dtype: int64

ExamplePython

# checking the distribution of Target Variable
heart_data['target'].value_counts()

Output

Output

1    165
0    138
Name: target, dtype: int64

1 --> Defective Heart

0 --> Healthy Heart

Splitting the Features and Target

ExamplePython

X = heart_data.drop(columns='target', axis=1)
Y = heart_data['target']

ExamplePython

print(X)

Output

Output

     age  sex  cp  trestbps  chol  ...  exang  oldpeak  slope  ca  thal
0     63    1   3       145   233  ...      0      2.3      0   0     1
1     37    1   2       130   250  ...      0      3.5      0   0     2
2     41    0   1       130   204  ...      0      1.4      2   0     2
3     56    1   1       120   236  ...      0      0.8      2   0     2
4     57    0   0       120   354  ...      1      0.6      2   0     2
..   ...  ...  ..       ...   ...  ...    ...      ...    ...  ..   ...
298   57    0   0       140   241  ...      1      0.2      1   0     3
299   45    1   3       110   264  ...      0      1.2      1   0     3
300   68    1   0       144   193  ...      0      3.4      1   2     3
301   57    1   0       130   131  ...      1      1.2      1   1     3
302   57    0   1       130   236  ...      0      0.0      1   1     2

[303 rows x 13 columns]

ExamplePython

print(Y)

Output

Output

0      1
1      1
2      1
3      1
4      1
      ..
298    0
299    0
300    0
301    0
302    0
Name: target, Length: 303, dtype: int64

#Train Test Split

ExamplePython

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, stratify = Y, random_state=3)

ExamplePython

print(X.shape, X_train.shape, X_test.shape)

Output

Output

(303, 13) (242, 13) (61, 13)

Comparing the performance of the models

ExamplePython

# list of models
models = [LogisticRegression(max_iter=1000), SVC(kernel='linear'), KNeighborsClassifier(), RandomForestClassifier()]

ExamplePython

def compare_models_train_test():

  for model in models:

    # training the model
    model.fit(X_train, Y_train)
    
    # evaluating the model
    test_data_prediction = model.predict(X_test)

    accuracy = accuracy_score(Y_test, test_data_prediction)

    print('Accuracy score of the ', model, ' = ', accuracy)

ExamplePython

compare_models_train_test()

Output

Output

Accuracy score of the  LogisticRegression(max_iter=1000)  =  0.7704918032786885
Accuracy score of the  SVC(kernel='linear')  =  0.7704918032786885
Accuracy score of the  KNeighborsClassifier()  =  0.6557377049180327
Accuracy score of the  RandomForestClassifier()  =  0.8032786885245902

#Cross Validation

Logistic Regression

ExamplePython

cv_score_lr = cross_val_score(LogisticRegression(max_iter=1000), X, Y, cv=5)

print(cv_score_lr)

mean_accuracy_lr = sum(cv_score_lr)/len(cv_score_lr)

mean_accuracy_lr = mean_accuracy_lr*100

mean_accuracy_lr = round(mean_accuracy_lr, 2)

print(mean_accuracy_lr)

Output

Output

[0.80327869 0.86885246 0.85245902 0.86666667 0.75      ]
82.83

Support Vector Classifier

ExamplePython

cv_score_svc = cross_val_score(SVC(kernel='linear'), X, Y, cv=5)

print(cv_score_svc)

mean_accuracy_svc = sum(cv_score_svc)/len(cv_score_svc)

mean_accuracy_svc = mean_accuracy_svc*100

mean_accuracy_svc = round(mean_accuracy_svc, 2)

print(mean_accuracy_svc)

Output

Output

[0.81967213 0.8852459  0.80327869 0.86666667 0.76666667]
82.83

Creating a Function to compare the models

ExamplePython

# list of models
models = [LogisticRegression(max_iter=1000), SVC(kernel='linear'), KNeighborsClassifier(), RandomForestClassifier()]

ExamplePython

def compare_models_cross_validation():

  for model in models:

    cv_score = cross_val_score(model, X,Y, cv=5)
    
    mean_accuracy = sum(cv_score)/len(cv_score)

    mean_accuracy = mean_accuracy*100

    mean_accuracy = round(mean_accuracy, 2)

    print('Cross Validation accuracies for ', model, '=  ', cv_score)
    print('Accuracy % of the ', model, mean_accuracy)
    print('----------------------------------------------')

ExamplePython

compare_models_cross_validation()

Output

Output

Cross Validation accuracies for  LogisticRegression(max_iter=1000) =   [0.80327869 0.86885246 0.85245902 0.86666667 0.75      ]
Accuracy % of the  LogisticRegression(max_iter=1000) 82.83
----------------------------------------------
Cross Validation accuracies for  SVC(kernel='linear') =   [0.81967213 0.8852459  0.80327869 0.86666667 0.76666667]
Accuracy % of the  SVC(kernel='linear') 82.83
----------------------------------------------
Cross Validation accuracies for  KNeighborsClassifier() =   [0.60655738 0.6557377  0.57377049 0.73333333 0.65      ]
Accuracy % of the  KNeighborsClassifier() 64.39
----------------------------------------------
Cross Validation accuracies for  RandomForestClassifier() =   [0.81967213 0.8852459  0.7704918  0.83333333 0.76666667]
Accuracy % of the  RandomForestClassifier() 81.51
----------------------------------------------

Keep going: ← Lesson 42 · Lesson 44 → · Track it in the Self Guide

Finished this one?

Part of Self Guide · Stage 16: ML 4 — Evaluating & Choosing Models. Progress is saved in this browser.

Guide

Previous Next

Projects in this lesson

ML & Vision

Machine Learning Playbook & Deep Learning Lab

Interactive machine learning educational playground and hands-on laboratory covering computer vision, regression, and deep neural nets.

Cross Validation Python Implementation

Key principles

Walkthrough

Importing the Dependencies

Importing the models

Data Collection and Processing

Splitting the Features and Target

Comparing the performance of the models

Logistic Regression

Support Vector Classifier

Creating a Function to compare the models

Machine Learning Playbook & Deep Learning Lab

Teaching Networking, DevOps and ML in Sinhala

Zero-Cost Hosting for InFocus SLUG 2025 — and 25 Hosting & Domain Options Compared

Life Vision: The Distributed Life

	age	sex	cp	trestbps	chol	fbs	restecg	…
0	63	1	3	145	233	1	0	…
1	37	1	2	130	250	0	1	…
2	41	0	1	130	204	0	0	…
3	56	1	1	120	236	0	1	…
4	57	0	0	120	354	0	1	…

	age	sex	cp	trestbps	chol	fbs	restecg	…
0	63	1	3	145	233	1	0	…
1	37	1	2	130	250	0	1	…
2	41	0	1	130	204	0	0	…
3	56	1	1	120	236	0	1	…
4	57	0	0	120	354	0	1	…

	age	sex	cp	trestbps	chol	fbs	restecg	…
0	63	1	3	145	233	1	0	…
1	37	1	2	130	250	0	1	…
2	41	0	1	130	204	0	0	…
3	56	1	1	120	236	0	1	…
4	57	0	0	120	354	0	1	…