Precision, Recall, F1 Score Python implementation
Accuracy සහ confusion matrix එකෙන් මොකද වුනේ කියලා කියනවා; precision, recall, F1 වලින් ඔයාට ඇත්තටම worry කරන failure mode එකට, model එක "කොච්චර හොඳද" කියලා compare කරන්න පුළුවන් number එකක් ලැබෙනවා.
Accuracy and the confusion matrix tell you what happened; precision, recall, and F1 give you a single, comparable number for "how good" a model is at the failure mode you actually care about.
Lesson 47 of 64 · Part 4 · Evaluating & Choosing Models · Model Evaluation · bilingual notes (English + සිංහල)
Why this matters. Accuracy and the confusion matrix tell you what happened; precision, recall, and F1 give you a single, comparable number for "how good" a model is at the failure mode you actually care about.
සිංහලෙන්: Accuracy සහ confusion matrix එකෙන් මොකද වුනේ කියලා කියනවා; precision, recall, F1 වලින් ඔයාට ඇත්තටම worry කරන failure mode එකට, model එක "කොච්චර හොඳද" කියලා compare කරන්න පුළුවන් number එකක් ලැබෙනවා.
What you'll use this for. In the medical projects (heart disease, diabetes), recall is usually prioritized over precision — this lesson is what lets you make and justify that exact choice.
සිංහලෙන්: Medical projects වල (heart disease, diabetes), සාමාන්යයෙන් precision එකට වඩා recall එකට priority දෙනවා — ඒ choice එක ගන්නත්, justify කරන්නත් ඉඩ දෙන්නෙ මේ පාඩම.
Key principles
- Precision asks "of what I flagged, how much was right?"; recall asks "of what was actually true, how much did I catch?". A model can have perfect precision by flagging only one, very obvious case — and terrible recall because it missed everything else. F1 balances the two into one number.
Precision අහන්නෙ "මම flag කරපු ඒවායින් කොච්චරද හරි?"; recall අහන්නෙ "ඇත්තටම තිබ්බ ඒවායින් කොච්චරක් මම අල්ලගත්තද?"
Walkthrough
Importing the Dependencies
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegressionData Collection & Processing
# loading the csv data to a Pandas DataFrame
heart_data = pd.read_csv('heart.csv')# print first 5 rows of the dataset
heart_data.head()| age | sex | cp | trestbps | chol | fbs | restecg | … | |
|---|---|---|---|---|---|---|---|---|
| 0 | 63 | 1 | 3 | 145 | 233 | 1 | 0 | … |
| 1 | 37 | 1 | 2 | 130 | 250 | 0 | 1 | … |
| 2 | 41 | 0 | 1 | 130 | 204 | 0 | 0 | … |
| 3 | 56 | 1 | 1 | 120 | 236 | 0 | 1 | … |
| 4 | 57 | 0 | 0 | 120 | 354 | 0 | 1 | … |
# print last 5 rows of the dataset
heart_data.tail()| age | sex | cp | trestbps | chol | fbs | restecg | … | |
|---|---|---|---|---|---|---|---|---|
| 298 | 57 | 0 | 0 | 140 | 241 | 0 | 1 | … |
| 299 | 45 | 1 | 3 | 110 | 264 | 0 | 1 | … |
| 300 | 68 | 1 | 0 | 144 | 193 | 1 | 1 | … |
| 301 | 57 | 1 | 0 | 130 | 131 | 0 | 1 | … |
| 302 | 57 | 0 | 1 | 130 | 236 | 0 | 0 | … |
# number of rows and columns in the dataset
heart_data.shapeOutput
(303, 14)# getting some info about the data
heart_data.info()Output
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 303 entries, 0 to 302
Data columns (total 14 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 age 303 non-null int64
1 sex 303 non-null int64
2 cp 303 non-null int64
3 trestbps 303 non-null int64
4 chol 303 non-null int64
5 fbs 303 non-null int64
6 restecg 303 non-null int64
7 thalach 303 non-null int64
8 exang 303 non-null int64
9 oldpeak 303 non-null float64
10 slope 303 non-null int64
11 ca 303 non-null int64
12 thal 303 non-null int64
… (3 more lines)# checking for missing values
heart_data.isnull().sum()Output
age 0
sex 0
cp 0
trestbps 0
chol 0
fbs 0
restecg 0
thalach 0
exang 0
oldpeak 0
slope 0
ca 0
thal 0
target 0
dtype: int64# statistical measures about the data
heart_data.describe()| age | sex | cp | trestbps | chol | fbs | restecg | … | |
|---|---|---|---|---|---|---|---|---|
| count | 303.000000 | 303.000000 | 303.000000 | 303.000000 | 303.000000 | 303.000000 | 303.000000 | … |
| mean | 54.366337 | 0.683168 | 0.966997 | 131.623762 | 246.264026 | 0.148515 | 0.528053 | … |
| std | 9.082101 | 0.466011 | 1.032052 | 17.538143 | 51.830751 | 0.356198 | 0.525860 | … |
| min | 29.000000 | 0.000000 | 0.000000 | 94.000000 | 126.000000 | 0.000000 | 0.000000 | … |
| 25% | 47.500000 | 0.000000 | 0.000000 | 120.000000 | 211.000000 | 0.000000 | 0.000000 | … |
| 50% | 55.000000 | 1.000000 | 1.000000 | 130.000000 | 240.000000 | 0.000000 | 1.000000 | … |
… 2 more rows
# checking the distribution of Target Variable
heart_data['target'].value_counts()Output
1 165
0 138
Name: target, dtype: int641 --> Defective Heart
0 --> Healthy Heart
Splitting the Features and Target
X = heart_data.drop(columns='target', axis=1)
Y = heart_data['target']print(X)Output
age sex cp trestbps chol fbs restecg thalach exang oldpeak \
0 63 1 3 145 233 1 0 150 0 2.3
1 37 1 2 130 250 0 1 187 0 3.5
2 41 0 1 130 204 0 0 172 0 1.4
3 56 1 1 120 236 0 1 178 0 0.8
4 57 0 0 120 354 0 1 163 1 0.6
.. ... ... .. ... ... ... ... ... ... ...
298 57 0 0 140 241 0 1 123 1 0.2
299 45 1 3 110 264 0 1 132 0 1.2
300 68 1 0 144 193 1 1 141 0 3.4
301 57 1 0 130 131 0 1 115 1 1.2
302 57 0 1 130 236 0 0 174 0 0.0
slope ca thal
0 0 0 1
1 0 0 2
2 2 0 2
3 2 0 2
… (9 more lines)print(Y)Output
0 1
1 1
2 1
3 1
4 1
..
298 0
299 0
300 0
301 0
302 0
Name: target, Length: 303, dtype: int64Splitting the Data into Training data & Test Data
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, stratify=Y, random_state=2)print(X.shape, X_train.shape, X_test.shape)Output
(303, 13) (242, 13) (61, 13)Model Training
model = LogisticRegression(max_iter=1000)# training the LogisticRegression model with Training data
model.fit(X_train, Y_train)Output
LogisticRegression(max_iter=1000)Model Evaluation
Accuracy Score
from sklearn.metrics import accuracy_score# accuracy on training data
X_train_prediction = model.predict(X_train)
training_data_accuracy = accuracy_score(Y_train, X_train_prediction)
print(training_data_accuracy)Output
0.8553719008264463print('Accuracy on Training data : ', round(training_data_accuracy*100, 2), '%')Output
Accuracy on Training data : 85.54 %# accuracy on test data
X_test_prediction = model.predict(X_test)
test_data_accuracy = accuracy_score(Y_test, X_test_prediction)
print(test_data_accuracy)Output
0.8032786885245902print('Accuracy on Test data : ', round(test_data_accuracy*100, 2), '%')Output
Accuracy on Test data : 80.33 %Precision
Precision is the ratio of number of True Positive to the total number of Predicted Positive. It measures, out of the total predicted positive, how many are actually positive.
from sklearn.metrics import precision_score# precision for training data predictions
precision_train = precision_score(Y_train, X_train_prediction)
print('Training data Precision =', precision_train)Output
Training data Precision = 0.8299319727891157# precision for test data predictions
precision_test = precision_score(Y_test, X_test_prediction)
print('Test data Precision =', precision_test)Output
Test data Precision = 0.8181818181818182Recall
Recall is the ratio of number of True Positive to the total number of Actual Positive. It measures, out of the total actual positive, how many are predicted as True Positive.
from sklearn.metrics import recall_score# recall for training data predictions
recall_train = recall_score(Y_train, X_train_prediction)
print('Training data Recall =', recall_train)Output
Training data Recall = 0.9242424242424242# recall for test data predictions
recall_test = recall_score(Y_test, X_test_prediction)
print('Test data Recall =', recall_test)Output
Test data Recall = 0.8181818181818182F1 Score
F1 Score is an important evaluation metric for binary classification that combines Precision & Recall. F1 Score is the harmonic mean of Precision & Recall.
from sklearn.metrics import f1_score# F1 score for training data predictions
f1_score_train = f1_score(Y_train, X_train_prediction)
print('Training data F1 Score =', f1_score_train)Output
Training data F1 Score = 0.8745519713261649# F1 Score for test data predictions
f1_score_test = recall_score(Y_test, X_test_prediction)
print('Test data F1 Score =', f1_score_test)Output
Test data F1 Score = 0.8181818181818182Precision, Recall, & F1 Score - function
def precision_recall_f1_score(true_labels, pred_labels):
precision_value = precision_score(true_labels, pred_labels)
recall_value = recall_score(true_labels, pred_labels)
f1_score_value = f1_score(true_labels, pred_labels)
print('Precision =',precision_value)
print('Recall =',recall_value)
print('F1 Score =',f1_score_value)# classification metrics for training data
precision_recall_f1_score(Y_train, X_train_prediction)Output
Precision = 0.8299319727891157
Recall = 0.9242424242424242
F1 Score = 0.8745519713261649# classification metrics for test data
precision_recall_f1_score(Y_test, X_test_prediction)Output
Precision = 0.8181818181818182
Recall = 0.8181818181818182
F1 Score = 0.8181818181818182Keep going: ← Lesson 46 · Lesson 48 → · Track it in the Self Guide
Finished this one?
Part of Self Guide · Stage 16: ML 4 — Evaluating & Choosing Models. Progress is saved in this browser.



