Project 18 Movie Recommendation System using Machine Learning with Python

මුළුමනින්ම වෙනස් ML problem වර්ගයක් තමයි මේක — label එකක් හෝ number එකක් predict කරනවා වෙනුවට, items අතර similarity measure කරලා, කෙනෙකුට ලඟදිට ආස වෙන්න පුළුවන් දේ suggest කරනවා.

This is a completely different kind of ML problem — instead of predicting a label or a number, you're measuring similarity between items to suggest what someone might like next.

18 May 20265 min readSelf Guide · Stage 16#Evaluating & Choosing Models#Hands-on Projects#Machine learning

Lesson 51 of 64 · Part 4 · Evaluating & Choosing Models · Hands-on Projects · bilingual notes (English + සිංහල)

Why this matters. This is a completely different kind of ML problem — instead of predicting a label or a number, you're measuring similarity between items to suggest what someone might like next.

සිංහලෙන්: මුළුමනින්ම වෙනස් ML problem වර්ගයක් තමයි මේක — label එකක් හෝ number එකක් predict කරනවා වෙනුවට, items අතර similarity measure කරලා, කෙනෙකුට ලඟදිට ආස වෙන්න පුළුවන් දේ suggest කරනවා.

What you'll use this for. This exact idea — content-based similarity — is the starting point behind recommendation engines on every major streaming and shopping platform.

සිංහලෙන්: මේ එකම අදහස — content-based similarity — ලොකු streaming සහ shopping platform ඕනම එකක recommendation engines වල පටන් ගන්නා ලක්ෂ්‍යයයි.

Key principles

Recommending is really just "find the most similar items" at scale. Movies get converted into feature vectors (genre, cast, keywords), then cosine similarity finds the closest matches — the same distance idea from Part 2's vector lesson, applied to a new problem.
Recommend කරන එක කියන්නෙ ඇත්තටම "scale එකකින් වඩාත්ම සමාන items හොයන්න" කියන එක

Walkthrough

Importing the dependencies

ExamplePython

import numpy as np
import pandas as pd
import difflib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

Data Collection and Pre-Processing

ExamplePython

# loading the data from the csv file to apandas dataframe
movies_data = pd.read_csv('/content/movies.csv')

ExamplePython

# printing the first 5 rows of the dataframe
movies_data.head()

	index	budget	genres	homepage	id	keywords	original_language	…
0	0	237000000	Action Adventure Fantasy Science Fiction	http://www.avatarmovie.com/	19995	culture clash future space war space colony so...	en	…
1	1	300000000	Adventure Fantasy Action	http://disney.go.com/disneypictures/pirates/	285	ocean drug abuse exotic island east india trad...	en	…
2	2	245000000	Action Adventure Crime	http://www.sonypictures.com/movies/spectre/	206647	spy based on novel secret agent sequel mi6	en	…
3	3	250000000	Action Crime Drama Thriller	http://www.thedarkknightrises.com/	49026	dc comics crime fighter terrorist secret ident...	en	…
4	4	260000000	Action Adventure Science Fiction	http://movies.disney.com/john-carter	49529	based on novel mars medallion space travel pri...	en	…

ExamplePython

# number of rows and columns in the data frame

movies_data.shape

Output

Output

(4803, 24)

ExamplePython

# selecting the relevant features for recommendation

selected_features = ['genres','keywords','tagline','cast','director']
print(selected_features)

Output

Output

['genres', 'keywords', 'tagline', 'cast', 'director']

ExamplePython

# replacing the null valuess with null string

for feature in selected_features:
  movies_data[feature] = movies_data[feature].fillna('')

ExamplePython

# combining all the 5 selected features

combined_features = movies_data['genres']+' '+movies_data['keywords']+' '+movies_data['tagline']+' '+movies_data['cast']+' '+movies_data['director']

ExamplePython

print(combined_features)

Output

Output

0       Action Adventure Fantasy Science Fiction cultu...
1       Adventure Fantasy Action ocean drug abuse exot...
2       Action Adventure Crime spy based on novel secr...
3       Action Crime Drama Thriller dc comics crime fi...
4       Action Adventure Science Fiction based on nove...
                              ...                        
4798    Action Crime Thriller united states\u2013mexic...
4799    Comedy Romance  A newlywed couple's honeymoon ...
4800    Comedy Drama Romance TV Movie date love at fir...
4801      A New Yorker in Shanghai Daniel Henney Eliza...
4802    Documentary obsession camcorder crush dream gi...
Length: 4803, dtype: object

ExamplePython

# converting the text data to feature vectors

vectorizer = TfidfVectorizer()

ExamplePython

feature_vectors = vectorizer.fit_transform(combined_features)

ExamplePython

print(feature_vectors)

Output

Output

  (0, 2432)	0.17272411194153
  (0, 7755)	0.1128035714854756
  (0, 13024)	0.1942362060108871
  (0, 10229)	0.16058685400095302
  (0, 8756)	0.22709015857011816
  (0, 14608)	0.15150672398763912
  (0, 16668)	0.19843263965100372
  (0, 14064)	0.20596090415084142
  (0, 13319)	0.2177470539412484
  (0, 17290)	0.20197912553916567
  (0, 17007)	0.23643326319898797
  (0, 13349)	0.15021264094167086
  (0, 11503)	0.27211310056983656
  (0, 11192)	0.09049319826481456
  (0, 16998)	0.1282126322850579
  (0, 15261)	0.07095833561276566
  (0, 4945)	0.24025852494110758
  (0, 14271)	0.21392179219912877
… (33 more lines)

Cosine Similarity

ExamplePython

# getting the similarity scores using cosine similarity

similarity = cosine_similarity(feature_vectors)

ExamplePython

print(similarity)

Output

Output

[[1.         0.07219487 0.037733   ... 0.         0.         0.        ]
 [0.07219487 1.         0.03281499 ... 0.03575545 0.         0.        ]
 [0.037733   0.03281499 1.         ... 0.         0.05389661 0.        ]
 ...
 [0.         0.03575545 0.         ... 1.         0.         0.02651502]
 [0.         0.         0.05389661 ... 0.         1.         0.        ]
 [0.         0.         0.         ... 0.02651502 0.         1.        ]]

ExamplePython

print(similarity.shape)

Output

Output

(4803, 4803)

Getting the movie name from the user

ExamplePython

# getting the movie name from the user

movie_name = input(' Enter your favourite movie name : ')

Output

Output

 Enter your favourite movie name : iron man

ExamplePython

# creating a list with all the movie names given in the dataset

list_of_all_titles = movies_data['title'].tolist()
print(list_of_all_titles)

Output

Output

['Avatar', "Pirates of the Caribbean: At World's End", 'Spectre', 'The Dark Knight Rises', 'John Carter', 'Spider-Man 3', 'Tangled', 'Avengers: Age of Ultron',

ExamplePython

# finding the close match for the movie name given by the user

find_close_match = difflib.get_close_matches(movie_name, list_of_all_titles)
print(find_close_match)

Output

Output

['Iron Man', 'Iron Man 3', 'Iron Man 2']

ExamplePython

close_match = find_close_match[0]
print(close_match)

Output

Output

Iron Man

ExamplePython

# finding the index of the movie with title

index_of_the_movie = movies_data[movies_data.title == close_match]['index'].values[0]
print(index_of_the_movie)

Output

Output

ExamplePython

# getting a list of similar movies

similarity_score = list(enumerate(similarity[index_of_the_movie]))
print(similarity_score)

Output

Output

[(0, 0.033570748780675445), (1, 0.0546448279236134), (2, 0.013735500604224323), (3, 0.006468756104392058), (4, 0.03268943310073386), (5, 0.013907256685755473),

ExamplePython

len(similarity_score)

Output

Output

ExamplePython

# sorting the movies based on their similarity score

sorted_similar_movies = sorted(similarity_score, key = lambda x:x[1], reverse = True) 
print(sorted_similar_movies)

Output

Output

[(68, 1.0000000000000002), (79, 0.40890433998005965), (31, 0.31467052449477506), (7, 0.23944423963486405), (16, 0.22704403782296803), (26, 0.21566241096831154),

ExamplePython

# print the name of similar movies based on the index

print('Movies suggested for you : \n')

i = 1

for movie in sorted_similar_movies:
  index = movie[0]
  title_from_index = movies_data[movies_data.index==index]['title'].values[0]
  if (i<30):
    print(i, '.',title_from_index)
    i+=1

Output

Output

Movies suggested for you : 

1 . Iron Man
2 . Iron Man 2
3 . Iron Man 3
4 . Avengers: Age of Ultron
5 . The Avengers
6 . Captain America: Civil War
7 . Captain America: The Winter Soldier
8 . Ant-Man
9 . X-Men
10 . Made
11 . X-Men: Apocalypse
12 . X2
13 . The Incredible Hulk
14 . The Helix... Loaded
15 . X-Men: First Class
16 . X-Men: Days of Future Past
… (13 more lines)

Movie Recommendation Sytem

ExamplePython

movie_name = input(' Enter your favourite movie name : ')

list_of_all_titles = movies_data['title'].tolist()

find_close_match = difflib.get_close_matches(movie_name, list_of_all_titles)

close_match = find_close_match[0]

index_of_the_movie = movies_data[movies_data.title == close_match]['index'].values[0]

similarity_score = list(enumerate(similarity[index_of_the_movie]))

sorted_similar_movies = sorted(similarity_score, key = lambda x:x[1], reverse = True) 

print('Movies suggested for you : \n')

i = 1

for movie in sorted_similar_movies:
  index = movie[0]
  title_from_index = movies_data[movies_data.index==index]['title'].values[0]
  if (i<30):
    print(i, '.',title_from_index)
    i+=1

Output

Output

 Enter your favourite movie name : bat man
Movies suggested for you : 

1 . Batman
2 . Batman Returns
3 . Batman & Robin
4 . The Dark Knight Rises
5 . Batman Begins
6 . The Dark Knight
7 . A History of Violence
8 . Superman
9 . Beetlejuice
10 . Bedazzled
11 . Mars Attacks!
12 . The Sentinel
13 . Planet of the Apes
14 . Man of Steel
15 . Suicide Squad
… (14 more lines)