Importing Datasets through Kaggle API

සැබෑ projects පටන් ගන්නෙ laptop එකේ කලින් ඉඳන් තියෙන file එකකින් නෙවෙයි — Professionals අතින් download කරනවා වෙනුවට, fresh, real-world datasets programmatic ව ගන්නෙ මෙහෙමයි.

Real projects don't start with a file already sitting on your laptop — this is how professionals pull fresh, real-world datasets programmatically instead of manual downloads.

5 May 20263 min readSelf Guide · Stage 13#Python & Data Basics#Data Pre-processing#Machine learning

Lesson 13 of 64 · Part 1 · Python & Data Basics · Data Pre-processing · bilingual notes (English + සිංහල)

Why this matters. Real projects don't start with a file already sitting on your laptop — this is how professionals pull fresh, real-world datasets programmatically instead of manual downloads.

සිංහලෙන්: සැබෑ projects පටන් ගන්නෙ laptop එකේ කලින් ඉඳන් තියෙන file එකකින් නෙවෙයි — Professionals අතින් download කරනවා වෙනුවට, fresh, real-world datasets programmatic ව ගන්නෙ මෙහෙමයි.

What you'll use this for. Every dataset used in the bigger projects later in this course (house prices, loans, heart disease) can be pulled this same way.

සිංහලෙන්: පස්සෙ එන loku projects වල (house prices, loans, heart disease) පාවිච්චි කරන dataset ඕනම එකක් මේ විදිහටම ගන්න පුළුවන්.

Key principles

An API key is a password for programs, not people. Your kaggle.json file lets your notebook authenticate automatically — treat it like a password, never share it publicly.
API key එකක් කියන්නෙ programs සඳහා password එකක්, මිනිසුන් සඳහා නෙවෙයි

Walkthrough

API - Application Programming Interface

ExamplePython

# installing the Kaggle library
!pip install kaggle

Output

Output

Requirement already satisfied: kaggle in /usr/local/lib/python3.7/dist-packages (1.5.10)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from kaggle) (2020.12.5)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.23.0)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.24.3)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.15.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.41.1)
Requirement already satisfied: python-slugify in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.0.1)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.8.1)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (3.0.4)
Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/dist-packages (from python-slugify->kaggle) (1.3)

Upload your Kaggle.json file

ExamplePython

# configuring the path of Kaggle.json file
!mkdir -p ~/.kaggle
!cp kaggle.json ~/.kaggle/
!chmod 600 ~/.kaggle/kaggle.json

Importing the Earthquake Dataset

ExamplePython

# API to fetch the dataset from Kaggle
!kaggle competitions download -c LANL-Earthquake-Prediction

Output

Output

Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)
Downloading seg_010eab.csv to /content
  0% 0.00/316k [00:00<?, ?B/s]
100% 316k/316k [00:00<00:00, 43.6MB/s]
Downloading seg_004ee5.csv to /content
  0% 0.00/324k [00:00<?, ?B/s]
100% 324k/324k [00:00<00:00, 5.17MB/s]
Downloading seg_007a37.csv to /content
  0% 0.00/325k [00:00<?, ?B/s]
100% 325k/325k [00:00<00:00, 105MB/s]
Downloading seg_004f1f.csv to /content
  0% 0.00/324k [00:00<?, ?B/s]
100% 324k/324k [00:00<00:00, 105MB/s]
Downloading seg_0042cc.csv to /content
  0% 0.00/319k [00:00<?, ?B/s]
100% 319k/319k [00:00<00:00, 102MB/s]
Downloading seg_004cd2.csv to /content
  0% 0.00/315k [00:00<?, ?B/s]
… (49 more lines)

ExamplePython

# extracting the compessed Dataset
from zipfile import ZipFile
dataset = '/content/train.csv.zip'

with ZipFile(dataset,'r') as zip:
  zip.extractall()
  print('The dataset is extracted')