How to scrape a website HTML using requests

1 min readWebscrapingData
7-Day Challenge

Land Your First Data Science Job

A proven roadmap to prepare for $75K+ entry-level data roles. Perfect for Data Scientist ready to level up their career.

Build portfolios that hiring managers love
Master the Python and SQL essentials to be industry-ready
Practice with real interview questions from tech companies
Access to the $100k/y Data Scientist Cheatsheet

Join thousands of developers who transformed their careers through our challenge. Unsubscribe anytime.

The requests library is probably one of the simplest library to use when you want to execute an HTTP call from Python script.

Asking for raw HTML

Some website will give you raw HTML only, because they are not supposed to give you a JSON back. It is the example with facebook.com

import requests # To make HTTP calls

url = "https://facebook.com"
response = requests.get(url)
getting facebook front-page data

And if you print the response.text it will print out the html content that your call retrieved.

print(response.text)
getting facebook front-page data
<!DOCTYPE html>
<html lang="de" id="facebook" class="no_js">
<head><meta charset="utf-8" /><meta name="referrer"
the first lines of the html response

In order to properly work with your data you will need now to parse the HTML.

You can do that using Beautiful Soup library in Python

7-Day Challenge

Land Your First Data Science Job

A proven roadmap to prepare for $75K+ entry-level data roles. Perfect for Data Scientist ready to level up their career.

Build portfolios that hiring managers love
Master the Python and SQL essentials to be industry-ready
Practice with real interview questions from tech companies
Access to the $100k/y Data Scientist Cheatsheet

Join thousands of developers who transformed their careers through our challenge. Unsubscribe anytime.

Free Newsletter

Master Data Science in Days, Not Months 🚀

Skip the theoretical rabbit holes. Get practical data science skills delivered in bite-sized lessons – Approach used by real data scientist. Not bookworms. 📚

Weekly simple and practical lessons
Access to ready to use code examples
Skip the math, focus on results
Learn while drinking your coffee

By subscribing, you agree to receive our newsletter. You can unsubscribe at any time.