Fastest Python Web Scraper - Exploring Sessions, Multiprocessing, Multithreading, and Scrapy

Описание: In this video, we will make a fast web scraper. We will begin with BeautifulSoup.
🚀 The first script takes 128 seconds and after optimization, takes as little as 2.5 seconds.
Finally, we will create a scrapy spider without optimization and see what kind of results we get.
We will use BeautifulSoup, Requests, Sessions, Multithreading, Multiprocessing, and Scrapy.
👩‍💻 Source Code: https://github.com/eupendra/fast_scra...
You can jump to the sections you like:
00:31 Scraper Objective
00:44 Creating Scraper with Requests+BS4
9:20 First Run
10:07 Sessions
13:58 Multiprocessing
17:22 Multithreading
22:36 Scrapy Without Optimization

Related videos
-------

👩‍💻 Watch the Playlist to Learn the Basics of Scrapy:    • Scrapy for Beginners  

👨‍💻Join all courses on my site: https://coderecode.com/membership

----------------------------------------------
What is Web Scraping?
In a nutshell: Web Scraping = Getting Data from Websites with Code

What is Scrapy?
Scrapy is a Python library to makes web scraping very powerful, fast, and efficient.

There are other libraries too like BeautifulSoup, for web scraping. However, when it comes to true power and flexibility, Scrapy is the most powerful.
Why Learn Scrapy?
Most powerful library for scraping
Easy to master
Cross-platform: doesn't matter which OS you are using
Cloud-ready: Can be run on the cloud with a free account

Most Important: You would be able to earn by taking up some of the web scraping gigs as a freelancer

#scrapy #fast #beautifulsoup #multiprocessing #multithreading

~-~~-~~~-~~-~
Please watch: "Making Scrapy Playwright fast and reliable"
   • How to make Scrapy Playwright fast and rel...  
~-~~-~~~-~~-~