The 9 Best Ways to Scrape Any Website with Claude Code

Описание: Join the AI MBA to learn personally from me how to build with Claude Code ⤵️
https://www.skool.com/the-ai-mba

Prefer reading tutorials to watching them? Check out my written guides ⤵️
https://ai-mba.io/tutorials

Summary ⤵️
I walk through 9 different ways to scrape any website using Claude Code — from simple direct scrapes and endpoint reverse engineering to social media APIs, Apify actors, Firecrawl, yt-dlp, Reddit JSON endpoints, and authenticated browser automation. Each method gets a live demo with real targets so you can see exactly when and how to use it.

Links 🚀
ScrapeCreators (social media APIs): https://scrapecreators.com/
Apify (actor-based scraping platform): https://apify.com
Firecrawl (web → markdown for AI): https://www.firecrawl.dev/
yt-dlp (YouTube metadata + subtitles): https://github.com/yt-dlp/yt-dlp
Agent Browser by Vercel (browser automation for AI agents): https://github.com/vercel-labs/agent-...

Follow me on other platforms 😈
🕊️ Twitter/X: https://x.com/aniketapanjwani
💼 LinkedIn:   / aniket-a-panjwani  
🤙 Work with me: https://contentquant.io

Why watch?

If this is your first view—hi, I'm Aniket! TLDR: I've worked for six years as an MLOps engineer at Zelle building financial AI models, automations and data pipelines saving consumers from tens of millions of dollars of fraud.

Now, I use my professional and academic expertise (PhD Economics from Northwestern University) to help businesses automate tedious but critical processes to drive revenue and innovation.

Hopefully I can help you improve your business, and in doing so, the rest of your life 🙏

Like, subscribe, and leave me a comment if you have a specific request! Thanks.

0:00 The 9 best ways to scrape any website with Claude Code
0:16 Who I am and what this video covers
0:37 Way 1: Direct scrape to CSV/SQLite
1:30 Demo: Scraping Hacker News front page
3:24 Way 2: Endpoint-first scraping
4:41 Demo: Reverse engineering UK property endpoints
9:10 Way 3: Social media APIs (ScrapeCreators)
10:25 Demo: Scraping Kevin Durant's Instagram
13:14 Way 4: Apify actor workflow
15:09 Demo: Google Maps restaurants in Buenos Aires
17:28 Way 5: Firecrawl → markdown → structured data
19:00 Demo: Economics job market candidates
23:34 Way 6: DIY HTML → markdown extraction
24:00 Demo: Open-source markdownify approach
26:10 Way 7: YouTube scraping via yt-dlp
26:58 Demo: Alex Hormozi channel metadata + subtitles
30:37 Way 8: Reddit JSON endpoint
31:29 Demo: Scraping r/ClaudeCode subreddit
32:37 Way 9: Agent Browser for authenticated sites
33:45 Demo: Login + scrape Skool community posts
36:25 Recap and when to use each method
37:11 Outro