YouTube Scraper: Videos, Channels & Metadata
Tested youtube data scraper routes on live videos: one command returns title, author, views and duration with no API key, plus the bulk CSV loop.
On this page
Every YouTube video serves its own metadata to any client that asks: title, channel, view count, duration, description. A youtube data scraper only has to ask, and the official Data API is one way to ask, metered at 10,000 free quota units a day where a metadata read costs 1 and search runs in its own bucket of 100 calls. I tested three routes against live videos today; one needs no key, no Python, and about 2 seconds per video.
Everything below ran October 16, 2026, from a home connection, against three stable videos: “Me at the zoo”, the 19-second clip that started YouTube, plus the Rick Astley and Gangnam Style music videos. YouTube changes its plumbing constantly. Re-test before you build on any of it.
What a YouTube data scraper can and can’t collect
Start with the edges, because the word “scraper” promises more than any of these routes deliver.
What works: one video per URL, returning title, author, full description, view count, duration in seconds, and the canonical watch URL. Feed it watch?v=, youtu.be, Shorts, embed, or live links; they all resolve to the same record.
What does not work, tested today: channel pages (youtube.com/@handle), playlists, and search results are refused outright. Comments, subtitles, and transcripts are not collected at all. No route in this article downloads media either; this is metadata, not video files. If your job is comments or transcripts, that is Route 2 territory and I say so again down there.
On legality, two layers that people conflate. robots.txt, which I pulled this morning, does not disallow /watch pages for generic crawlers (it bans /watch_ajax, /results, and a few other paths). YouTube’s Terms of Service are a stricter contract than robots.txt and restrict automated access without permission, so low volume, public metadata, and no republishing of personal data is the posture I recommend. Not legal advice; Is Web Scraping Legal? is the plain-English version.
Three routes to YouTube data
Most youtube webscraping questions are really one of three jobs, and each job has a right tool.
| Route | Gets you | Needs | My test today |
|---|---|---|---|
Magpie youtube vertical |
Typed record per video: title, author, description, views, duration | Nothing but one binary | 3/3 videos parsed, 2.1s each |
| YouTube Data API v3 | Everything: comments, captions, channels, playlists, search | Google Cloud project + API key | Quota model verified against Google’s doc dated 2026-10-15; no key on this machine, not demoed live |
| yt-dlp | Metadata plus subtitles, formats, whole-channel walks | Python + one package | v2026.08.19, one video, 2.4s |
Search web scraper youtube and the front page is hosted-API listicles quoting per-record prices. What none of them show is the output. I will.
Route 1: video metadata with one command, no key
Magpie is a local CLI whose youtube vertical is one of 27 zero-LLM extractors: no API key, no per-record cost, one fetch per video. The command and the real record it returned about ten minutes ago:
magpie vertical "https://www.youtube.com/watch?v=jNQXAC9IVRw" --name youtube{
"vertical": "youtube",
"record": {
"author": "jawed",
"description": "Microplastics are accumulating in human brains…",
"duration_s": 19,
"title": "Me at the zoo",
"url": "https://www.youtube.com/watch?v=jNQXAC9IVRw",
"views": 444429058
}
}(Abridged; the full description runs ten lines.) Six fields, typed, done in 2.1 seconds. The extractor parses the player data embedded in the watch page, and if you drop --name youtube it still picks the right extractor from the URL, so magpie vertical <url> is enough.
This makes it a youtube url scraper in a literal sense too: whatever shape of link you feed it, the fetch goes to one canonical address. I threw all five accepted shapes at it:
| URL shape | Result |
|---|---|
youtube.com/watch?v={id} |
Parsed |
youtu.be/{id} |
Parsed, same record |
youtube.com/shorts/{id} |
Parsed, same record |
youtube.com/embed/{id} |
Parsed, record URL rewritten to canonical watch?v= |
youtube.com/live/{id} |
Accepted by the extractor (same 11-character ID rule) |
The IDs must be exactly 11 characters; that is the whole matching rule, read from the extractor’s source. Anything else bounces. Channel and playlist URLs fail with vertical extractor does not handle URL, which is at least an honest error. A made-up ID fails twice: no player data, then the oEmbed fallback 404s, and you get a typed error naming both.
Bulk: three videos to CSV in 15 seconds
Magpie’s --format flag does not speak CSV yet, so the loop pipes through jq. This exact command, timed:
printf '%s\n' \
"https://www.youtube.com/watch?v=jNQXAC9IVRw" \
"https://youtu.be/dQw4w9WgXcQ" \
"https://www.youtube.com/watch?v=9bZkp7q19f0" > urls.txt
while read -r u; do
magpie vertical "$u" --name youtube
sleep 2
done < urls.txt \
| jq -s -r '[.[] | .record] | .[] |
[.title, .author, (.views|tostring), (.duration_s|tostring), .url] | @csv'Real output from my run:
"Me at the zoo","jawed","444429600","19","https://www.youtube.com/watch?v=jNQXAC9IVRw"
"Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster)","Rick Astley","1827141898","213","https://www.youtube.com/watch?v=dQw4w9WgXcQ"
"PSY - GANGNAM STYLE(강남스타일) M/V","officialpsy","6093642283","252","https://www.youtube.com/watch?v=9bZkp7q19f0"Three videos, 14.6 seconds wall time including the two-second politeness sleeps. Notice the view counts are not the ones in the JSON above: “Me at the zoo” gained 542 views between my two runs, twenty minutes apart. The numbers move, so timestamp your exports. For everything CSV-shaped after that point (headers, Excel, dedupe), Scrape a Website to CSV or JSON covers the general case.
The failure mode that looks like data
One behavior worth more than the happy path. If the watch page serves no player data, which the extractor’s source says happens for videos that gate playback (age-restricted, members-only), the extractor does not error. It silently falls back to YouTube’s oEmbed endpoint and returns a record with title and author, but views: 0, duration_s: 0, and an empty description.
That record parses. It looks like data. It is a stub. The detection rule: a record with a title and zero views is a fallback record, so refetch once, and if it happens again, that video needs the Data API route. I could not reproduce a gated video on demand for this test, so that mechanism is read from the extractor’s source, not demonstrated live. The both-sources-fail path I did hit: a nonexistent ID returns no player data and oEmbed failed … HTTP 404, which is how a dead link should die.
Route 2: the official API when metadata isn’t enough
The YouTube Data API v3 is free and stays free at this scale, but it wants a Google Cloud project, the API enabled, and a key. What you get for the setup is everything the vertical refuses: comments, caption tracks, channel listings, playlists, search.
The quota model, verified against Google’s own calculator page updated 2026-10-15: 10,000 units per day for all endpoints combined, plus two separate buckets of 100 calls per day each for search.list and videos.insert. Metadata reads are cheap: videos.list costs 1 unit and takes up to 50 video IDs per call, so one request covers 50 videos where the vertical needs 50 fetches. commentThreads.list is also 1 unit; captions.list is 50. Quotas reset at midnight Pacific.
curl "https://www.googleapis.com/youtube/v3/videos?part=snippet,statistics&id=jNQXAC9IVRw&key=YOUR_KEY"I have no key on this machine and won’t invent output, so the shape above is documented API, not a live run. The honest trade: setup and quota juggling versus zero setup and per-video fetching. Under roughly a thousand videos, I take the vertical. Past that, or the moment comments enter the picture, the API wins and it isn’t close.
Route 3: yt-dlp, the heavy machinery
yt-dlp (Python, v2026.08.19 on my box) is the community’s answer to “YouTube changed something again”. It releases near-weekly because it has to. For metadata it does everything the vertical does and most of what it refuses:
yt-dlp --skip-download --no-warnings \
--print "%(title)s | %(view_count)s | %(duration_string)s | %(channel)s" \
"https://www.youtube.com/watch?v=jNQXAC9IVRw"Real output: Me at the zoo | 444429058 | 0:19 | jawed, in 2.4 seconds. Beyond that one line: --write-subs and --write-auto-subs pull caption files, --flat-playlist walks a channel’s video list, --write-comments scrapes comment threads at the cost of a long crawl. The cost is a Python environment and a tool that breaks when YouTube ships a change, then ships a fix within days. If a yt-dlp command fails, update before you debug; a stale version is the most common cause.
Keep it polite, keep it unblocked
One request per video, sleep 2 between them, no parallelism against youtube.com. I verified this morning that watch pages are not in robots.txt’s disallow list, but politeness is about more than robots: YouTube rate-limits aggressively by IP reputation, and a scraper that hammers it trains YouTube’s defenses for everyone. The delay patterns that keep a youtube web scraper welcome long-term are the same ones in Scrape Without Getting Blocked.
Same engine, two skins: the CLI above is AGPL-3.0-or-later with a paid commercial option and no run counter. The Desktop app wraps it with 5 free runs per install, then $49/month, where a failed run costs nothing.
FAQ
Is it legal to scrape YouTube?
Two layers: robots.txt permits watch-page crawls, but the Terms of Service restrict automated access without permission as a contract matter. Public metadata at low volume is the common practice; republishing personal data is where people get burned. Not legal advice; see Is Web Scraping Legal?.
Is the YouTube Data API free?
Yes. The default allocation is 10,000 units per day, and a 50-video metadata batch costs 1 unit, so tens of thousands of videos per day cost nothing. Search has its own 100-calls-per-day bucket. You need a Google Cloud project and an API key.
Can I scrape YouTube comments?
Not with Magpie’s vertical; comments are not built (neither are transcripts). Use commentThreads.list on the Data API at 1 unit per call, or yt-dlp’s --write-comments, which works but crawls slowly.
How do I export YouTube data to CSV?
The jq loop in the bulk section prints one CSV row per video. For the API route, pipe the JSON response through the same jq filter. The general patterns live in Scrape a Website to CSV or JSON.
Why does my scrape show 0 views?
That is the oEmbed fallback: the watch page served no player data, so you got title and author only. Refetch once; if it persists, the video gates playback and the Data API route is the fix.
Pick by the job
A pile of video links to a spreadsheet: the vertical loop, no key, fifteen seconds. Comments, captions, channels, or volume past a thousand videos: the Data API and its quota math. Subtitles or whole-channel walks: yt-dlp, updated this week. Magpie’s channel support is the gap I’d most like closed, and until it is, @handle URLs bounce.
If your targets are other platforms, the same one-command pattern covers Reddit and Hacker News: the Reddit scraper guide and the sibling site guides under scrapers by site. The social media scraper roundup ranks what’s legal to collect across platforms, and when a hosted actor fits better, the Apify comparison for YouTube covers the per-record math.
Live-tested October 16, 2026. YouTube’s player internals move; the routes don’t.
