How to Scrape Email Addresses and Contacts From a Website

Most tutorials that show you how to scrape email from website pages hand you a regex and wish you luck. Run one against a real site and about a fifth of what comes back is junk: I crawled 8 pages of Debian’s contact section and harvested 27 addresses, of which at least 6 were documentation placeholders like [email protected] and [email protected]. This guide runs it the other way around: ask the page’s structured data first, crawl plus regex second, an LLM schema for the messy rest, then filter hard and mind the law. Every command ran today against live sites.

Where emails actually live (I checked)

The tutorials assume email addresses are sprinkled through page text. On modern sites they mostly aren’t. Here is what I found today, one load per site, from a home connection:

Location	Found it on	Notes
mailto: links	debian.org/contact (14 unique)	The reliable spot for organizations that publish addresses at all
JSON-LD email/telephone fields	rogersgardens.com (1 of 6 business sites I probed)	Structured, typed, machine-readable when present
Plain text	debian.org contact wiki pages	Usually addresses already inside mailto: links
Nothing	momofuku.com, flourbakery.com, tartinebakery.com	Contact form only; there is no address to scrape

That last row is the one nobody mentions. Restaurants and retailers increasingly publish no email at all, and no scraper can extract what isn’t there.

Before you start

• The Magpie CLI installed. Free download, single Go binary, no runtime dependencies. The commands below are the plain CLI; if you want to drive it from an agent, it also ships a 13-tool MCP server.
• jq or Python for filtering output.
• Ten minutes. Legality: scraping what a page publishes is generally the easy part; using the addresses is where the law bites. The web scraping legality guide covers the baseline, and the GDPR/CAN-SPAM section below covers email specifically.

One environment note: my requests to debian.org came back in French because Debian negotiates language by IP. If a site serves you the wrong locale, pass --lang en-US to Magpie’s fetch. Same page, same command, English output.

Step 1: Scrape email from website structured data (JSON-LD)

Sites that want to be understood by machines describe themselves in JSON-LD (a JSON block that mirrors schema.org vocabulary; the JSON-LD glossary entry has the anatomy). Contact data in JSON-LD beats regex because it arrives typed: this string is the phone number, this one is the address.

Magpie’s zero-LLM local_business vertical reads that block and hands back a record. Zero-LLM means no API key and no extraction cost:

magpie scrape "https://rogersgardens.com/" --vertical local_business --format jsonl

{
  "url": "https://rogersgardens.com/",
  "title": "Newport Beach Nursery & Garden Center | Roger's Gardens",
  "vertical": "local_business",
  "record": {
    "address": {
      "country": "US", "locality": "Corona del Mar",
      "postalCode": "92625", "region": "CA",
      "street": "2301 San Joaquin Hills Road"
    },
    "email": "[email protected]",
    "name": "Roger's Gardens",
    "telephone": "+1-949-640-5800"
  }
}

(abridged). One command, one business: name, public inbox, phone, full street address. If you only want to look at the markup rather than extract a record, magpie scrape "https://rogersgardens.com/" --page-format json returns a structured_data sidecar with every JSON-LD block on the page, and the JSON-LD extraction guide goes deeper on that.

Failure case: the vertical fires on LocalBusiness, Restaurant, Organization and other business types, but it can only read what the site ships. Most of the six business homepages I probed had no contact fields in their JSON-LD at all. When the record comes back empty, move to the next method.

Step 2: Crawl the site and harvest with a regex

When the markup is silent, brute force it. The pattern: crawl the site into a corpus, then run one regular expression over the text.

magpie crawl "https://www.debian.org/contact" --corpus \
  --max-pages 8 --max-depth 1 --rate 2 --out debian.jsonl

python3 - <<'EOF'
import json, re
seen = {}
for line in open("debian.jsonl"):
    rec = json.loads(line)
    for e in re.findall(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}',
                        rec.get("markdown", "")):
        seen.setdefault(e, []).append(rec["url"])
for addr in sorted(seen):
    print(addr)
EOF

That’s the real Debian run from the intro: 8 pages fetched, 27 unique addresses. It works because Magpie’s crawl stores cleaned markdown per page, and the cleaning keeps addresses in link text. Two details worth knowing before you trust it:

• The noise is not a bug in your regex, it’s the web. The 27 included [email protected], [email protected], and several [email protected]-style placeholders from documentation examples. Six-plus junk out of 27 is normal for a site that documents things. Fix: filter to domains you expect (@debian.org cut my list to 14), drop known placeholder domains (example.com, domain.com, yourdomain), and eyeball the survivors.
• Harvest mailto: from HTML, not from markdown. Magpie’s markdown rewrites mailto: hrefs into page-relative links, so regexing hrefs out of the markdown fails. The addresses still survive as link text, and if you specifically need the raw hrefs, --page-format html preserves them (14 unique mailto: values on that same page).

Failure case: quality-gated pages. Big chains put their location pages behind Cloudflare; a Dunkin’ store page returned HTTP 403 to my fetch and Magpie reported quality blocked (empty) instead of pretending otherwise. It detects bot challenges, it does not defeat them.

Step 3: An LLM schema for pages that resist both

Some pages bury contact info in prose: “Reach our press team at [email protected], or call the Amsterdam office.” Regex finds [email protected] but loses who owns it. Magpie’s schema-driven extraction sends the cleaned page to an LLM and enforces a JSON shape you define:

# contact-schema.yaml
type: object
properties:
  emails:
    type: array
    items:
      type: object
      properties:
        address: { type: string }
        purpose: { type: string, description: "what this inbox is for" }
  phones: { type: array, items: { type: string } }

magpie scrape "https://example.org/contact" \
  --schema contact-schema.yaml --provider ollama

The provider is your choice: a hosted key for Anthropic, OpenAI, or OpenRouter, or a local Ollama model if you don’t want a page’s text leaving your machine. Per-page cost is whatever your provider charges for the cleaned text; the --max-cost flag caps it per run. I cover the mechanics of shaping these schemas on the structured text extraction guide.

Full disclosure on my test: I verified the schema flag and provider list against the CLI today, but my local LLM gateway token was refusing requests when I ran it, so the JSON above is the command shape, not captured output. If your key works, it works.

Phones and addresses: build the full contact scraper

An email alone is a weak contact record. If you’re building a contact scraper for sales or outreach, you want the phone, the address, and the opening hours, and they usually live in the same JSON-LD block as the inbox. That’s why I’d start every contact scrape at the structured data, not at a regex: one --vertical local_business call returns all of it, typed.

The address scraper case

The address object in the record above has street, locality, region, postal code, and country as separate fields, ready for a spreadsheet or a geocoder. A job that looks like an address scraper — dealer locators, store directories, franchise sites — is the same one command, and the typed fields save the address-parsing regexes people usually write. To export, pipe the JSONL records into a CSV with two lines of Python, or follow the scrape-to-CSV guide for the full pipeline.

What Magpie won’t do here

Plainly, since this page will rank for “email scraper” queries: Magpie has no dedicated email finder. I checked today: its 27 zero-LLM verticals contain no email extractor, and there’s no command that takes a company name and returns an address. It harvests what pages publish; it does not find people. If you need “find the CTO’s email at Acme,” that’s an email-finder product’s job (Hunter.io and friends search by name plus domain, a different mechanism entirely). Also by design: no CAPTCHA or bot-protection bypass, so pages that wall their contact info from automated access stay walled.

What it does do well here: the JSON-LD record in one command, polite crawling with robots.txt respected and per-host rate limiting, and cleaned markdown that makes the regex step trivial. The whole pipeline runs locally, so the addresses you collect never pass through a third party’s servers.

The legal part: GDPR and CAN-SPAM

Not legal advice, and a lawyer should sign off before you run a business on this. But here is the shape of it, with the sources.

CAN-SPAM (US). Collecting an address that a website publishes isn’t by itself a federal crime, but sending to it can be. The FTC’s compliance guide lists harvesting email addresses from websites, and building lists via dictionary attacks, among the acts that carry criminal penalties including imprisonment — specifically when the site’s posted policy says it won’t share addresses. If you send commercial email in the US, you owe every recipient accurate headers and a valid physical postal address in the message. The opt-out link must work, and you honor requests within 10 business days. Each violating email exposes you to a fine of up to $53,088 (the FTC’s inflation-adjusted figure).

GDPR (EU/EEA). An email address is personal data the moment it identifies a person, which covers most individual inboxes and plenty of role-adjacent ones. You need a lawful basis to scrape and store them, and “it was publicly visible” is not one. B2B cold outreach is not automatically illegal: many teams claim legitimate interest under Article 6(1)(f) and document the balancing test, while B2C marketing email generally needs consent. Enforcement is real: in October 2022 France’s CNIL fined Clearview AI €20 million for scraping personal data without a basis, and the ceiling for violations is €20 million or 4% of global turnover, whichever is higher.

The practical rules I’d actually follow:

• Scrape pages that publish addresses for public contact. Skip sites whose policy forbids harvesting, and skip anything behind a login or a bot wall.
• Filter out individuals where your use doesn’t justify them; prefer role inboxes (press@, support@, info@).
• Record where every address came from (URL and date). The Magpie corpus JSONL already carries the source URL per page, which is most of that job.
• Honor opt-outs within days, not weeks. And if you’re in the EU, treat cold email to individuals as consent-required until a lawyer tells you otherwise.

FAQ

Is it legal to scrape email addresses from a website? Scraping publicly published addresses is generally lawful in itself. The liability shows up in what you send: CAN-SPAM criminalizes sending to addresses harvested against a site’s posted policy, and GDPR requires a lawful basis to process personal data. See the legality guide.

Why doesn’t a regex find the email on most modern sites? Three reasons: the address is rendered client-side by JavaScript, it’s obfuscated as name [at] domain dot com, or the site never publishes one and offers a contact form instead. I probed three restaurant chains today; none had any scrapeable address.

Can I scrape emails from LinkedIn or Google Maps? Both explicitly forbid automated collection and both actively block it; LinkedIn has years of litigation behind that position. There’s a page on LinkedIn scraping legality if that’s your actual question.

What’s the difference between an email scraper and an email finder? A scraper harvests addresses that a page publishes. A finder (Hunter.io, Snov.io) guesses or looks up the address behind a name plus domain, from its own database. Scrapers respect what’s public; finders are where the legal exposure usually starts.

How do I keep my scraped list clean? Filter to domains you recognize, drop placeholder domains like example.com, deduplicate with source URLs, and manually review before any send. My Debian crawl had 6 placeholder addresses in 27 results; that ratio is typical.

The order of operations

JSON-LD first: one command gives you a typed record when the site ships structured data. Crawl plus regex second, with a junk filter you wrote yourself, not one you copied. LLM schema third for the prose-heavy pages.

Sites redesign and contact pages move. magpie watch diffs a page on a schedule and can fire a webhook when it changes, which is your cue to re-run the harvest against whatever the new layout publishes. And before any of this becomes a send button, read the CAN-SPAM harvesting rule and the GDPR lawful-basis requirement again, because the fine per email is bigger than your entire scraping budget.