Extract
Describe the record you want and any of seven providers turns pages into validated JSON. The part nobody else ships: it learns selectors and stops billing you.
On this page
What extraction is
A schema is a JSON Schema (draft 2020-12, YAML or JSON) naming the fields you want — title, price, in_stock, upc. Give it to a scrape (the schema chip), a crawl (Extracted mode), or Re-extract over content already fetched, and the LLM fills the record; the schema validates the output before you see it. Authoring is covered on Schemas.
The fastest path is the draft button: describe the record in one sentence on the Scrape screen and Magpie writes the schema fitted to the page you just scraped. The badge on the draft names the provider, token count and cost.
Providers
Seven providers, bring your own key (except the two that need none):
| Provider | Auth | Billing |
|---|---|---|
anthropic |
API key | Metered |
openai |
API key | Metered |
openrouter |
API key | Metered, costed from usage |
opencode-go |
API key | Flat plan, exempt from cost caps |
opencode-zen |
API key | Pay-as-you-go credits |
ollama |
none — local daemon | Free |
codex |
your logged-in Codex CLI | Your subscription |
Keys go into the OS keychain from Settings → Providers, never a plain file, and are never displayed again. Every screen that can spend money also takes a per-run key that applies to that run only.
The selector cache
The first extraction on a new domain runs the LLM and records which CSS selectors produced each field. After a few pages agree, extraction switches to those selectors — later pages in the same crawl, and later scrapes with the same schema, extract with zero LLM calls. The telemetry bar says selector cache hit when that happens.
When a site redesign breaks the cached selectors, two remedies:
- Heal — Settings → Cache inspector lists every cached domain with its schema, selector and sample counts. Heal drops them; the next scrape re-synthesizes with an LLM pass.
- Bypass selector cache — the Advanced checkbox on Scrape, for one run.
Costs and caps
Three layers bound spend, smallest first:
- Max cost USD per run — the Advanced field; the run aborts before a call that would pass it.
- Cost ceiling on crawls — same guard across every page of the run.
- Monthly budget — Settings → Budget. Warns at 80%, stops metered calls at 100% until the 1st. Flat-plan and local providers never stop; CLI runs count but are not stopped.
The cost hint beside every Run button prices the next call before you make it, from the same projection the caps use.
Zero-LLM extraction
Verticals extract structured records with no LLM, no key, no cost — 27 of them, from GitHub repos to WooCommerce products. Pick one as the extractor chip on Scrape, or magpie vertical <url> in the CLI. auto dispatches by URL pattern; the permissive ones (og, article, event, job_posting, local_business, rss, woocommerce_product) only fire when named explicitly, so the default output shape stays stable.
For text that only looks structured, Search’s --scrape-top and the Search screen’s Scrape top fetch hits as Markdown in the same call.