← WebExtract

WebExtract API docs

Base URL: https://api.webextract.dev (same app at https://webextract.dev). Machine-readable: /openapi.json · /llms.txt · /.well-known/x402.

Framework guides: Vercel AI SDK (@webextract/ai-sdk) · LangChain (langchain-webextract for Python, @webextract/langchain for JS).

Endpoints

Prices and provenance

product.price_detail is the recommended price field. Each entry in prices[] has type (list, sale, current), amount, currency, and the matched variant when the page proves it. It never contains 0. When no price can be trusted, status is unresolved and unresolved.reason says why (for example no_offer_in_structured_data, variant_not_matched, zero_price). When the URL selects a variant (?variant=, ?sku=, ?size=…), the flat product.price is that variant's price. If the variant cannot be matched on a page whose variants have different prices, product.price is null instead of another variant's price. Listing and category pages return product: null with product_unresolved.reason: "listing_page". content_source is raw_html (no JavaScript is executed); product.source is the extraction method (json-ld, opengraph, microdata, heuristic).

Both data endpoints also accept POST with a JSON body {"url": "https://..."} (or a form body).

curl -H "X-Billing: free" "https://api.webextract.dev/v1/extract?url=https://example.com"
curl -X POST https://api.webextract.dev/v1/markdown -H "X-Billing: free" -H "Content-Type: application/json" -d '{"url":"https://example.com"}'

Auth and billing

• Free tier (opt-in on REST): 50 calls/day per IP, no key. Send X-Billing: free (or ?free=1). X-Free-Remaining is returned only in free mode. Without the opt-in, a request with no key and no payment gets HTTP 402 x402_required. MCP keeps 50 free calls/day with no header (same per-IP counter). Cloud and agent egress IPs can rotate, so the counter may vary.
• Prepaid key: Authorization: Bearer <key>. Buy credits at /#pricing; credits never expire. X-Credits-Charged / X-Credits-Remaining headers.
• x402 pay per call (no account): x402 v2, scheme exact, USDC on Base (recommended), Polygon, Arbitrum One, World Chain, Monad, Avalanche C-Chain or OP Mainnet. Ethereum mainnet is not supported. Payment settles at the quoted price and only when the request succeeds.

X-Billing header: free opts into the free tier. x402 returns the 402 with payment requirements (reason x402_requested). x402-testnet uses the sandbox (below). Any other value returns 400 invalid_billing_mode.

Order of checks: 1) url validation (400, never billed) → 2) Authorization: Bearer key (credits; wins over everything below) → 3) payment header PAYMENT-SIGNATURE (wins over X-Billing: free) → 4) X-Billing: x402 → 402 x402_requested → 5) X-Billing: free or ?free=1 → free quota, or 402 free_limit_reached with X-Free-Remaining: 0 → 6) otherwise 402 x402_required.

Note on ?free=1: it is read from the API's own query string. If your target URL has its own query string, URL-encode it (curl -G --data-urlencode "url=https://a.com/?x=1" -d free=1 ...) or simply use the header.

SDK note: official x402 SDKs (@x402/evm 2.28, Python x402 2.25) do not include OP Mainnet and World Chain USDC in their default asset lists; add them to spendControls.allowedAssets, and register only the network(s) where your wallet holds USDC (or pass a requirements selector), otherwise the SDK picks the first option (Base).

HTTP 402 Payment Required

The x402 v2 requirements are in the PAYMENT-REQUIRED response header (base64 JSON). Sign one option and retry with PAYMENT-SIGNATURE. The JSON body says why payment is needed; the same value is in the X-Payment-Reason header:

{"error":"payment_required","reason":"x402_required","message":"Payment required. ... or try it free: add header 'X-Billing: free' ...",
 "x402":{"price":"$0.005","network":"eip155:8453","networks":["eip155:8453","eip155:137",...],"asset":"USDC"},
 "stripe":{"buy_url":"https://webextract.dev/#pricing","packs":[...]}}

With a prepaid key and no credits left you get 402 {"error":"no_credits"}.

Errors

All errors are JSON and never charged: 400 missing or invalid url (checked before any billing) or unknown X-Billing value · 401 invalid API key · 402 payment required / out of credits · 413 page over 3 MB · 415 unsupported content type · 422 target site blocked or errored · 451 robots.txt disallows the URL.

Sandbox (free test money only)

Test an x402 client end to end without real money. Same endpoints and responses:

https://api.webextract.dev/sandbox/v1/extract?url=...
https://api.webextract.dev/sandbox/v1/markdown?url=...
# or on the normal endpoints:
curl -H "X-Billing: x402-testnet" "https://api.webextract.dev/v1/markdown?url=https://example.com"

• Accepts only Base Sepolia (eip155:84532) test USDC 0x036CbD53842c5426634e7929541eC2318f3dCF7e (EIP-712 name USDC, version 2), settled by the https://x402.org/facilitator testnet facilitator. Get test USDC at faucet.circle.com.
• Test money only: no real payment is taken, no free-tier calls or credits are used, and sandbox endpoints are not listed in any discovery directory.
• Rate limited to 20 requests/minute and 300/day per IP (HTTP 429 with Retry-After).

MCP

Remote streamable-http server: https://api.webextract.dev/mcp (stateless JSON). Tools extract_url and page_markdown, both with outputSchema and structuredContent. Registry name dev.webextract/extract. Server card: /.well-known/mcp/server-card.json. MCP calls use the free tier (50/day per IP, no header needed) or a Bearer key; for pay-per-call use the REST endpoints with x402.

Crawling policy

We identify as WebExtractBot, obey robots.txt, never bypass CAPTCHAs or anti-bot protection, and refuse private network addresses.