WebExtract

Any URL → clean Markdown + fixed-schema JSON, built for AI agents and RAG pipelines.

What you get

• markdown: main content with nav/ads/scripts stripped, tables as GFM tables
• metadata: title, authors, published/modified dates, canonical URL, language
• product: name, brand, SKU/GTIN, price, currency, availability, rating
• tables: headers + rows with row_count / column consistency checks

Try it free

50 calls/day per IP, no key needed:

curl "https://THIS_HOST/v1/extract?url=https://example.com"

Pricing — prepaid credits, no subscription

Option A — card: pay via Stripe Checkout. Your API key is shown once on the success page.

• Credits never expire.
• No charge on failed fetches (404, timeout, blocked, robots.txt disallowed).
• Structured JSON at no extra cost: /v1/extract (Markdown + product/price, tables, metadata) = 1 credit; /v1/markdown (Markdown only) = 0.4 credit.
• If structured fields come back empty, you're only charged the Markdown rate.

Option B — no account (AI agents): pay $0.005 per structured call ($0.002 for /v1/markdown) in USDC on Base via x402. Past the free tier, requests without a key get HTTP 402 with payment requirements; x402 clients pay and retry automatically. Discovery: /.well-known/x402.

Use your key

curl -H "Authorization: Bearer lx_..." "https://THIS_HOST/v1/extract?url=https://example.com/product"
curl -H "Authorization: Bearer lx_..." "https://THIS_HOST/v1/balance"

Docs

Endpoints: GET /v1/extract?url=<page> → JSON: markdown, metadata, product (name/price/currency/availability via JSON-LD, microdata, OpenGraph or page heuristics), tables. GET /v1/markdown?url=<page> (or POST {"url": ...}) → Markdown only. Full schema: /openapi.json · agent summary: /llms.txt.

Auth: none for the free tier (50/day per IP); Authorization: Bearer <key> for prepaid credits (GET /v1/balance shows remaining); or pay per call with x402 ($0.005 USDC on Base, discovery at /.well-known/x402).

MCP: remote streamable-http server at https://api.webextract.dev/mcp, tools extract_url and page_markdown. Listed in the official MCP Registry as dev.webextract/extract.

Billing: Credits never expire. No charge on failed fetches (404, timeout, blocked, robots.txt disallowed). Structured JSON at no extra cost: /v1/extract = 1 credit, /v1/markdown = 0.4 credit. If structured fields come back empty, you're only charged the Markdown rate. robots.txt always respected. x402 payments settle at the quoted price.

Errors: 400 bad URL · 401 bad key · 402 out of credits / payment required · 422 site blocked · 429 free limit · 451 robots.txt disallow.

Crawling policy

We identify as WebExtractBot, obey robots.txt, never bypass CAPTCHAs or anti-bot protection, and refuse private network addresses.

OpenAPI · llms.txt