Every page,
turned into clean data.
Webcrawl fetches, escalates past blockers, and hands back structured markdown or JSON — as a REST API, or a native MCP tool inside Claude Code.
$ curl -X POST /scrape -d '{"url":"...", "output":"markdown"}'
Noise in. Signal out.
Pick a page. Run it. See exactly what comes back — no signup, no waiting.
Five calls. Every shape of data you need.
Same operations, two front doors: REST for any language, or native MCP tools for Claude Code.
Scrape
Give it a URL. Get back clean markdown or typed JSON — it keeps trying until it gets through.
Extract
Describe the fields you want. No CSS selectors to write or maintain when the page changes.
Crawl
Point it at a site. It follows links to a depth you set and returns clean pages for each one.
Map
Get every URL on a site fast, without fetching page content — useful for scoping a crawl first.
Search
Web results, optionally fetched and cleaned in the same call.
Call it from wherever you build.
Same five operations, two doors in: plain REST, or a native tool inside Claude Code.
Every failure maps to a specific next move.
Not blind retries — each response code routes to the action that actually resolves it.
| Response | Action | Escalates to |
|---|---|---|
| 403 Forbidden | Switch proxy | retry automatically |
| 429 Too Many Requests | Back off 2s → 4s → 8s | retry, same pass |
| Empty body / 0 bytes | Try harder | disguised pass |
| JS challenge detected | Try harder | full simulation |
| 5xx Server Error | Wait 5s, retry | same pass |
| All layers exhausted | Structured error | no silent failure |
What people actually run it for.
Price and stock watching
Extract price, stock, and spec fields from competitor product pages on a schedule — get an alert the moment something changes.
"Pricing page changes, we know within the hour."
Lead list building
Map a directory site, crawl each listing, and extract structured contact and firmographic fields into a ready-to-use table.
"A clean lead list without a scraping team."
Docs-to-Markdown pipelines
Crawl a docs site and get every page back as clean Markdown — ready to feed into a RAG index or an internal wiki.
"Any doc site becomes clean, structured Markdown."
One price. Hosted for you.
A flat $99 license for the hosted workspace — no per-page billing, no infrastructure to run yourself.
Webcrawl — Workspace license
Every operation, the full anti-bot stack, and the MCP server — hosted, with a monthly credit allowance.
- REST API + Claude Code MCP server
- 3-layer anti-bot fallback, adaptive retry
- Scrape, crawl, map, search, and extract
- Monitors, job history, and team seats
Before you ask
Do I need to host anything?+
No — Webcrawl is a hosted workspace. Sign up, grab an API key, and start calling the REST API or the MCP server right away.
What's included in the $99 license?+
Every operation (scrape, crawl, map, search, extract), the full anti-bot stack, the MCP server, a monthly credit allowance, and team seats — one flat price, no per-page billing.
Is this compliant to use?+
Webcrawl is fetch infrastructure — it's on you to respect each target site's terms of service and robots.txt. We don't support or condone scraping in violation of a site's terms.
What happens after I pay?+
You'll get instant access to your account dashboard to create API keys, invite teammates, and start running jobs.