Now live · free credits on signup

The web, as an API.

The web scraping API that turns any URL into clean, schema-validated JSON. Reliable fetching, data scraping, and one endpoint your team never has to maintain.
Get an API key 500 free credits · no card
Try in playground
  • Billed only on success
  • Browser-grade fetching
runs live · no signup
https://
// smartscraper · smartbrowsePrompt it, or record itSmartScraper reads the page from a prompt; SmartBrowse records the clicks & logins that reach it.
~/wsai — smartscraperlive

SmartScraper — a URL and a plain-English prompt in, schema-validated JSON out.

One SDK, every stack — drop it in and go

pnpm addwebscrape-ai
// by the numberssnapshot
Requests processed
128k
since launch
First-attempt success
96%
before retries
Median latency
520ms
full pipeline
Schema-valid first pass
98%
benchmark corpus
Features

Built for developers who need reliable scraped data

A complete extraction toolkit — every feature audited, versioned, and shipped from one endpoint.

AI Agent Extraction

Describe what you want in plain English. Our agents understand the page semantically and return exactly the structured data you asked for.

user_prompt: "Extract top 5 stories
               with title, url, points"

✓ 5 items · 312ms
  [{ title, url, points }, …]

SmartBrowse

Record clicks and navigation once in a visual studio, then replay the recipe as an API on real Chrome. No code, no selectors.

rec ▸ click "Next page"rec ▸ extract 24 rows✓ saved recipe✓ replay ×1000

Intelligent Chunking

Long pages are split, processed in parallel, and deduped on merge — so even huge documents extract fast and completely.

splitparallelmergededup

Content Reduction

A local NLP layer strips navbars, ads, and noise before extraction — cutting cost and latency by 50–80% with no accuracy loss.

noise removed80%
1.48M chars → 340k chars·1.2 MB → 240 KB

Schema Enforcement

Validate, repair, guarantee. JSON Schema in, conforming data out — with one automatic repair pass when a field misses.

validate schema✓ 5/5 fields matched✓ no repair needed

Auto-recovery

Failed fetches retry automatically with backoff — and you are never charged for a failed request.

free retriesautomatic backoff

Any Input Source

URLs, raw HTML, Markdown, or PDFs — one API for all of them.

URL·HTML·MD·PDF
How It Works

One request. Six stages. Clean JSON.

Every request flows through the same deterministic pipeline — fetch, clean, reduce, extract, validate. No black box.
~/wsai — pipeline.trace live
horizontal · trace6 stages
URLPOST /v1/smartscraperrequest
request
fetchworker fetch12ms405 KB html
405 KB html
cleanhtml dom cleaner3ms80 KB dom
80 KB dom
reducenlp filter23ms5 KB text
5 KB text
extractvlm layer480ms5 fields
5 fields
JSONschema · validated2ms200 OK
total0/6 stages·~520ms
Use Cases

What you can build

Teams use SmartScraper to power product feeds, agents, dashboards, and entire web scraping pipelines.

E-commerce monitoring

Track prices, stock, and reviews across thousands of product pages with consistent JSON schemas.

Lead generation

Extract names, emails, titles, and company info from directories and profiles at scale.

News & content intel

Pull articles, authors, dates, and entities from any publisher into clean, queryable data.

AI agent tooling

Plug structured web data into LangChain, n8n, or your own agents — no scrapers to maintain.

Why SmartScraper

Stop fighting selectors and broken scrapers

A side-by-side look at what you actually get out of the box.
~/wsai — compare
compare.grid8 capabilities
capabilitySmartScraperDIY scraperHeadless browser
any site, no selectorsSmartScraper: yes — no selectors neededDIY scraper: no — hand-write selectorsHeadless browser: partial — you write the parser
structured outputSmartScraper: yes — schema-valid JSONDIY scraper: no — validate it yourselfHeadless browser: partial — raw HTML, no schema
auto-recoverySmartScraper: yes — retries & repairsDIY scraper: no — build your own retriesHeadless browser: partial — reloads, never re-parses
long pagesSmartScraper: yes — auto-chunked to fitDIY scraper: no — split it yourselfHeadless browser: partial — whole DOM, you chunk
noise removalSmartScraper: yes — nav & ads strippedDIY scraper: no — pay tokens for noiseHeadless browser: partial — full DOM, nothing trimmed
input formatsSmartScraper: yes — PDF, HTML & MarkdownDIY scraper: no — wire each formatHeadless browser: partial — HTML only, no PDF
maintenanceSmartScraper: yes — survives redesignsDIY scraper: no — breaks on redesignHeadless browser: partial — markup shift breaks it
infrastructureSmartScraper: yes — one API callDIY scraper: no — run your own fleetHeadless browser: partial — you host the browser
Developer Experience

One endpoint. Any language.

Add web scraping to any stack — official SDKs for Python, Node, Go, Rust & Java, or any plain HTTP client. Schema-validated output means no post-processing.
~/wsai — request
# fetch the top 5 HN stories
curl -X POST https://api.webscrape.ai/v1/smartscraper \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \
  -d '{
    "website_url": "https://news.ycombinator.com",
    "user_prompt": "Extract the top 5 stories with title, url, and points"
  }'
# fetch the top 5 HN stories
from webscrape_ai import Client

with Client() as client:  # auth via WEBSCRAPE_API_KEY
    resp = client.smartscraper(
        website_url="https://news.ycombinator.com",
        user_prompt="Extract the top 5 stories with title, url, and points",
    )
    print(resp.data.result)
// fetch the top 5 HN stories
import { Webscrape } from "webscrape-ai";

const client = new Webscrape();  // auth via WEBSCRAPE_API_KEY
const res = await client.smartscraper({
  website_url: "https://news.ycombinator.com",
  user_prompt: "Extract the top 5 stories with title, url, and points",
});
console.log(res.data.result);
// fetch the top 5 HN stories
// go get github.com/webscrape-ai/webscrape-ai/sdk/go
client, _ := webscrape.New()  // auth via WEBSCRAPE_API_KEY

resp, _ := client.SmartScraper(context.Background(), &webscrape.SmartScraperRequest{
  WebsiteURL: "https://news.ycombinator.com",
  UserPrompt: "Extract the top 5 stories with title, url, and points",
})
fmt.Println(string(resp.Data.Result))
// fetch the top 5 HN stories
use webscrape_ai::{blocking::Client, SmartScraperRequest};

let client = Client::from_env()?;  // auth via WEBSCRAPE_API_KEY
let resp = client.smartscraper(
    SmartScraperRequest::new(
        "https://news.ycombinator.com",
        "Extract the top 5 stories with title, url, and points",
    ),
)?;
println!("{}", resp.data.result);
// extraction pipeline
  • fetch12ms
  • clean3ms
  • reduce23ms
  • extract480ms
  • validate2ms
total~520ms
~/wsai — response
response.json200 OK
// 200 OK · extracted in ~520ms
{
  "status": "completed",
  "data": {
    "result": {
      "stories": [
        {
          "title": "Show HN: I built a real-time code editor",
          "url": "https://example.com/editor",
          "points": 342
        },
        {
          "title": "Why Rust is the future of systems programming",
          "url": "https://example.com/rust",
          "points": 281
        },
        // … 3 more stories
      ]
    }
  },
  "credits_used": 5,
  "credits_remaining": 495,
  "request_id": "req_aB3xY9Kp"
}
Pricing

Simple, transparent pricing

Start free, scale as you grow. Pay only for what you use, no hidden fees.

Founding launch30% off for life on any subscription with code FOUNDING

Free

Free forever — credits refill every month, not a one-time trial.

$0

≈ 60 AI extractions / month

  • 500 starting credits
  • 300 free credits every month
  • 1 concurrent request
  • 10 requests / minute
  • 7-day data retention
  • Limited SmartBrowse

Hobby

For side projects and prototypes.

$19

≈ 1,000 AI extractions / month

  • 5,000 credits / month
  • 10 concurrent requests
  • 100 requests / minute
  • Standard fetch tier
  • 30-day data retention
  • 20% off extra credit

Enterprise

Tailored solutions for large organizations.

Custom

Unlimited extractions

  • Unlimited credits
  • Custom rate limits
  • Dedicated infrastructure
  • Dedicated routing
  • 99.9% SLA guarantee
  • Dedicated account manager
  • On-premise deployment

Base rate: 1 credit / Scrape · 5 / SmartScraper AI extraction · 2 / SmartBrowse run replay

AI agent? Read the plain-text version at /pricing.md.

FAQ

Frequently asked questions

Everything teams ask before going to production.
// frequently asked · 6 entries
// changelog

What's new

The latest platform updates, newest first.

~/wsai — changelog
shipping.log5 releases
  • Public launch — API + dashboard livev1.0
  • Official SDKs — Go, Python, Node, Rust & Java5 languages
  • Extraction engine rebuilt in Gop50 −40%
  • Private beta — early access opensinvite-only
  • Isolated microVM browsers + managed networkingincluded

Ready to extract structured data?

Get an API key in seconds — 500 free credits, no card required. Or try it first in the playground.

~/wsai — extract live
extract · run5 stages