Developer

Scrape GitHub repository data into JSON

Point one API call at a GitHub repository and get the full name, description, primary language, stars, and forks back as schema-validated JSON.

One call

Send the URL and a plain-language description of the fields. That's the whole integration.

curl https://api.webscrape.ai/v1/smartscraper \
  -H "X-API-Key: wsg_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "website_url": "https://github.com/nuxt/nuxt",
    "user_prompt": "Extract the repository full name, description, primary language, star count, and fork count."
  }'

You get back JSON shaped to match — nothing to parse:

{
  "status": "completed",
  "data": {
    "result": {
      "full_name": "string",
      "description": "string",
      "language": "string",
      "stars": "number",
      "forks": "number"
    }
  },
  "credits_used": 5,
  "credits_remaining": 495,
  "request_id": "req_a1b2c3d4"
}

Example response shape; data.result matches the fields you asked for. Failed fetches return a non-completed status and cost nothing.

Fields you'll usually want from GitHub

Field	Type	What it is
`full_name`	string	owner/repo
`description`	string	Repository description
`language`	string	Primary language
`stars`	number	Star count
`forks`	number	Fork count

GitHub repo pages are largely static and a public REST API exists too — a good target for structured repository data.

How it works

Tiered fetching. A fast request carrying a real browser's fingerprint handles most pages. When a site needs a full browser, turn on stealth mode and the request runs through a hardened stealth browser instead.
AI extraction. The cleaned page is extracted into data matching your fields; pass an output_schema and the result is validated against it, with one repair pass on a miss.
One wallet, success-only. Proxies, browser time, and extraction all come out of one credit balance, and a failed fetch costs nothing.
Blueprints for repeat runs. Once an extraction works, pin it: subsequent runs replay reverse-engineered selectors with no AI step in the loop.

When it breaks

A page returns a 200 with an empty shell: that can look like success to the fast tier, so force the browser tier for that request if it recurs.
A field comes back wrong after a redesign: re-run live extraction, or re-pin the Blueprint. Drift surfaces as validation_warnings, not a silent null.

FAQ

Is there a GitHub scraping API?

Point webscrape.ai's /v1/smartscraper endpoint at a GitHub URL and describe the fields you want. You get back structured JSON matching those fields, with no separate proxy or browser to set up. Pass an output_schema when you want it validated against a strict shape.

Do I need proxies to scrape GitHub?

No. Proxies and browser handling are built in and come out of one credit balance. Stealth mode adds a small per-request surcharge, so you only pay extra on the requests where you turn it on.

What happens when GitHub changes its layout?

Live extraction is anchored to the fields you describe, so it survives most redesigns. For repeat runs you can pin a Blueprint; if the page drifts, the run returns validation_warnings instead of silently dropping fields.

Other Developer scrapers

Stack Overflow

Grab a key and point it at a GitHub page. New accounts get free credits — no card.

Open the playground All scrapers