# ExtractHub

> ExtractHub is a web scraping API and web data extraction platform. An extractor is a saved, versioned web scraper for one kind of page (request settings plus field selectors) built in ExtractHub Studio. Owners release extractors as immutable, semantically versioned releases with an output schema, and anyone with an API key calls a pinned version (for example @user/slug@1) through a realtime REST endpoint, async jobs, the Batch API or the remote MCP server for AI agents, paying per successful extraction from prepaid USD credit.

Extractor = web scraper: an extractor is a saved, versioned web scraper for one kind of page (request settings plus fields and selectors), built and tested in ExtractHub Studio. Its id is @username/slug; when an owner renames, the old @oldname/slug ids keep working.

Version: releasing an extractor creates an immutable semver release (request options, extractor config and output schema). Calls must pin one: @username/slug@1 (latest 1.x), @username/slug@1.2.3 (exact) or @username/slug@draft (owner only). Responses carry X-Extractor-Version; deprecated majors add Deprecation and Sunset headers; yanked versions answer 410 to exact pins.

Realtime extract: GET /api/extractor/extract?target_url=...&extractor_id=... or POST /api/extractor/extract with {"extractor_id": ..., "target": {"url": ...}}, authenticated with the API key in the X-API-Key header or as Authorization: Bearer <key>. A request still queued for capacity after the wait limit becomes an async job (202 with job_id, status_url, result_url) unless async_fallback=false.

Batch API: POST /api/extractor/batch with extractor_id and targets [{url}], then GET /api/extractor/batch/{id} for progress and GET /api/extractor/batch/{id}/results for NDJSON results. Succeeded URLs are charged, and so are failed URLs whose scrape reached the provider (target error, block page, extraction error, provider timeout; once per URL); a batch pauses when credit runs out, resumes once credit is available and fails if the pause expires; POST /api/extractor/batch/{id}/cancel stops it.

Webhooks: batch.completed, batch.failed, batch.paused, job.completed, job.failed, signed with X-Webhook-Signature (t=...,v1=HMAC-SHA256) and retried for about 24 hours. Payloads never contain result data.

Sandbox: test API keys (exth_test_...) work on every extractor endpoint without fetching pages or using credit; sandbox_scenario simulates success, failed, slow, timeout_to_job and insufficient credit.

MCP server: AI agents (Claude Code, Claude Desktop, Cursor, ...) connect to the remote MCP server at https://api.extracthub.io/mcp (Streamable HTTP) with an ExtractHub API key as Authorization: Bearer <key> or X-API-Key. Tools: search_extractors, get_extractor, get_account, extract, get_job, cancel_job, create_batch, get_batch, get_batch_results, cancel_batch, plus extractor authoring: validate_extractor_config, create_extractor, update_extractor, release_extractor, set_sample_output, delete_extractor, plus pipelines: validate_pipeline, create_pipeline, update_pipeline, get_pipeline, list_pipelines, run_pipeline, get_pipeline_job, cancel_pipeline_job. Tool calls go through the same API endpoints, so pricing, rate limits, sandbox and usage logging are identical to REST. The free tools (catalog, account and authoring tools) also count toward the per-minute rate limit and are request-logged with cost 0. Agents can build extractors: validate_extractor_config (with sample_html: local dry run, no fetch) -> create_extractor (private draft) -> extract with @user/slug@draft -> update_extractor -> release_extractor (same release as the Studio, version from the output schema change); create / update / release need a live API key; delete_extractor removes a private, never released draft; extract with raw_html: 'link' returns a signed 1 h download link to the full page HTML (results are otherwise cut at 64 KB). Multi-step scrapes (list page -> detail pages) are pipelines: validate_pipeline (chain of pinned extractors, each step feeding the next through url_output_field, plus a cost estimate) -> create_pipeline -> run_pipeline (billed like a dashboard run) -> get_pipeline_job (progress, results preview, signed download link); also update_pipeline, get_pipeline, list_pipelines, cancel_pipeline_job. Saving and running pipelines need a live API key; schedules stay off unless explicitly enabled. Setup guide: https://extracthub.io/mcp.

Billing: prepaid USD credit, pay as you go, no subscription. Charged per scraped URL by tier: direct (no proxy, no JavaScript) $0.50 per 1,000 requests, HTTP extraction through the proxy $2.50 per 1,000 requests, JavaScript rendering (always through the proxy) $3.50 per 1,000 requests, same price for realtime, jobs, batches, pipelines and MCP. Sandbox test keys, Studio direct fetches and 50 Studio proxy fetches per day are free. Optional auto top-up and low-balance emails.

Errors: one body shape { success: false, error, errors: [{ code, message, path? }], requestId } with stable codes such as validation_error (400), unauthorized (401), insufficient_balance (402), rate_limited (429), upstream_error / target_error (502), queue_timeout (503), upstream_timeout (504).

## Docs

- [API reference](https://extracthub.io/api-reference): interactive reference of every extractor endpoint
- [OpenAPI document](https://extracthub.io/extractor-openapi.json): machine-readable spec (OpenAPI 3)
- [MCP server](https://extracthub.io/mcp): web scraping MCP server: connect Claude, Cursor and other AI agents with an API key
- [Extractor builder agent skill](https://extracthub.io/agent-skill): AI web scraper builder: open-source skill that lets an AI agent build, test and release extractors
- [Pipeline builder agent skill](https://extracthub.io/agent-skill#pipelines): companion skill: chain released extractors (list page, then detail pages) into a scheduled pipeline
- [Web scraper catalog](https://extracthub.io/extractors): public extractors (versioned web scrapers) with their versions, schema and examples
- [Web scrapers by site](https://extracthub.io/scrapers): public extractors grouped by the website they scrape

## Web scrapers by site

- [Google scraper](https://extracthub.io/scrapers/google): 5 public extractors for google.com, play.google.com, e.g. extractor_id=@extracthub1/google-serp@2
- [YouTube scraper](https://extracthub.io/scrapers/youtube): 3 public extractors for youtube.com, e.g. extractor_id=@extracthub1/youtube-video@2
- [Amazon scraper](https://extracthub.io/scrapers/amazon): 2 public extractors for amazon.com, e.g. extractor_id=@extracthub1/amazon-product@1
- [eBay scraper](https://extracthub.io/scrapers/ebay): 2 public extractors for ebay.com, e.g. extractor_id=@extracthub1/ebay-search@1
- [Etsy scraper](https://extracthub.io/scrapers/etsy): 2 public extractors for etsy.com, e.g. extractor_id=@extracthub1/etsy-listing@1
- [Indeed scraper](https://extracthub.io/scrapers/indeed): 2 public extractors for indeed.com, e.g. extractor_id=@extracthub1/indeed-job@1
- [LinkedIn scraper](https://extracthub.io/scrapers/linkedin): 2 public extractors for linkedin.com, e.g. extractor_id=@extracthub1/linkedin-job@1
- [Toscrape scraper](https://extracthub.io/scrapers/toscrape): 2 public extractors for books.toscrape.com, quotes.toscrape.com, e.g. extractor_id=@extracthub1/quotes-toscrape-list@1
- [Walmart scraper](https://extracthub.io/scrapers/walmart): 2 public extractors for walmart.com, e.g. extractor_id=@extracthub1/walmart-product@1
- [Yelp scraper](https://extracthub.io/scrapers/yelp): 2 public extractors for yelp.com, e.g. extractor_id=@extracthub1/yelp-business@1
- [Zillow scraper](https://extracthub.io/scrapers/zillow): 2 public extractors for zillow.com, e.g. extractor_id=@extracthub1/zillow-property@1
- [AliExpress scraper](https://extracthub.io/scrapers/aliexpress): 1 public extractor for aliexpress.com, e.g. extractor_id=@extracthub1/aliexpress-product@1
- [Apple scraper](https://extracthub.io/scrapers/apple): 1 public extractor for apps.apple.com, e.g. extractor_id=@extracthub1/apple-app-store-app@1
- [Best Buy scraper](https://extracthub.io/scrapers/bestbuy): 1 public extractor for bestbuy.com, e.g. extractor_id=@extracthub1/bestbuy-product@1
- [GitHub scraper](https://extracthub.io/scrapers/github): 1 public extractor for github.com, e.g. extractor_id=@extracthub1/github-repo@1
- [Target scraper](https://extracthub.io/scrapers/target): 1 public extractor for target.com, e.g. extractor_id=@extracthub1/target-product@1
- [Tripadvisor scraper](https://extracthub.io/scrapers/tripadvisor): 1 public extractor for tripadvisor.com, e.g. extractor_id=@extracthub1/tripadvisor-hotel@1
- [Trustpilot scraper](https://extracthub.io/scrapers/trustpilot): 1 public extractor for trustpilot.com, e.g. extractor_id=@extracthub1/trustpilot-reviews@1
- [Y Combinator scraper](https://extracthub.io/scrapers/ycombinator): 1 public extractor for news.ycombinator.com, e.g. extractor_id=@extracthub1/hacker-news-stories@1

## Optional

- [Home](https://extracthub.io/): product overview and FAQ
- [Full text for LLMs](https://extracthub.io/llms-full.txt): this file plus the FAQ and every public extractor
- [Dashboard](https://dash.extracthub.io): sign in, API keys, Studio, billing

## Public extractors

- [Yelp - business page](https://extracthub.io/extractor/@extracthub1/yelp-business): Yelp (yelp.com) business page: name, rating, reviews and photos counts, claimed flag, categories, address, coordinates, opening hours, public business phone… Call with extractor_id=@extracthub1/yelp-business@1.
- [Yelp - search results](https://extracthub.io/extractor/@extracthub1/yelp-search): Yelp (yelp.com) search results: position, name, business link, rating, approximate reviews count, neighborhood, price range, open status, categories, review… Call with extractor_id=@extracthub1/yelp-search@1.
- [TripAdvisor - hotel page](https://extracthub.io/extractor/@extracthub1/tripadvisor-hotel): TripAdvisor hotel page (Hotel_Review): hotel id, name, rating, reviews count, ranking, hotel class, price range, featured offer, address, coordinates, phone… Call with extractor_id=@extracthub1/tripadvisor-hotel@1.
- [Target - product page](https://extracthub.io/extractor/@extracthub1/target-product): Target (target.com) product page: TCIN, DPCI, UPC, name, brand, current and regular price, price text, rating, reviews and questions counts, highlights… Call with extractor_id=@extracthub1/target-product@1.
- [Best Buy - product page](https://extracthub.io/extractor/@extracthub1/bestbuy-product): Best Buy (bestbuy.com) product page: SKU, product id, name, brand, model, color, price, currency, comparable value and savings, availability, every offer (new… Call with extractor_id=@extracthub1/bestbuy-product@1.
- [LinkedIn Jobs - public job page](https://extracthub.io/extractor/@extracthub1/linkedin-job): LinkedIn public (guest, no login) job page (linkedin.com/jobs/view/...): job id, title, company, company page and logo, location (city, region, country)… Call with extractor_id=@extracthub1/linkedin-job@1.
- [LinkedIn Jobs - public job search](https://extracthub.io/extractor/@extracthub1/linkedin-jobs): LinkedIn public (guest, no login) job search page (linkedin.com/jobs/search?keywords=...&location=...): every job card with job id, title, link, company… Call with extractor_id=@extracthub1/linkedin-jobs@1.
- [YouTube - video page](https://extracthub.io/extractor/@extracthub1/youtube-video): Metadata of a YouTube video watch page (www.youtube.com/watch?v=...): video id, URL, title, channel name / id / URL, subscriber count and view count (from the… Call with extractor_id=@extracthub1/youtube-video@2.
- [Google Search - results page (SERP)](https://extracthub.io/extractor/@extracthub1/google-serp): Organic web results incl. video result cards (title, Google redirect URL, direct URL for videos, displayed URL / video meta, source, snippet), People also ask… Call with extractor_id=@extracthub1/google-serp@2.
- [Zillow - property details](https://extracthub.io/extractor/@extracthub1/zillow-property): Zillow property page (homedetails): zpid, address, coordinates, price, status, price cut, beds, baths, living area, at-a-glance facts, full description… Call with extractor_id=@extracthub1/zillow-property@1.
- [AliExpress - product page](https://extracthub.io/extractor/@extracthub1/aliexpress-product): AliExpress product page (/item/<id>.html): product id, title, brand, current price, original price, currency, availability, rating, reviews count, sold count… Call with extractor_id=@extracthub1/aliexpress-product@1.
- [Indeed - job page](https://extracthub.io/extractor/@extracthub1/indeed-job): Indeed.com job page (/viewjob?jk=...): job key, title, company, company rating and reviews count, location, job type, salary text with min / max / period /… Call with extractor_id=@extracthub1/indeed-job@1.
- [Indeed - job search results](https://extracthub.io/extractor/@extracthub1/indeed-jobs): Indeed.com job search results page (/jobs?q=...&l=...): every job card (main results and the grouped carousels) with job key, title, link, company, location… Call with extractor_id=@extracthub1/indeed-jobs@1.
- [Etsy - listing page](https://extracthub.io/extractor/@extracthub1/etsy-listing): Etsy.com listing page (/listing/<id>/...): listing id, title, price (lowest price for listings with options) and highest option price, original price on sale… Call with extractor_id=@extracthub1/etsy-listing@1.
- [Etsy - search results](https://extracthub.io/extractor/@extracthub1/etsy-search): Etsy.com search results page (/search?q=...): the 64 listings on the page with listing id, title, link, image, price, original price, discount, shop name… Call with extractor_id=@extracthub1/etsy-search@1.
- [Walmart - product page](https://extracthub.io/extractor/@extracthub1/walmart-product): Walmart.com product page (/ip/...): item id, name, brand, model, UPC, current price, was price, savings, rating, reviews count, availability, seller… Call with extractor_id=@extracthub1/walmart-product@1.
- [Walmart - search results](https://extracthub.io/extractor/@extracthub1/walmart-search): Walmart.com search results page (/search?q=...): the products on the page with item id, name, link, image, price, currency, rating, reviews count, seller… Call with extractor_id=@extracthub1/walmart-search@1.
- [YouTube - channel videos](https://extracthub.io/extractor/@extracthub1/youtube-channel-videos): Latest videos of a YouTube channel (www.youtube.com/@handle/videos): channel name, id, handle, description, subscriber and video counts, plus the videos with… Call with extractor_id=@extracthub1/youtube-channel-videos@1.
- [eBay - search results](https://extracthub.io/extractor/@extracthub1/ebay-search): eBay.com search results page (/sch/i.html?_nkw=...): the listings on the page with item id, title, link, image, price, original price, condition, subtitle… Call with extractor_id=@extracthub1/ebay-search@1.
- [Google Maps - place details](https://extracthub.io/extractor/@extracthub1/google-maps-place): Details of a Google Maps place page (www.google.com/maps/place/...): name, category, rating, review count, full address, phone, website, Plus Code, opening… Call with extractor_id=@extracthub1/google-maps-place@1.
- [eBay - item page](https://extracthub.io/extractor/@extracthub1/ebay-item): eBay.com item page (/itm/ID): title, price, currency, list price, condition, availability, quantity available and sold, shipping cost, return window, brand… Call with extractor_id=@extracthub1/ebay-item@1.
- [Amazon - product page](https://extracthub.io/extractor/@extracthub1/amazon-product): Amazon.com product page (/dp/ASIN): ASIN, title, brand, buy-box price, list price, rating, review count, availability, seller, feature bullets, images and the… Call with extractor_id=@extracthub1/amazon-product@1.
- [Amazon - search results](https://extracthub.io/extractor/@extracthub1/amazon-search): Amazon.com search results page (/s?k=...): the products on the page with ASIN, title, link, image, price, list price, rating, ratings count… Call with extractor_id=@extracthub1/amazon-search@1.
- [YouTube - video transcript](https://extracthub.io/extractor/@extracthub1/youtube-transcript): Transcript (captions) of a YouTube video (www.youtube.com/watch?v=...): timestamped segments with start time, duration and text. Plays the video in a browser… Call with extractor_id=@extracthub1/youtube-transcript@2.
- [Zillow - search results](https://extracthub.io/extractor/@extracthub1/zillow-search): Zillow search results page (city, ZIP, neighborhood): total result count, pages, next page link, and the listings on the page (41 per page) as schema.org… Call with extractor_id=@extracthub1/zillow-search@1.
- [Trustpilot - company reviews](https://extracthub.io/extractor/@extracthub1/trustpilot-reviews): Trustpilot company review page: company name, TrustScore, review counts, rating distribution, categories and the reviews on the page (author, country, rating… Call with extractor_id=@extracthub1/trustpilot-reviews@1.
- [Apple App Store - app page](https://extracthub.io/extractor/@extracthub1/apple-app-store-app): Apple App Store app page (apps.apple.com): app id, bundle id, name, subtitle, developer, category, rating, rating count, price, version, release notes, size… Call with extractor_id=@extracthub1/apple-app-store-app@1.
- [GitHub - repository page](https://extracthub.io/extractor/@extracthub1/github-repo): Public GitHub repository page: owner, name, description, website, topics, stars, forks, watchers, license, default branch, commits, releases, open issues and… Call with extractor_id=@extracthub1/github-repo@1.
- [Hacker News - stories list](https://extracthub.io/extractor/@extracthub1/hacker-news-stories): Stories from a Hacker News list page (front page, newest, ask, show, past): rank, id, title, link, points, author, time, comment count and the next page link. Call with extractor_id=@extracthub1/hacker-news-stories@1.
- [Google Play - app page](https://extracthub.io/extractor/@extracthub1/google-play-app): Details of a Google Play app page (play.google.com/store/apps/details?id=...): app id, name, developer, category, rating, exact rating count, downloads… Call with extractor_id=@extracthub1/google-play-app@1.
- [Google local results (places list)](https://extracthub.io/extractor/@extracthub1/google-local-results): Places from Google's local results page (google.com/search?q=<query>&tbm=lcl): name, rating, review count, category, price range, address or service area… Call with extractor_id=@extracthub1/google-local-results@1.
- [Google Maps - places search](https://extracthub.io/extractor/@extracthub1/google-maps-search): Places from a Google Maps search (www.google.com/maps/search/<query>): name, Google Maps place URL, place ID, coordinates, rating, review count, category… Call with extractor_id=@extracthub1/google-maps-search@1.
- [Quotes to Scrape - quote list page](https://extracthub.io/extractor/@extracthub1/quotes-toscrape-list): Every quote on a quotes.toscrape.com list page (text, author, author page URL, tags) plus the next-page URL. Call with extractor_id=@extracthub1/quotes-toscrape-list@1.
- [Books to Scrape](https://extracthub.io/extractor/@extracthub1/books-to-scrape): Example of an extractor to scrape and extract books from https://books.toscrape.com. This extractor was made with AI assistance using this prompt: Get all… Call with extractor_id=@extracthub1/books-to-scrape@1.