ExtractHub MCP server: a web scraper for AI agents

Set up your client

claude mcp add --transport http extracthub https://api.extracthub.io/mcp --header "Authorization: Bearer YOUR_EXTRACTHUB_API_KEY"

{
  "mcpServers": {
    "extracthub": {
      "url": "https://api.extracthub.io/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_EXTRACTHUB_API_KEY"
      }
    }
  }
}

{
  "mcpServers": {
    "extracthub": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://api.extracthub.io/mcp",
        "--header",
        "Authorization:${EXTRACTHUB_AUTH}"
      ],
      "env": {
        "EXTRACTHUB_AUTH": "Bearer YOUR_EXTRACTHUB_API_KEY"
      }
    }
  }
}

URL: https://api.extracthub.io/mcp
Transport: Streamable HTTP
Header: Authorization: Bearer YOUR_EXTRACTHUB_API_KEY   (or X-API-Key: YOUR_EXTRACTHUB_API_KEY)

Replace YOUR_EXTRACTHUB_API_KEY with your key. Keep keys out of shared repositories: prefer user-level configuration or an environment variable.

Tools

A typical session: search_extractors for the site, get_extractor to check the output schema and price, then extract with the pinned id (for example @user/slug@1) or create_batch for many URLs. The server also offers an extract_from_url prompt that walks the agent through these steps.

Build extractors from your agent

No extractor for a site yet? An agent can write one and ship it without opening the Studio: validate_extractor_config checks the config and, with the page HTML as sample_html, runs it locally with the real extraction engine (nothing is fetched, no credit). create_extractor saves it as a private draft, extract with the @user/slug@draft pin tests it on real pages, update_extractor fixes it, and release_extractor releases version 1.0.0 exactly like the Studio (later versions follow the output schema change). The release sample can also come from the stored result of one of your own jobs or batch items (sample_from_job_id / sample_from_batch_item), and set_sample_output later replaces a released version's sample with a real result of that exact version. Pages over the 2 MB sample_html limit go in as sample_raw_html_url (the link from extract with raw_html: "link"), and set_preview_page replaces a released version's preview page (and its sample, from the same page) without a new release. delete_extractor removes a private draft that was never released (failed attempts, probes); released extractors are never deleted by an agent. Creating, updating, releasing and deleting are free but need a live API key; a test key can only validate. The create_extractor_for_site prompt walks the agent through these steps. For the best results install the open-source extractor builder skill: it teaches the agent how to inspect a page, pick a fetch mode and write stable selectors before it calls these tools.

Multi-step scrapes: pipelines

For a list page and then every detail page it links to, the agent builds a pipeline: a chain of pinned extractors where each step feeds the next through one of its output fields (url_output_field, e.g. books[].url). validate_pipeline checks the chain and estimates the cost without saving; create_pipeline saves it (it opens and edits in the dashboard pipeline builder); run_pipeline starts a run, billed exactly like a run from the dashboard; get_pipeline_job shows progress per step and, when it is done, the first records and a download link for the full dataset. Saving, updating, running and cancelling need a live API key. Schedules are saved off unless the agent explicitly turns them on, which it should only do with your OK: scheduled runs keep spending credit.

What to ask your agent

Once the server is connected, describe the data you want in plain language. The agent picks the tools; a few examples:

• “Find an ExtractHub extractor for Amazon product pages and show me which fields it returns.”

  Tools: search_extractors → get_extractor

• “Extract the title, price and rating from https://www.example.com/product/123.”

  Tools: search_extractors → extract (→ get_job if the queue is busy)

• “Run these 200 product URLs as a batch, then summarise the results once it is done.”

  Tools: get_account → create_batch → get_batch → get_batch_results

• “How much ExtractHub credit do I have left, and is this a test key?”

  Tools: get_account

• “There is no extractor for this site yet: build one for https://www.example.com/product/123 and release it.”

  Tools: validate_extractor_config → create_extractor → extract (@draft) → update_extractor → release_extractor

• “Scrape every book of this category page, then each book's detail page, and give me one dataset.”

  Tools: validate_pipeline → create_pipeline → run_pipeline → get_pipeline_job

Billing and limits

• extract and create_batch use the same prepaid credit and prices as the API: a URL is charged when it was scraped (also when the call then fails after the provider fetched the page); failures before that are refunded. Catalog, account, status and cancel tools are free. run_pipeline charges every step of the run the same way, capped by the pipeline's maximum cost per run.
• MCP calls count toward your account's per-minute rate limit and wait in the same fair queue as API requests. A busy queue turns extract into an async job; the agent then polls get_job.
• Large results are shortened in the tool answer (about 64 KB) with a note; batch results stay complete at the batch results URL. extract with raw_html: "link" returns a signed download link (valid 1 hour, plain text) to the full page HTML instead of a shortened copy, at no extra cost.
• The free tools (search_extractors, get_extractor, get_account and the extractor authoring tools) also count toward the per-minute limit and show up in your request logs with a cost of 0, so a looping agent cannot flood the API. release_extractor with fetch_sample runs one normal, billed extract.
• Errors come back as tool errors with the API error code and a next step (for example a top-up link for insufficient_balance).

FAQ