Versioned web scrapers: why your scrapers need semantic versioning

A scraper that you call from production code is an API, whether you planned it that way or not. Somebody depends on the field names it returns, on price being a number, on products being a list. A versioned web scraper makes that contract explicit: every release is an immutable version with an output schema, and callers decide which changes reach them.

This post explains how ExtractHub versions its extractors (hosted, versioned scrapers): how the semver bump is computed from the output schema, what a caller pins, how deprecation and yanking work, and the trade-offs of each pin. If you have not built an extractor yet, start with how to build a web scraper with Claude Code.

Two kinds of change break a scraper

A scraper’s output changes for two unrelated reasons:

1. The target site changes. A new layout, a renamed class, a block that moved. The scraper owner did nothing, the output got worse. That is a monitoring problem: you detect it with health checks and fix it.
2. The scraper owner changes the output. A field is renamed, a price string becomes a number, a single value becomes a list. Each of those is a reasonable improvement, and each breaks a caller that parsed the old shape.

Most homemade scrapers handle neither. The second one is the subject of this post, and it is the one versioning solves: a fix for the first kind should reach callers at once, a change of the second kind should never reach them without their consent.

What a versioned web scraper looks like

In ExtractHub the editable working copy of an extractor is its draft. Saving the draft, in Extractor Studio or with the update_extractor MCP tool, changes nothing for callers. Releasing the draft takes a snapshot that is never edited afterwards:

• the request options (URL, method, proxy and country, JavaScript rendering, waits, browser actions),
• the extractor config (fields, selectors, transforms),
• the output schema derived from that config,
• a changelog (required for a new major) and the sample output of the release check.

Each release gets a semver number. The first release is 1.0.0. Callers address a release with a pin on the extractor id, @owner/slug@<pin>, and every call must carry one.

Because the request options are part of the snapshot, the version also decides how the page is fetched, and therefore the price tier of each call (plain fetch, proxy or JavaScript; see pricing).

Semantic versioning for scrapers: the bump comes from the output schema

The owner does not pick the version number freely. When you release, the draft’s output schema is compared with the latest release of the same major line, and the diff sets the minimum bump:

Change to the output schema	Minimum bump	Illustrative example
None (only selectors or request options changed)	patch	a fixed selector for title
Only additions	minor	a new rating field
A field’s type is now known (it was not before)	minor	a value read from embedded JSON, typed on the release sample
A default value changed	minor	inStock defaults to false
A field removed or renamed	major	sku removed, title renamed to name
A type changed	major	price string becomes price number
A scalar became a list, or the reverse	major	image becomes images[]
A nested structure changed	major	seller string becomes an object
A field became required, or stopped being required	major	price marked required: true

Two rows deserve a comment.

A rename is a removal plus an addition. The diff has no way to know that name is the new title, and a caller reading title breaks either way. So pick names carefully before the first release.

required changes are major in both directions. In ExtractHub, an empty top-level required field fails the request with 422 required_fields_missing, and a list item whose required sub-field is empty is dropped from the list. Making a field required therefore turns some former successes into failures and removes items; making it optional lets empty values back in where callers relied on a value. Neither is safe to ship to callers who did not ask for it.

The owner can raise the computed bump (release a schema-identical change as a minor version to signal a behaviour change, for instance) but never lower it: a release request below the minimum fails with bump_too_low. A new major after the first release needs a changelog (changelog_required), and releasing a draft identical to the latest version is refused as unchanged.

What the owner sees before releasing is the diff itself. One entry, as an illustrative example:

{
  "path": "products[].price",
  "change": "changed",
  "breaking": true,
  "before": "string",
  "after": "number",
  "reason": "type changed"
}

In Studio the release dialog shows this diff and the next version number. Over the MCP server, release_extractor with dry_run: true returns the same plan without releasing anything.

How to pin a scraper version: @1, @1.2.3 or @draft

Every call names a pin. There are three forms:

Pin	Resolves to	Who can use it
@you/example-product@1	the latest non-yanked release in major 1	anyone who can see the extractor
@you/example-product@1.2.3	exactly that release	anyone who can see the extractor
@you/example-product@draft	the current draft	the owner only

A realtime call pinned to a major:

curl -X POST "https://api.extracthub.io/api/extractor/extract" \
  -H "X-API-Key: $EXTRACTHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "extractor_id": "@you/example-product@1",
    "target": { "url": "https://www.example.com/p/123" }
  }'

The X-Extractor-Version response header tells you which release served the call, for example 1.2.3 (or draft). Log it next to the data: when a value looks wrong, you know which selectors produced it.

An id without a pin is rejected with a 400 that lists what you can pin, so a missing pin fails at the first test instead of silently following every future change. An illustrative example of the body:

{
  "success": false,
  "error": "A version pin is required. Available majors: @you/example-product@1, @you/example-product@2",
  "details": { "code": "pin_required", "majors": [1, 2] }
}

Other pin errors use the same shape: invalid_pin (400), draft_owner_only (403), version_not_found (404) and version_yanked (410).

Which pin to choose

• Major pin (@1) for most integrations. You receive selector fixes the day they are released, and new fields appear without a code change. You never receive a removed field, a type change or a new required rule. One thing to keep in mind: a patch may change request options, such as turning on JavaScript rendering for a site that started needing it, and that can change the price tier of your calls.
• Exact pin (@1.2.3) when you need everything frozen: the selectors, the fetch mode and the tier. The cost is that fixes never reach you; when the site changes, the exact version stays broken until you move your pin.
• @draft while you build or debug your own extractor. Do not ship it: the draft changes every time you save.

The same pins work everywhere the extractor runs: realtime GET and POST extract, the Batch API, async jobs, MCP and pipelines. A pipeline node stores its pin, and each step run records the version that actually ran.

Fixing an older major after a breaking release

Releasing 2.0.0 does not retire major 1. Callers on @1 keep running 1.x, and the owner can still release fixes into that line: in the Studio release dialog by choosing the major line, or with release_extractor and base_major: 1 over MCP. The fix is diffed against the latest 1.x release and becomes 1.x.y+1 or 1.x+1.0 by the same rules. Only a new major moves to the next free major number.

This is what makes a breaking change cheap to ship. You can improve the output for new callers today and keep the old line healthy until its callers have moved.

Deprecating a major: Deprecation and Sunset headers

When an old major should go away, the owner deprecates it in Studio with a sunset date at least 30 days out and an optional message. From then on, every response of that major carries:

X-Extractor-Version: 1.4.2
Deprecation: true
Sunset: Mon, 14 Dec 2026 00:00:00 GMT
X-Deprecation-Message: Prices are numbers in v2, see the changelog

The Sunset header follows RFC 8594. Calls keep working after the sunset date; nothing is switched off automatically, so a missed migration degrades into a warning, not an outage. The MCP extract tool surfaces the same information as a deprecation object in its result, so an agent caller sees it too.

Users who saved the extractor, and owners of pipelines pinned to the deprecated major, get an in-app notification and an email (unless they turned extractor emails off). A new major is announced the same way. New minor and patch versions are not announced: by design they need no action.

On the caller side, treat a Deprecation header like a failing test that has not failed yet: log it, alert on it, and read the changelog of the next major before the sunset date. The owner can also undo a deprecation, which clears the sunset date and the message.

Yanking a version that should never have shipped

Deprecation is for an old line that still works. Yanking is for one release that is wrong: it leaks data it should not, or returns values that are worse than nothing. The owner yanks an exact version in Studio with a required reason; it cannot be undone there.

After a yank:

• @1 resolves to the latest remaining release of major 1, skipping the yanked one. Callers on the major pin move back to the previous good version on their next call.
• @1.2.3 (the yanked version) answers 410 version_yanked with the reason and, when the major still has a release, a replacement pin such as @you/example-product@1.
• If the yanked release was the last one of its major, the major has nothing left to serve and @1 stops resolving too.

Users who saved the extractor and owners of pipelines pinned to the yanked version (or to its whole major, when it was the last release) are notified with the reason.

Since releases are immutable, a yank does not edit history. It removes one version from resolution and tells the people affected why.

Versioning and health checks work together

Versioning protects callers from changes the owner makes. It does nothing against changes the site makes, which is why released versions are also health checked (public extractors on a schedule, private ones when the owner turns checks on): a probe on the version’s sample page flags a version whose required fields come back empty. The fix is a selector change with the same output schema, which is a patch, which reaches every @1 caller without any action on their side. That loop, detect, patch, deliver to the major pin, is the main reason to pin majors rather than exact versions.

You can see the versions, pins, deprecation state and output schema of any public extractor in the extractor catalog, or ask for them over MCP with get_extractor. The full list of endpoints and error codes is in the API reference.

A scraper versioning checklist

Whether or not you use ExtractHub, these rules make a scraper safe to depend on:

1. Release immutable versions that include the fetch options, not only the selectors.
2. Derive the version number from the output, not from how much work the change was.
3. Refuse unpinned calls so no caller follows breaking changes by accident.
4. Report the version that ran on every response.
5. Keep old majors fixable after a breaking release.
6. Deprecate with a date and a machine-readable signal, and do not switch anything off at the date.
7. Yank with a reason, and point callers at a replacement.