Generate an API client from the docs when there is no OpenAPI spec

You need to call an API. It has decent documentation, a README or a few reference pages with endpoints, parameters and example responses. What it doesn’t have is an OpenAPI spec. So you’ve got nothing to feed a client generator, and no real way of telling which parts of the docs you can actually rely on.

The usual answer is to write the client by hand, reading the docs as you go. That works, mostly. The trouble is that every assumption you make along the way ends up in the code, and nothing records which lines came from the docs and which came from your guesses. Six months later, neither do you.

I wanted to take a different route with this one. Before writing any client code, we turn the docs pages into a contract: an OpenAPI 3.1 file where every operation and field points back to the page it came from, and everything the docs didn’t say is marked as missing rather than filled in. Then we generate the client from that contract.

The tool is docs2contract, a small Python CLI I built for this. Tabstack reads the pages, and everything else runs locally.

Contract first, then the client

A client generated straight from docs (by a model, say) can look finished. It has types on every field and a function for every endpoint. What you can’t tell from the result is which of those types the docs actually stated, and which were filled in simply because a function signature needs a type.

A contract makes that difference visible, because it doesn’t guess. If the docs don’t say a field is an integer, neither does the contract. Each operation, parameter and field records the URL it came from, and whether it was described or only seen in an example. And when two pages disagree, the contract keeps both statements and leaves the value undefined instead of quietly picking one.

Once you have that, a client generator can take it from there, and the gaps show up as loose types instead of confident wrong ones.

The extraction schema, and why “not stated” is a value

Every page is read against the same fixed JSON schema. A stable shape is what lets results from different pages be merged later on. This is the part that describes a response field:

{
  "name": { "type": "string", "description": "Dotted path for nested fields, e.g. account.email" },
  "type": { "type": "string", "enum": ["string", "integer", "number", "boolean", "array", "object", "not_stated"] },
  "required": { "type": "string", "enum": ["yes", "no", "not_stated"] },
  "nullable": { "type": "string", "enum": ["yes", "no", "not_stated"] },
  "description": { "type": "string" },
  "evidence": { "type": "string", "enum": ["documented", "example_only"] }
}

The first choice that matters is that not_stated is a real value you can get back. “The docs say this is optional” and “the docs don’t say” are two different facts, and a boolean required can’t tell them apart.

It matters more than you’d expect, because of how extraction fills a schema. Tabstack’s schema-design guide says an optional field the page doesn’t fill comes back as null or doesn’t come back at all. A required field is worse: it gets a value anyway, so a required number comes back as 0 and a required boolean as false even when the page never mentions them. That’s why nothing in this schema is marked as required. A false you didn’t ask for looks exactly like a false the docs stated.

So docs2contract treats null, a missing key, an empty string and any value outside the enum as not_stated. It never turns them into false, an empty string or a guessed type. It doesn’t map synonyms either: "int" stays unknown rather than becoming "integer".

The other one is evidence, which separates a description from an example. A field that only appears in a sample response is marked example_only. An example shows you one instance. It tells you what type the value had that one time, but not whether the field is always there or whether it can be null.

Extracting the declared pages

You list the pages yourself. docs2contract doesn’t crawl and doesn’t follow links. For the Hacker News API, which is documented in a single README with no OpenAPI file, the list is one line:

echo "https://github.com/HackerNews/API" > urls.txt
uv run docs2contract extract --urls urls.txt --out runs/hn --nocache
uv run docs2contract merge runs/hn --out runs/hn/openapi.json

The repository ships the run this post walks through in runs/no-spec/. extract won’t write into a directory that already holds page artifacts, so give your own run a new one. The commands below use the recorded run.

extract makes one Tabstack /extract/json call per URL, and it doesn’t retry. Each call costs credits, and a retry is just another paid call. --nocache forces a fresh fetch, so the run manifest reflects what the page said today. The manifest also records the commit, a hash of the extraction schema and the tool versions, so you can trace a run later.

Extraction can vary from run to run, so a fresh run of this page may give different counts from the recorded one below. From that one page, the merge produced 10 operations, which matches the 10 endpoints the README documents: item, user, maxitem, six story lists and updates. The README gives its base URL as https://hacker-news.firebaseio.com/v0/ and writes paths like /v0/item/<id>.json. The contract normalises those to a server of https://hacker-news.firebaseio.com and paths like /v0/item/{id}.json.

Merging, conflicts and source URLs

The item operation from the generated contract looks like this, trimmed down:

{
  "operationId": "get_v0_item_id_json",
  "tags": ["Items"],
  "parameters": [
    {
      "name": "id",
      "in": "path",
      "required": true,
      "description": "The item's unique id.",
      "schema": {},
      "x-evidence": "documented",
      "x-source-url": "https://github.com/HackerNews/API",
      "x-docs-gap": ["type"]
    }
  ],
  "x-docs-gaps": ["success_status", "error_responses", "pagination"],
  "x-source-urls": ["https://github.com/HackerNews/API"]
}

"schema": {} with x-docs-gap: ["type"] is the contract saying “a type exists, and the docs didn’t tell me which one”. The operation-level x-docs-gaps says the page also didn’t state a success status, error responses or a pagination style.

That empty schema is actually a little unfair to the README. Its prose does say that items “are identified by their ids, which are unique integers”. The extraction just didn’t connect that sentence to the id parameter. And the same README says users are identified by case-sensitive ids, which are usernames, so the two id parameters have different types even though they share a name and a gap. So a gap in the report means the contract doesn’t know, and that isn’t always the same as the docs never saying it. Either way, the report is where you look before you rely on a field.

When an operation appears on more than one page, the pages are merged one attribute at a time. If one page says a parameter is required and another says it isn’t, the contract records both statements with their URLs and leaves required out. The Hacker News sample is a single page, so it has no conflicts. In the test fixtures, a conflict shows up in the gap report like this:

- parameter `expand`, `required`: `yes` (https://docs.example.test/a); `no` (https://docs.example.test/b)

That fixture is synthetic. It shows how the mechanism works, and it isn’t a result from a real API.

The gap report

uv run docs2contract gaps runs/no-spec/openapi.json --out runs/no-spec/gaps.md

The gap report is built from the contract alone, so the two can’t disagree, and it’s written for a person to read. For the Hacker News sample, the summary is:

Each of the 62 is one row in the report. A field or parameter gets one row, which can list several missing attributes. An operation gets one row per missing attribute, and the missing auth scheme gets a row of its own. Across them, the docs didn’t state:

• the type of 20 fields, and the nullability of 22;
• the type of 2 path parameters;
• the item shape of 2 array fields;
• a success status, error responses or a pagination style for any of the 10 operations;
• any response fields at all for 7 operations;
• an auth scheme.

The user operation is a good one to look at closely:

## `GET /v0/user/{id}.json`

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |
| parameter | path:id | type | https://github.com/HackerNews/API |
| field | id | type, nullable | https://github.com/HackerNews/API |
| field | created | type, nullable | https://github.com/HackerNews/API |
| field | karma | type, nullable | https://github.com/HackerNews/API |
| field | about | type, nullable | https://github.com/HackerNews/API |
| field | submitted | type, nullable | https://github.com/HackerNews/API |

Most of that is an accurate description of the page. The README’s field tables have two columns, “Field” and “Description”, so they don’t state that karma is a number. Required properties are marked in bold, and for users that’s id, created and karma. The extraction recorded exactly those three as required and the rest as not required, which is why required doesn’t appear in the gap list above.

None of this is a criticism of the README, which is short and easy to read. A contract just needs the things that prose usually leaves out.

The full report is in the appendix.

Validation

uv run docs2contract validate runs/no-spec/openapi.json
# runs/no-spec/openapi.json: valid OpenAPI 3.1

The contract is checked with openapi-spec-validator. If it fails, merge still writes it to disk and exits with code 2, so you can see what broke.

Some OpenAPI rules collide with “not stated”, and the contract records the collision rather than hiding it:

• OpenAPI requires every path parameter to be required: true. The contract sets it, and keeps what the docs actually said in x-docs-required-stated.
• OpenAPI tools treat a parameter with no required key as optional. When the docs didn’t say, the contract leaves the key out and adds x-docs-gap: [required]. Tools that ignore x- keys will see “optional”, so check the gap report before relying on one.
• The extraction schema doesn’t capture the response media type, so bodies are written as application/json and marked x-docs-assumed-media-type: true.

Generating the client and testing it against a mock

uv run docs2contract client runs/no-spec/openapi.json --out generated/client

This runs openapi-python-client on the contract, unmodified. Its documentation says it accepts OpenAPI 3.1, and on these contracts it did, so there was no need to convert to 3.0 first. This is what it generated for the item endpoint:

def sync_detailed(
    id: Any,
    *,
    client: AuthenticatedClient | Client,
) -> Response[GetV0ItemIdJsonResponseDefault]:

id: Any is the gap report showing up in code. The docs, as extracted, didn’t state a type, so the client doesn’t claim one. A guessed int would look tidier, but it would turn a gap in the docs into a confident assumption in your code. The Default in the return type is another gap. No success status was stated, so the response is written under default rather than 200.

The repository’s tests generate this client from the recorded contract, import it, and call it against a respx mock. No real API is involved. This is trimmed from tests/test_sample_contracts.py:

route = respx_mock.get("https://hacker-news.firebaseio.com/v0/item/8863.json").mock(
    return_value=httpx.Response(200, json={"id": 8863, "by": "dhouston", "type": "story"})
)
response = module.sync_detailed(8863, client=package.Client(base_url=HN))
assert route.call_count == 1
assert response.parsed.by == "dhouston"

That test proves the client sends the request the contract describes. It doesn’t prove the API behaves the way the docs say it does. The README’s example for the story lists is a JSON array of ids, but the extraction recorded no response fields for those endpoints, so the contract has no response body for them at all. The generated client returns parsed=None for those calls, and you’d have to read response.content yourself. Where fields were recorded, it assumes an object. A mock built from the contract can’t catch either case. Make one real call to each endpoint you depend on before you trust the client.

Two more things turned up while I was building this. openapi-python-client drops a response entirely if an array has no items schema, so the contract writes items: {} (“any item”) and flags it as a gap. It also turns an Authorization header parameter into a required function argument, even though OpenAPI says a parameter with that name should be ignored. So the contract moves those headers aside, and auth goes through the generated AuthenticatedClient instead.

The ground-truth score: what the docs gave, what the extraction missed

You can’t score the Hacker News sample, because there’s no official spec to compare it against. To see what extraction gets right and wrong, I used an API that has both docs and a spec: three GitHub REST API reference pages (licenses, gitignore and codes of conduct, at version 2022-11-28). I compared the result with GitHub’s official OpenAPI description, trimmed down to the 7 operations those three pages document.

There’s no overall percentage. Each category reports “matched / possible” on its own, because one number would hide where things went wrong. Parameter and field counts only cover operations that were found, which is why their denominators change between runs. I ran it twice on the same day, once at Tabstack’s default effort and once at max, which the docs recommend for JavaScript-heavy pages:

The result that matters most is one the table can’t show you. In each run, at least one page came back successfully with no endpoints at all, and it was a different page each time. Run 1 lost the licenses and codes-of-conduct pages. Run 2 lost the gitignore page, which had worked fine in run 1. Every one of those responses was an HTTP 200, so none of them counted as a failure. The gap report lists pages like that under “Retrieved URLs returning no endpoints”. With one run per setting, I can’t tell you whether --effort max helped. What it does show is that a single run isn’t enough to call a page empty.

Where a page did return endpoints, the result was close to the official spec. No parameter or type disagreed, and run 2 missed three response fields: body, html_url and license.html_url. In run 2, every response field came from example JSON rather than a described schema, so the extraction marked them all example_only and listed their required flags as gaps. I think that’s the right call. An example tells you what one response looked like, not which fields are always going to be there.

This is two runs against three pages, and it says nothing about extraction on docs in general.

When to write the spec by hand instead

This approach earns its keep when the docs are the only description you have, and you want a record of what they did and didn’t say before you build on them. I wouldn’t reach for it when:

• the API is large and you need all of it, because you have to declare every page and every page is a paid call;
• the real contract lives in behaviour, not docs. If the docs are thin or out of date, extracting them faithfully gets you a faithful copy of thin docs;
• you own the API. Then write the spec where the code is, and generate the docs from it.

In the middle ground, use the contract as a starting point. Take the gap report and fill the gaps from real responses or from the API’s maintainers, writing each answer into the spec as you go. The gaps are your to-do list.

Try it on one page

Pick one docs page from an API you use and run it through:

git clone https://github.com/Mozilla-Ocho/docs2contract
cd docs2contract
uv sync --frozen
export TABSTACK_API_KEY=...
echo "https://..." > urls.txt
uv run docs2contract extract --urls urls.txt --out runs/mine
uv run docs2contract merge runs/mine --out runs/mine/openapi.json
uv run docs2contract gaps runs/mine/openapi.json --out runs/mine/gaps.md

Then read gaps.md before anything else, because that’s where the page’s limits are written down.

Tabstack: Never trained on. Private by default. Built by Mozilla. See the Trust page.

Appendix: the full gap report for the Hacker News sample

This is runs/no-spec/gaps.md exactly as generated.

# Gap report

What the declared docs pages did not say. Every item here is left undefined in the contract. Nothing was guessed.

| Item | Count |
|---|---:|
| Operations | 10 |
| Gap items | 62 |
| Seen only in examples | 2 |
| Conflicts | 0 |
| Declared URLs not retrieved | 0 |
| Retrieved URLs returning no endpoints | 0 |

## Document

- `auth`: No auth scheme stated. No `securitySchemes` entry was emitted.

## `GET /v0/askstories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/beststories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/item/{id}.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |
| parameter | path:id | type | https://github.com/HackerNews/API |
| field | id | type, nullable | https://github.com/HackerNews/API |
| field | deleted | type, nullable | https://github.com/HackerNews/API |
| field | type | type, nullable | https://github.com/HackerNews/API |
| field | by | type, nullable | https://github.com/HackerNews/API |
| field | time | type, nullable | https://github.com/HackerNews/API |
| field | text | type, nullable | https://github.com/HackerNews/API |
| field | dead | type, nullable | https://github.com/HackerNews/API |
| field | parent | type, nullable | https://github.com/HackerNews/API |
| field | poll | type, nullable | https://github.com/HackerNews/API |
| field | kids | type, nullable | https://github.com/HackerNews/API |
| field | url | type, nullable | https://github.com/HackerNews/API |
| field | score | type, nullable | https://github.com/HackerNews/API |
| field | title | type, nullable | https://github.com/HackerNews/API |
| field | parts | type, nullable | https://github.com/HackerNews/API |
| field | descendants | type, nullable | https://github.com/HackerNews/API |

## `GET /v0/jobstories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/maxitem.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/newstories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/showstories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/topstories.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | response_fields | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |

## `GET /v0/updates.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |
| field | items | nullable, items | https://github.com/HackerNews/API |
| field | profiles | nullable, items | https://github.com/HackerNews/API |

Seen only in examples (one instance, not a contract):

- `items` (https://github.com/HackerNews/API)
- `profiles` (https://github.com/HackerNews/API)

## `GET /v0/user/{id}.json`

Sources: https://github.com/HackerNews/API

| Kind | Name | Not stated | Source |
|---|---|---|---|
| operation | - | success_status | - |
| operation | - | error_responses | - |
| operation | - | pagination | - |
| parameter | path:id | type | https://github.com/HackerNews/API |
| field | id | type, nullable | https://github.com/HackerNews/API |
| field | created | type, nullable | https://github.com/HackerNews/API |
| field | karma | type, nullable | https://github.com/HackerNews/API |
| field | about | type, nullable | https://github.com/HackerNews/API |
| field | submitted | type, nullable | https://github.com/HackerNews/API |