Road to 11.0.0
Package: @lgriffin/esi.ts · Plan revision: 1 (2026-09-27, measured at v10.2.3, master 519c945) · Status: complete; kept as the record of how 11.0 was built
Historical. 11.0.0 shipped on 2026-09-28 and 12.0.0 followed in October 2026; every phase below is done except #270 in Phase 6, which needs repository settings only the maintainer can change. Work after 11.0 is tracked in GitHub issues and Beads, not here.
The schedule for the next major. CHARTER.md says what the project holds itself to; this document says in what order the remaining work lands, what each phase must prove before it is called done, and what has to be true before the 11.0.0 release pull request merges. It is written so that a fresh session can pick up any phase from this file alone, without the conversations that produced it.
Precedence: the running code and CI are the fact, the charter is the intent, this roadmap is the order. When this file and a workflow disagree, fix the file in the same pull request.
How to read this
- A phase is a unit of the release: Phases 0 to 7 in the order the plan set on 2026-09-26, plus Track S, the SDE programme, which runs alongside them on its own cadence.
- Every phase has a scope (what is in and what is not), an order (the pull requests it lands, each mergeable on its own), a definition of done that a reviewer can check from CI and the repository rather than from memory, and the tracking that already exists for it. GitHub issues carry
release::11.0and onephase::*label (Phases 0 and 1 closed before the labels existed); beads mirror them. - Done and Open rows cite evidence: a merged pull request, a file, a check name. A row with no evidence is a claim to verify before relying on it.
- The work follows the rules every other change follows: SEMVER.md before every commit, the
ears-gherkin-devskill for every behaviour change, one concern per pull request, no floor lowered, nothing lands onmasterexcept through a pull request that passesci-success.
Where 11.0.0 stands
- The release pull request is already open. release-please opened #405 (
chore(master): release 11.0.0) on 2026-09-26 because four breakingfix(schemas)!:andfix(core)!:commits (the 2026-08-18 compatibility date and the schemas it needed) are on master. It stays open and keeps accumulating until the release gate at the end of this document is met. Merging it earlier ships 11.0.0 without the Node 22 floor, the new client, or the SDE work; that decision belongs to the maintainer, not to a session. - Decisions recorded (CHARTER, 2026-09-26): the target is 11.0.0, not 1.0.0; Node 22 becomes the floor as a declared breaking change (REL-05); pino and zod stay runtime dependencies; Jest, npm, release-please and Dependabot stay the toolchain.
- Decided 2026-09-27: no bridge between the core and the SDE inside this package. A
StaticDataResolverport was considered and cut; an ESI-to-SDE shim, if it ever exists, is a separate package above both. Nothing in this repository is added for it. - Release blockers outside the phases (
phase::now): #383 and #378, the release-please token problem, closed by #468 and #474: release-please's tag now triggersrelease.ymland the canary without a manual dispatch (gap register row 30). What is left outside the phases needs the maintainer and follows the release in 11.x (decided 2026-09-28): branch protection including administrators (#270), the npm trusted publisher (#548), and theESI_HEALTH_*secrets for the live-health nightly (#545;nightly-live-health.ymlruns as "not configured" until they exist).
Phase map
| Phase | Name | Label | State | Evidence |
|---|---|---|---|---|
| 0 | Audit and generator decision | — | Done | AUDIT.md, docs/spikes/generator/, #404 |
| 1 | Generator | — | Done | src/generated/operations.generated.ts, spec:generate:check and spec:coverage in CI, #411 |
| 2 | Architecture lock | phase::2-architecture | Done | PRs 4 to 12 merged (#472, PR 12 below) |
| 3 | Types and errors | phase::3-types | Done | #470, #471, #473, #480, #482; #295, #266, #298 closed |
| 4 | Logging and import-time behaviour | phase::4-logging | Done | #481, #486; #296, #268, #265 closed |
| 5 | Testing and CI alignment | phase::5-testing | Done | #297, #271, #277, #382, #380 |
| 6 | Security and supply chain | phase::6-security | Done¹ | #239, #270, #256 |
| 7 | Release, Node 22 and the docs rewrite | phase::7-release-docs | Done | #264, #272, #273, #278 |
| S | SDE programme (twelve runs) | phase::s-sde | Done | Runs 1 to 12 merged, the last #527; Track S |
¹ Phase 6 is done except #270 (branch protection for administrators), which is the maintainer's.
Ordering between phases: 2 before 3 (the builder is what Phase 3 moves ClientRegistry behind); 3 before 4 (typed errors first, then the logger that reports them); 5 and 6 are independent of 3 and 4 and may interleave; 7 is last because it documents the finished API and carries the feat!: Node floor bump that turns the release pull request into the real 11.0.0. Track S has no dependency on Phases 2 to 7 and takes one night a week throughout.
Phase 0 · Audit and generator decision · Done
Scope. A measured snapshot of what is hand-written, what is generated, which ESI operations the client covers, and where each gate stands; a decision between an in-repo operation emitter and templating over @hey-api/openapi-ts.
Evidence. AUDIT.md (code inventory, operation coverage, mutation baseline, plan gates, transport rules). docs/spikes/generator/SPIKE.md recommends the in-repo emitter; five spike operations compile under strict, exactOptionalPropertyTypes, noUncheckedIndexedAccess and isolatedDeclarations.
Still owed by this phase: nothing. The numbers in AUDIT.md are re-run, not edited, when a later phase changes them (the commands are listed with each table).
Phase 1 · Generator · Done
Scope. One typed function and one *Meta constant per spec operation, emitted from the vendored OpenAPI document, checked in CI.
Evidence. npm run spec:generate writes src/generated/operations.generated.ts (233 operations at compatibility date 2026-08-18). spec:generate:check fails when the file is stale; spec:coverage fails when a spec operation lacks exactly one Meta. spec-refresh.yml re-vendors the snapshot on a GitHub runner. Merged in #411, #412.
Phase 2 · Architecture lock · Done
Scope. The seams the new client stands on: ports, the generated operations wired to the request pipeline, the layer rule, a scope tree, and finally a builder that hands out a public view and per-identity views over one shared runtime. Nothing in this phase removes a legacy surface; EsiClient, CustomEsiClient and EsiApiFactory keep working unchanged, and the old and new trees are kept apart by lint:layers.
Landed (each is a merged pull request on master):
| PR | What | Merged |
|---|---|---|
| 4 | npm run lint:layers: ports import nothing, generated code imports only ports, core imports nothing above it. Baseline of two files, shrink-only. | #418 |
| 5 | The six ports in src/core/ports/ (CacheStore, Clock, HttpTransport, Logger, OperationTransport, TokenProvider) and systemClock. | #421 |
| 6 | Per-client tenant (X-Tenant) and userAgent options, validated in the constructor and the setters. | #422 |
| 7 | meta.getOpenApiYaml() through the pipeline; textResponse endpoints; no direct fetch left in a client. | #439 |
| 8 | Dropped: the beta-route overlay is unnecessary since the 2026-08-18 spec carries every beta route. | noted in #442 |
| 9 | src/adapters/PipelineTransport implements OperationTransport over the existing pipeline; generated operations share budgets and cache. | #442 |
| 10 | createScopeTree(transport) and PublicScopeTree: operations arranged by path prefix (tree.character(id).wallet.journal.get()). | #445 |
Remaining, in order. Each is one pull request; the design is the one written for #432 (shared runtime, Identity, cache keyed by character id).
- PR 10b · Cache and dedupe keyed by identity. Done:
src/core/util/callerIdentity.tskeys authenticated cache and deduplication entries bycharacter:<id>from an SSO token'ssubclaim once ESI has accepted the token (a 2xx or 304), else by the header hash; the character's ETag is sent asIf-None-Matchbefore acceptance so a refreshed token's first revalidation is a 304 hit while a forged claim earns a 401. Five Rules under 0050-etag-caching and 0051-resilience. Before it, the ETag cache key hashed the bearer token, so a token refresh emptied a character's cache. Key the cache and the in-flight deduplicator by a stable identity (character id where a token carries one, else the token hash), so ETags survive a refresh and two identities never share an entry. EARS rules intests/bdd/features/core/under the caching feature; scenarios at the transport seam that refresh a token mid-sequence and assert a 304 hit. Behaviour-compatible for callers that never refresh; classify asfix:orfeat:per SEMVER.md. - PR 11 · The builder and the shared runtime. Done:
@lgriffin/esi.ts/client(src/client/) exportscreateEsi, the ports,ScopeTreeandPublicScopeTree.createEsibuilds one runtime throughconfigureApiClientand refuses to construct without a legal user agent (decision (e) in #422);esi.publicis aPublicScopeTreewhose transport also rejects a scoped operation at runtime withNO_AUTH_TOKEN(closes #183);esi.as(identity)is an immutableScopeTreeover the same runtime for anIdentityfromEsiTokenManager.identity(id),identityFromTokenoridentityFromProvider;esi.shutdown()releases the runtime's timers. Thirteen Rules in0056-shared-runtime.feature;guides/MULTI-CHARACTER.md,examples/multi-character.ts,examples/public-vs-authenticated.tsand the README section (#432);tests/typetests/client.test-d.ts; the consumer contract and size budget cover the sub-path.feat:. - PR 12 · Mock transport and quickstart. Done:
createMockTransport()in./testingimplementsHttpTransportfor consumers' tests: a route table keyed by method and ESI path template (or a regular expression),timesto retire a route, a record of every request's method, URL, headers and body,unroutedfor the requests no route answered (rejected with anEsiConfigurationError, which the pipeline now passes through unchanged from any transport: one Rule in0051-resilience.feature), andreset(). Five Rules in0057-mock-transport.feature;tests/tdd/testing/mockTransport.test.ts;tests/typetests/testing.test-d.ts; the consumer contract exercises it; the README's "Testing your application without ESI" block runs anas()call against it undertest:docs-examples; TESTING.md "Testing your application".feat:.
Definition of done.
- The three pull requests above are merged; #183 and #432 closed by them.
tests/typetests/has a test-d file for the new entry asserting that an authenticated operation on the public view is a type error and thatas()returns a view with it.npm run test:consumerexercises the new sub-path (the consumer contract fails on an unexercisedexportsentry, so this is automatic once the sub-path exists).- The CHARTER's Part 2 layer table gains a row for the new tree with status Enforced by
lint:layers; ARCH-08 (construction parity) names the builder. - No entry is added to
BASELINEinconfig/eslint/layers.rules.cjs.
Phase 3 · Types and errors · Done
Scope. Everything a consumer sees in a type or a catch block. Three threads, all additive in a minor and completed in the major only where a removal is unavoidable and deprecated first.
- Layer baseline to empty.
src/core/ClientRegistry.ts(imports every domain client) andsrc/core/configureApiClient.ts(importsEsiClientConfig) move out of core or the type moves in, and theirBASELINEentries are deleted.tests/tdd/layers/layers-lint.test.tsfails if an entry stops violating the rule, so the deletion is forced.refactor:; API report unchanged. - Error fidelity (#295, #266; ARCH-07).
EsiValidationErroris never retryable; network faults get a typed class instead ofEsiErrorwith status 0; safe mode and token refresh stop collapsing distinct failures into one type;isCircuitOpenis exported from./errors; string-typed plumbing errors become anEsiConfigurationErrorfamily. New classes extend the existing ones soinstanceof EsiErrorkeeps working: additive,feat:. Any change to what an existing class carries or when it is thrown is classified per SEMVER.md and, if breaking, held for the samefeat!:commit as the Node floor. responseSchemastructural (#298; DES-02). Every endpoint definition that returns a body declares a schema. The generator (scripts/spec/spec-generate-core.ts) or a test derived from it fails when a definition lacks one, so the rule is a check, not a review comment. Where a schema is added for a route that had none, response validation begins for that route: an additive tightening, documented in the changelog under Changed with the route names.- Compiler flags.
exactOptionalPropertyTypesis on forsrc/(tests and scripts keep it off intsconfig.test.jsonand thets-nodeblock).isolatedDeclarationsis checked over the layers 11.0 exposes (the pipeline, ports, generated operations, client runtime, auth and the entry points) bynpm run typecheck:isolated(tsconfig.isolated.json,scripts/package/isolated-declarations.ts), inci-fast.ymlandci.yml. It is not applied to the hand-written Zod schemas or the endpoint maps: their exported types are inferred by design (a schema's type is the schema; an endpoint map's per-key literal types drive every client method's return type), and the flag would have wanted 366 hand-written schema types and flattened the maps. The maintainer chose this scoped check on 2026-09-27 over dropping the flag or annotating everything. Consumers are unaffected by the flags themselves; any type that changes shape as a result is classified.
Landed: thread 1, layer baseline to empty: ClientRegistry.ts moved to src/clients/, EsiClientConfig (and EsiDatasource) to src/core/EsiClientConfig.ts with EsiClient.ts re-exporting both, BASELINE is {} and the layers test fails if an entry is added (refactor:, API report unchanged). Thread 2, error fidelity: EsiNetworkError, CircuitOpenError and the EsiFaultError family (EsiConfigurationError, EsiParseError, EsiPaginationError, EsiTokenRefreshError) extend EsiError; EsiValidationError and faults are not retryable; every guard is on ./errors (breaking, in the 11.0.0 release pull request). Thread 3, responseSchema structural: npm run spec:response-schemas fails when an endpoint returning a JSON body declares no schema (#471). Thread 4, exactOptionalPropertyTypes on (#473); isolatedDeclarations checked over the exposed layers by typecheck:isolated.
Definition of done. BASELINE is {}; the four ERRORS.md sections for the new classes exist with a scenario each in tests/bdd/features/core/0051-resilience.feature or the errors feature; npm run ears gives every new requirement a verdict; the responseSchema check runs in ci.yml lint-and-build; tsconfig.json carries exactOptionalPropertyTypes and typecheck:isolated holds in CI; etc/esi.ts.api.md regenerated and the api-semver job green with no undeclared loss; #295, #266, #298 closed.
Phase 4 · Logging and import-time behaviour · Done
Scope. What the package does before the first request and what it writes about each one.
- Sanitise at the logger (#296; SEC-02, ARCH-09). Every log line that carries a URL passes through
sanitizeUrlat the logger boundary, not at each call site; the call sites that bypass the per-client logger (resolveLogger) migrate to it. Verified by a scenario that logs a request carrying a token in the query and asserts the redacted form at the logger seam. - Lint gate (#265).
npm run lintforbidsloggerUtilglobal imports insidesrc/core/requestPipeline(an ESLintno-restricted-importsblock next to the layer rule, run with--no-inline-config). - Lazy logger and
sideEffects: false(#268; ARCH-06). No pino instance is built at import time;package.jsondeclares"sideEffects": false;npm run sizebudgets fall or hold, and a new test imports the root entry with pino mocked and asserts it was not constructed.
Landed: items 1 and 2 (#481): sanitizeUrl runs at the logger boundary, the rate limiter, cache, batch and token manager log through the per-client helpers, and a no-restricted-imports block in npm run lint and lint:layers keeps the global loggerUtil out of src/core/requestPipeline/ and src/clients/; ARCH-09 and SEC-02 at Enforced. Item 3 (#486): the default logger builds its pino instance on first use, package.json declares "sideEffects": false, tests/tdd/core/importSideEffects.test.ts holds both and no .size-limit.cjs budget was raised; ARCH-06 at Enforced. LOGGING.md's "Call sites" section names the per-client helpers and the lint as the mechanism.
Definition of done. The three issues closed; .size-limit.cjs budgets not raised for this phase; LOGGING.md's "call sites" section rewritten to say every site uses the per-client logger, with the lint named as the mechanism; CHARTER ARCH-06, ARCH-09 and SEC-02 at Enforced.
Phase 5 · Testing and CI alignment · Open
Scope. The gates the charter promises exist in ci.yml and the nightlies, and the ones that flake are made deterministic.
- CI matches the gate matrix (#297; GATE-01..06, REL-02). Release gate runs the validators the charter lists; never-run validators (
validate:spec,validate:versionswhere they are missing) are wired or deleted; the dead docs job goes; the generated-freshness diff runs where Part 5 says. QUALITY-GATES.md's matrix is edited in the same pull request so the YAML and the table say the same thing. - Lint tests, block knip, incremental Stryker (#271; TEST-09, GATE-04, TEST-07).
npm run lintcoverstests/; knip blocks at release; the PR mutation gate stays (already done) and its stale-survivor reuse (#380) is fixed so a strengthened test can raise a floor from a pull request. - Nightlies file issues (#277; GATE-05). Mutation and Schemathesis open or comment on one fixed-title issue on failure and close it on the next green night, the pattern
nightly-spec-drift.ymlalready uses. Done: the no-retry, interleaving and consumer-matrix nightlies followed on 2026-09-28, and GATE-05 is Enforced. - Deterministic mutation (#382). The 43 mutants that flip between nightlies on unchanged code move to fake timers;
detectedByEveryRunis then no longer the only way to seed a floor. - Live health. AUDIT.md's one "Not met" gate: a nightly smoke against Tranquility that calls one authenticated route with a maintained token, opening an issue on failure. Needs a repository secret; the executor writes the workflow and the maintainer adds the secret. Landed 2026-09-28:
nightly-live-health.ymlrunsnpm run health:live(scripts/quality/live-health.ts), which refreshes the token withEveSsoClientand calls/statusand/characters/{id}/onlinethrough the realEsiClient; one fixed-title issue while it fails, a warning on an outage, and "not configured" until the maintainer addsESI_HEALTH_CLIENT_ID,ESI_HEALTH_REFRESH_TOKENand, for a confidential app,ESI_HEALTH_CLIENT_SECRET. AUDIT.md's gate reads Met. - The exclusion register (TEST-11).
npm run earsgains a section listing every unwanted-behaviour Rule whose response is negated (shall not), with its scenarios and verdicts, in the step summary andreports/ears/; TESTING.md's standalone EARS section documents it;tests/bdd/GUIDE.mdgets a worked example of writing an exclusion as a Rule. Each later phase that touches an area writes that area's exclusions as Rules (Phase 3 the responses it does not validate, Phase 4 the log fields it does not emit, Phase 6 the headers it does not honour); this item builds the register, not the Rules. - The charter audit (PROC-06).
scripts/quality/charter-audit.ts(pure logic incharter-audit-core.ts, tested undertests/tdd/scripts/) parses every####requirement block ofguides/CHARTER.mdwith the same rulesspec:auditapplies to aRule:(oneshall, a named system, one of the five patterns, the banned-word list) and fails an Enforced row whose "Verified by" line names no script, job or file that exists.npm run charter:auditjoinscheck:all,scripts/quality/verify-local-core.tsandci.yml'sspec-auditjob. PROC-06 moves to Enforced in the same pull request. - Method-level traceability for the clients (TEST-10). The script Track S Run 4 writes for
IStaticDataProvideris extended to every public method of the 39 domain clients: a method counts as covered when a Rule's text or a bound step calls it by name;npm run spec:coverage:clientsprints the uncovered list grouped by client and--cigates againstscripts/spec/client-spec-coverage-baseline.json, which may only shrink. Committed exactly as measured; no new scenarios in this item. TEST-10 moves to Enforced.
Landed: item 1, CI matches the gate matrix: validate-release runs every validator the charter lists for it, validate:esi and validate:spec run in static-analysis, the freshness diff covers the rate-limit-group and scope files, and validate:auth-scopes fails both directions of DES-04 (CHARTER gap register row 25). Item 6, the exclusion register: npm run ears lists every shall not Rule with its verdict and scenarios in reports/ears/ears-report.md and the job summary; TESTING.md and tests/bdd/GUIDE.md describe it. TEST-11 stayed Partial until the exclusions themselves were written as Rules (below). Its follow-up: REL-03 and GATE-06 Enforced (validate:versions covers the docs-site selector, which release-please bumps; package-scripts.test.ts fails on a document naming an undefined npm run), validate:esi fails on a definition the spec lacks unless excepted, and validate:esi:vendored runs it offline in check:local and in spec-refresh.yml.
Also landed: item 7, the charter audit: npm run charter:audit (scripts/quality/charter-audit.ts, checks in charter-audit-core.ts) parses every #### block of CHARTER.md with the spec-audit rules and fails an Enforced row naming no script, job or file that exists; in check:all, verify-local and ci.yml's spec-audit job; PROC-06 Enforced; twenty-three requirements reworded to one shall. Item 6, the exclusion register: npm run ears lists every shall not Rule with its verdict and scenarios in reports/ears/ears-report.md and the job summary; TESTING.md and tests/bdd/GUIDE.md describe it. TEST-11 stayed Partial until the exclusions themselves were written as Rules (below). Item 2, first part: npm run lint covers tests/ with the test relaxations declared in eslint.config.mjs, and TEST-09 is Enforced; with the knip part below and the stale-survivor fix for #380 (mutation:pr re-mutates, with --force, every reused file a changed or deleted test covered in the nightly report, and runs a test-only pull request over the directories its tests reach), item 2 is done.
Landed (2026-09-28): item 6, the exclusions themselves. Every exclusion of an ESI behaviour the guides stated in prose is an unwanted-behaviour Rule with a scenario, each checked against a mutant that acts on the behaviour, in the feature for its area: Phase 3's responses not validated (0053: a body that fails the schema is not fetched again, absurd values and unknown enum members pass), Phase 4's log fields not emitted (0058: the access token), Phase 6's headers not honoured (0052, 0055: Content-Type, Expires, X-Pages on later pages and past page 1000, the bearer token on public routes and in URLs), with the statuses not retried (0051), the cache features the page and stream paths skip (0050), the JWT signature (0054) and the SDE files and peers not read (sde/0020, sde/0021). npm run ears lists 29 exclusions, all verified; TEST-11 is Enforced.
Also landed: item 2's knip part: release.yml runs npx knip without --no-exit-code on a clean baseline, knip.jsonc treats tests, scripts and examples as entry points and gives each exception a reason, and validate and check:all block on it; GATE-04 Enforced. ci.yml still only reports.
Also landed 2026-09-28: item 8, method-level traceability for the clients: npm run spec:coverage:clients (scripts/spec/client-spec-coverage.ts, checks in client-spec-coverage-core.ts, the call graph, baseline and command line shared with spec:coverage:sde through scripts/spec/method-coverage-core.ts) reads the public instance methods each of the 39 classes extending BaseEsiClient declares, inherited ones excluded, and counts one as covered when a Rule names it or a bound step calls it on that client, the type checker resolving the call; legacy defineFeature scenarios bind by title. 173 of 384 methods are covered; the 211 uncovered, most of them the fetchAll* and stream* page helpers, are committed as measured in scripts/spec/client-spec-coverage-baseline.json, grouped by client, and spec:coverage:clients:ci fails an addition, a stale entry or an unlisted uncovered method in check:all, verify-local and the spec-audit job. No scenarios added; TEST-10 Enforced.
Definition of done. Every row of QUALITY-GATES.md's matrix matches a job; GATE-01 to GATE-06, TEST-10, TEST-11 and PROC-06 at Enforced in the CHARTER; the five issues closed; the AUDIT.md "Plan gates today" table re-run with Live health at Met.
Phase 6 · Security and supply chain · Open
Scope. What the OpenSSF Scorecard measures and what the charter's SEC rows still call Partial.
- Scorecard (#239 epic, #243 Code-Review, #246 Best Practices badge). The score and the date are recorded in the CHARTER's next revision.
- Branch protection for administrators and the remaining SEC-07 item (#270). Repository settings the maintainer changes; the executor verifies with the API and records the result.
- Trusted publishing.
publish-npmmoves fromNPM_TOKENto npm trusted publishing (OIDC), and the registries receive the tested tarball (npm publish release-artifacts/<tarball> --provenance) rather than a rebuild, closing the first item in RELEASE.md's Known state. Credentialed jobs: the maintainer approves the workflow change and configures npm. - Optional JWKS verification (#256). Done in 11.x:
verifyAccessToken,SsoJwksandEsiTokenManager'sverifyTokensflag, additive,feat:.
Landed: the repository side of items 1 and 2. Every workflow declares read-only top-level permissions, and tests/tdd/workflows/workflow-permissions.test.ts holds that and lists each job-level write scope (Token-Permissions). sign-and-publish-assets attests SLSA build provenance for the tarball, SBOM and docs archive, verifies it with gh attestation verify, and attaches lgriffin-esi.ts-X.Y.Z.intoto.jsonl beside the cosign bundles (Signed-Releases). The CycloneDX SBOM and .github/CODEOWNERS were already in place, and both publish jobs already upload the tested tarball rather than a rebuild (item 3's second half). Left for the maintainer, with steps and verifying API calls in SECURITY.md §5: branch protection including administrators with required approvals (SEC-07, #243), Best Practices registration (#246), and the npm trusted publisher that item 3's workflow change waits on. v10.2.2 and v10.2.3 have no release assets, so Signed-Releases rises only as signed releases ship.
Also landed 2026-09-28: the score is recorded, 8.3 / 10 on master 9486fe2c (CHARTER header, SECURITY.md "Where the score stands"), and the two checks the repository could still move were moved: Vulnerabilities (the four OSV advisories were vite and esbuild under docs-site/, now overridden to vite 7 with a clean audit and a passing site build) and License (LICENSE is the full GPL-3.0 text; the copyright line and CCP's trademark notice are in NOTICE, shipped in the package). scorecard.yml reads branch protection through SCORECARD_TOKEN when the maintainer adds it and falls back to the default token until then. Left for the maintainer: that secret, administrators included with required approvals, Best Practices registration and the trusted publisher.
Definition of done. SEC-04, SEC-06 and SEC-07 at Enforced; RELEASE.md Known state has no entry about a rebuilt tarball; the Scorecard number in the CHARTER header is newer than 2026-09-27.
Phase 7 · Release, Node 22 and the docs rewrite · Open
Scope. The last phase, because it documents the finished API and carries the one commit that makes the release a major on purpose.
- Node 22 floor (REL-05, #278).
engines.nodeto>=22.12.0,.nvmrcto 22, the unit and consumer matrices to 22 and 24, CONTRIBUTING.md and SECURITY.md's support table updated, in one commit titledfeat!: require Node 22with aBREAKING CHANGE:footer that tells a Node 18 or 20 consumer to stay on 10.x. Any other break held from Phases 3 to 6 lands in its owntype!:commit in the same pull request, each with its migration line; the pull request title isfeat!:. - Docs rewrite (#264, #272, #273, #276, #257). A first pass landed on 2026-09-27: the README is an orientation page over measured numbers,
guides/USAGE.mdandguides/AUTHENTICATION.mdhold what the README used to restate, rootTESTING.mdis folded intoguides/TESTING.md, and ARCHITECTURE.md covers the ports, adapters and layer rule. What remains is below and is rewritten against the new client once Phase 2 lands. This phase addsscripts/docs/sync-docs.tsto copyguides/intodocs-site/guide/, a VitePress build in the release workflow deployed with the TypeDoc output under/api/,scripts/docs/doc-metrics.tswritingetc/doc-metrics.json, and a banner check invalidate:versionsthat fails on a stale README or site version;guides/MUTATION-TESTING.mdfolds intoguides/TESTING.md; the README shrinks to an orientation page written against the new client, every snippet checked bytest:docs-examples;guides/MULTI-CHARACTER.mdand Track S'sguides/sde/are in the site. The Beads blocks in AGENTS.md and CLAUDE.md become pointers (tests/tdd/scripts/agent-docs.test.tskeeps the SemVer copies identical). - Changelog (#275, #377). 9.2 to 9.6 backfilled; the duplicate-title entries that merge commits produce are fixed in
release-please-config.jsonor by squash discipline, whichever the test intests/tdd/scripts/can check. - Deprecations that 11.0.0 announces, not removes.
EsiApiFactory's named methods andEsiTokenManager.createClientget@deprecatedJSDoc pointing at the builder andas(). Removal is 12.0.0 at the earliest.
Landed: item 1, the Node 22 floor. engines.node is >=22.12.0 and .nvmrc 22, in a feat!: require Node 22 commit whose BREAKING CHANGE: footer sends Node 18 and 20 consumers to 10.x; the unit matrix is Node 22 and 24, the consumer contract in ci.yml and release.yml runs Node 22 and 24 across the oldest, repository and latest TypeScript, and every workflow that pinned Node 20 pins 22. CONTRIBUTING.md, SECURITY.md's support table (now with a Node column), USAGE.md's upgrade note, RELEASE.md and the docs site say so; CHARTER REL-05 is Enforced. Item 4, the deprecations. EsiApiFactory.createAllianceClient, createCharacterClient, createCorporationClient, createMarketClient, createUniverseClient, createFleetClient, createAssetsClient, createWalletClient and createMailClient, and EsiTokenManager.createClient, carry @deprecated JSDoc naming createEsi() with esi.public or esi.as(identity) (for the token manager, esi.as(tokens.identity(id))) and removal in 12.0.0 at the earliest; no runtime warning, no behaviour change (feat:). EsiApiFactory.createClient(type) stays undeprecated as the legacy single-client path. USAGE.md, AUTHENTICATION.md, MULTI-CHARACTER.md, ERRORS.md, ARCHITECTURE.md, CHARTER ARCH-08 and the docs site say so. Item 2's site half (#264). npm run docs:site builds TypeDoc, then scripts/docs/sync-docs.ts generates docs-site/guide/ from README.md and every guides/*.md (a link to another guide becomes a page link, a link to a repository file a GitHub URL, an image is copied beside the page) and one showcase page per examples/*.ts from its header and source, grouped by category, and VitePress builds the site; the generated pages are git-ignored and the hand-written copies in docs-site/ are gone. ci.yml's documentation job builds the whole site on every pull request; release.yml's deploy-docs calls docs-site.yml, which publishes it to gh-pages with TypeDoc under /api/ and can be dispatched on master between releases. The version selector reads package.json at build time and validate:versions -- --site checks the built page; CHARTER DOC-03 is Enforced. Item 2's metrics part (#272): npm run docs:metrics writes etc/doc-metrics.json from the source and rewrites every count marked <!-- metric:name --> in README.md, the guides and the site's home page, and validate:versions fails on a stale count or README release line; CHARTER DOC-04 is Enforced. Item 2's mutation fold (#273), 2026-09-28: guides/MUTATION-TESTING.md is folded into guides/TESTING.md as its "Mutation testing" section and deleted, every link points at the section, and tests/tdd/scripts/sync-docs.test.ts fails on a guide link to a missing guide or heading; CHARTER DOC-01 is Enforced. Still open in item 2: the README rewrite, the Beads pointers and Track S's guides/sde/.
Definition of done. The release gate below is met in full; chore(master): release 11.0.0 (#405) shows the Node floor under Breaking Changes with its migration line; the docs site deploys from the release workflow and its version banner matches package.json.
Track S · The SDE programme
src/sde (6,289 hand-written lines, 99 methods on IStaticDataProvider, 102 files in SDE_FILE_REGISTRY) is a side module: nothing in src imports it and it imports nothing from src. That isolation is a fact today and a rule almost nowhere, and the quality tiers that make the core dependable stop at its door. Track S takes it from "isolated and best-effort" to "isolated by lint, specified per method, mutated, fuzzed, benchmarked and checked against CCP's real export every night", in twelve unattended overnight runs plus a standing mutation-kill run. It ships inside 11.0.0.
Where the SDE stands (measured 2026-09-27, master 519c945)
| Tier | Core today | SDE today | Status |
|---|---|---|---|
| Layer lint | ports, generated and core all constrained | Run 1: lint:layers holds both directions and memory-entry-bundle.test.ts keeps ./sde/memory free of the peers (ARCH-10) | Enforced |
| EARS specification | 402 requirements, step library, spec entries | 7 features, 21 rules, 23 scenarios for 99 methods. All 7 step files are legacy defineFeature files listed in legacyStepFiles (scripts/spec/spec-audit-exceptions.json). No requirement for ingestion, optional peers, locale, ZIP metadata or ./sde/memory | Gap |
| Unit mutation | src/core, per-directory ratchet, five shards | Not mutated: config/mutation/stryker.config.mjs and config/mutation/unit-shards.json cover src/core only | Gap |
| BDD mutation | sharded, floors 11.7 to 42.8 | In the rest shard of config/mutation/bdd-shards.json; floor 10.6 | Gap |
| Type mutation | . 10.5, ./errors 24 | ./sde 0, ./sde/memory 0 in config/mutation/type-thresholds.json; no test-d file for either | Gap |
| Properties and fuzz | 13 files, 10k runs nightly | None | Gap |
| Contract against upstream | live spec, nightly drift issue | Run 9: nightly-sde.yml runs sde:drift against CCP's current export every night and keeps one issue open on drift (GATE-07) | Enforced |
| Real data | recorded payloads re-recorded nightly | Run 9: nightly-sde.yml runs sde-real-data.test.ts with SDE_REQUIRE_DATA=1 against the current export, one table per family | Enforced |
| Benchmarks and soak | bench:ab and heap soak nightly | Run 10: six sde/ tasks in the catalogue (A/B on pull requests, nightly trend) and the SDE soak; nightly-sde.yml runs both against the real export and publishes the numbers; the README quotes measured figures | Enforced |
| Determinism | clock port | Two new Date() exemptions in scripts/quality/determinism-baseline.json (SdeDataProvider.ts, ingestion/SdeDatabaseBuilder.ts) | Partial |
| Consumer, docs examples, size | in place | Optional-peer probe, doc type-checks and .size-limit.cjs budgets all cover ./sde and ./sde/memory | Enforced |
| Nightlies | 16 scheduled workflows | Run 9: nightly-sde.yml at 05:15 UTC; the four examples/sde-*.ts run in it through examples:nightly -- --tier sde | Enforced |
The three commitments
- Isolated, by lint.
src/sdemay import Node built-ins,zod, its own files andsrc/core/portsonly; nothing undersrcoutsidesrc/sdemay import it../sde/memorynever reachesnode:fs,js-yaml,adm-ziporbetter-sqlite3. An ARCH requirement in the charter, enforced bylint:layersand a bundle test. - Wired up, meaning the machinery, not the pipeline. No port, no adapter, no name-resolution helper (decided 2026-09-27, above). Every gate the core has also covers
src/sde: its own mutation shards and floors, spec entries and a per-method coverage gate, a property tier, a real-data nightly with a drift issue, benchmarks, and a charter row for each. - Bullet-proof, per method. Every public method on
IStaticDataProvideris backed by at least one EARS rule and one scenario, gated the wayspec:coveragegates generated operations. Mutation floors ratchet from 10.6 toward 90. The real CCP export is loaded every night and any new file, field or removed record opens an issue before a user hits it.
How an overnight run works
Each run is one fresh, unattended session on a branch, scoped so a single night is enough. It ends with a pull request and never touches master. Its "done when" is machine-checkable, so the morning review is a glance at CI. Branches are named by conventional type and scope (PROC-04): <type>/sde-<run-slug>, for example chore/sde-isolation-lint.
Every run's session starts with this preamble, then the run's brief:
You are working unattended on lgriffin/ESI.ts. Read guides/ROADMAP.md, Track S, and take the run named below.
Clone, run `npm ci`, then `npm run check:local -- --fast` and confirm it is green before changing anything.
Work on a branch named `<type>/sde-<run-slug>`. Never commit to master. Never push --force.
Use the ears-gherkin-dev skill for every behaviour change: EARS rule and Gherkin scenario first, red, then green.
Classify the change per guides/SEMVER.md before committing; no `!`, no BREAKING CHANGE, no public-contract removals. If a step needs a breaking change, stop and write it up in the PR instead.
Conventional commits, one concern per commit, attribution trailer on each.
Finish with `npm run check:local` (full) green, `npm run api-report` if any export changed, open a PR titled as the run says, and put in the PR body: what changed, every gate you ran with its result, and anything you could not finish and why.
If a gate cannot be made green in one night, ship the partial with a shrink-only baseline and say so in the PR body. Do not lower any existing floor.Runs that must not proceed alone. A run stops and writes up instead of deciding when: a rule for existing behaviour reveals the behaviour is wrong (fixing it may be a visible change); a gate needs an existing floor lowered; a public export must be renamed or removed; anything wants the core to import the SDE. Everything else is the run's call, recorded in the PR body.
Wave A · Lock the boundary
Run 1 · Symmetric layer lint and the SDE charter row · 1 night · chore
- Goal. Turn "the SDE shares no code with the pipeline" from a sentence in CHARTER.md into two lint rules and a bundle test.
- Done when.
lint:layersfails on (a) any file outsidesrc/sdeimportingsrc/sde/**and (b) anysrc/sdefile importingsrc/**other thansrc/core/ports;tests/tdd/layers/layers-lint.test.tscovers both directions with positive and negative fixtures; a newtests/tdd/sde/memory-entry-bundle.test.tsbuilds (or usesdist/if present) and assertsdist/sde/memory.mjsanddist/sde/memory.jscontain none ofnode:fs,js-yaml,adm-zip,better-sqlite3; CHARTER gains ARCH-10 "Side-module isolation" in EARS form, status Enforced, mechanismlint:layersplus that test; ARCHITECTURE.md's layer table row for side modules names both. - Brief. Extend
config/eslint/layers.rules.cjs: add a rule group forsrc/sdeallowingnode:*,zod,./and../withinsrc/sde, andsrc/core/portsonly; add a rule forbiddingsde/import paths from anysrcfile outsidesrc/sde. Keep the existingABOVE_COREentry. Do not change anysrc/sderuntime code. - PR.
chore(lint): enforce SDE isolation in both directions - Marker. an
sderule group inconfig/eslint/layers.rules.cjs.
Run 2 · Mutation plumbing: own shards, own floors, clock port · 1 night · chore
- Goal. Give the SDE the same ratchet the core has, so every later run's tests count.
- Done when.
config/mutation/bdd-shards.jsonhas ansdeshard (src/sde, ingestion included) andrestexcludes it;config/mutation/unit-shards.jsonhas ansdeshard and its$invariantis widened from "partitionsrc/core" to the directories the unit run mutates;tests/tdd/mutation-ratchet/bddShards.test.tsandunitShards.test.tspass;config/mutation/stryker.config.mjsmutatessrc/sde/**/*.tswith the existing excludes plussrc/sde/SdeTestDataFactory.ts(a test fixture, commented as such);config/mutation/unit-thresholds.jsongainssrc/sdeandsrc/sde/ingestionseeded from a local run minus 2 points and marked provisional in the PR body (the nightly re-seeds them; floors come from the nightly, never from a laptop);nightly-mutation.ymlpicks the new shards up (it reads the shard files withjq, so no YAML change is expected; confirm); a local--dryRunOnlymutant count for each new shard is recorded in the shard file's$balance; the twonew Date()sites inSdeDataProvider.ts:73andingestion/SdeDatabaseBuilder.ts:213take an injectableClockfromsrc/core/ports(constructor option, defaultsystemClock) and their entries leavescripts/quality/determinism-baseline.json, with a unit test passing a fixed clock and asserting the timestamp. - Brief. Read the "Mutation testing" section of
guides/TESTING.md,config/mutation/bdd-shards.jsonandconfig/mutation/unit-shards.jsonfirst. Runnpx stryker run config/mutation/stryker.config.mjs --mutate "src/sde/**/*.ts" --concurrency 4locally; if it exceeds 90 minutes, run per subdirectory. The clock change is behaviour-neutral. - PR.
chore(mutation): mutate src/sde in its own shards and seed its floors - Marker. an
sdeshard inconfig/mutation/bdd-shards.json. - Landed 2026-09-27. Both shard files carry an
sdeshard (the unit invariant now coverssrc/coreandsrc/sde); the unit run mutatessrc/sde/**minus the test-data factory;src/sdeandsrc/sde/ingestionare scored on their own in both tiers (directoryOftreats the SDE like the core) and have provisional floors, unit 85.6 and 71.8, BDD 20.6 and 0, until the nightly re-seeds them;SdeDataProviderandSdeDatabaseBuildertake aClock, defaulting tosrc/sde/clock.ts, the SDE's own copy of the real clock, because the SDE may import nothing of the core but the ports (solint:determinismallow-lists two clock modules, and the twonew Date()entries left the baseline).
Wave B · Specify it properly
Run 3 · Convert the seven legacy SDE step files · 1 night · test
- Goal. Move the SDE onto the step library and spec entries so it is audited, bound and mutated like a converted core domain.
- Done when.
tests/bdd/specs/sde/0001…0007-*.spec.tsexist (eachbindFeature(__filename)); one step per file undertests/bdd/steps/{given,when,then}/; fixtures intests/bdd/support/sde.ts;tests/bdd/step-definitions/sde/is gone and its seven entries are removed fromlegacyStepFiles; thebdd:sdescript points atspecs/sde;npm run bdd:steps,spec:auditandears -- --only=sdeare green with identical scenario counts before and after. - Brief. Follow
tests/bdd/README.mdsections 6 to 8 andtests/bdd/GUIDE.mdexactly. Do not reword any Rule text. Where two legacy files define the same step phrase with different bodies, unify the phrase in the step library and adjust the scenario text, recording each such change in the PR body. Runlint:bdd-seamandlint:suite-healthafter conversion. Paste the before and afterears --only=sdesummaries in the PR body. - PR.
test(bdd): convert SDE step definitions to the step library - Marker. no
sde/entries inlegacyStepFiles. - Landed 2026-09-28.
tests/bdd/specs/sde/0001…0007bind the seven features against 66 new step files andtests/bdd/support/sde.ts; no scenario text or Rule text changed, andears --only=sdereports 21 requirements across 7 features, 23 scenarios, before and after. The World carries the open provider (this.sde) and the After hook closes it. The one step the legacy files repeated with different nouns (the result should contain at least N …) became one regular-expression step.
Run 4 · Method-level spec coverage gate · 1 night · chore
- Goal. Know exactly which of the 99 interface methods have no requirement, and make that number shrink-only.
- Done when.
scripts/sde/sde-spec-coverage.ts(pure logic insde-spec-coverage-core.ts, tested undertests/tdd/scripts/) parsesIStaticDataProvider.tsfor method names andtests/bdd/features/sde/**plus the bound SDE steps for method references, and prints a table of method, rule count, scenario count;npm run spec:coverage:sdeexists;--cigates againstscripts/sde/sde-spec-coverage-baseline.json, which may only shrink (theexport-coverage-baseline.jsonpattern); the check is incheck:all,scripts/quality/verify-local-core.tsandci.yml'sspec-auditjob; CHARTER TEST-10 moves from Gap to Partial (Phase 5 item 8 extends the same script to the domain clients and moves it to Enforced). - Brief. Model on
scripts/spec/spec-coverage.tsandscripts/package/export-coverage.ts. A method counts as covered when a Rule's text or a bound step body calls it by name on the provider. Print the uncovered list grouped by entity family in the section order ofsrc/sde/README.md(Types, Universe, Market, Dogma, Industry, Character/Lore, NPC, Agents, Certificates, Skins, Stations, Extensions, Missions, Notifications, Generic, Lifecycle); this list is the input to Runs 5 and 6. Commit the baseline exactly as measured. Write no new scenarios in this run. - PR.
chore(spec): gate IStaticDataProvider method coverage against the EARS specification - Marker.
scripts/sde/sde-spec-coverage.tsexists. - Landed 2026-09-28.
npm run spec:coverage:sdereports 23 of 101 provider methods (the interface has grown two since this run was written) named by a Rule or reached by a bound step, directly or through anytests/bddfunction, the type checker deciding what a call's receiver is; the 78 uncovered are committed by family inscripts/sde/sde-spec-coverage-baseline.json, andspec:coverage:sde:cifails an addition, a stale entry, or an unlisted uncovered method, incheck:all,verify-localand thespec-auditjob. CHARTER TEST-10 is Partial. The baseline is the input to Runs 5 and 6.
Run 5 · EARS requirements, entity families I · 1 to 2 nights · test
- Goal. Requirements and scenarios for every uncovered method in Types, Universe (regions, constellations, systems, stargates, planets, moons, stations), Market groups and Dogma (attributes, effects, type attributes and effects).
- Done when. Those families show 0 uncovered in
spec:coverage:sde; each new Rule is one atomicshallwith a rationale paragraph and, where the method returns nullable, exactly one lookup-present and one lookup-absent scenario, or where it returns a collection, a "returns every X of Y, ordered by ID" scenario asserting order and count; scenarios run againstMemorySdeProviderbuilt fromSdeTestDataFactory; the baseline shrunk;spec:auditgreen. - Brief. Match the voice of
tests/bdd/features/sde/0001-static-data-lookup.feature: rule, why it matters to a caller, scenario. Add features0008+, one per family; never edit existing rule text. Every FK-traversal method gets a scenario where the parent exists but has no children (empty array, not null). If a method's behaviour is ambiguous from the code (whatsearchTypesByNamedoes with an empty fragment, for example), write the Rule for the current behaviour and flag it in the PR body as "needs a decision" rather than changing it. Stop when the families are covered or after the second night, whichever comes first. - PR.
test(sde): specify type, universe, market and dogma lookups - Marker. Types, Universe, Market and Dogma at 0 uncovered.
- Landed 2026-09-28. Features
0008to0012(type classification, universe geography, celestial bodies, market groups, dogma definitions): 23 Rules and 30 scenarios over an extendedMemorySdeProviderdata set built intests/bdd/support/sde.tsfrom the factory's hierarchical set; 36 new step files. Types, Regions, Search, Universe, Market and Dogma are at 0 uncovered, so the baseline fell from 78 to 56 (45 of 101 covered). Two behaviours are recorded as they are and flagged for a decision rather than changed: neither provider orders a table it reads whole (both return load order, so the whole-table scenarios assert membership and count, not order), and an empty search fragment matches every record up to the limit. Leigh decided the first on 2026-09-28 ("ordered by promise"): both providers sort every table by ID at load time, a Rule under feature 0008 holds it, and the fuzz oracle expects it of both. The empty fragment is still open.
Run 6 · EARS requirements, entity families II and lifecycle · 1 to 2 nights · test
- Goal. The remaining families (Industry, Character/Lore, NPC, Agents, Certificates, Skins, Stations, Type extensions, Missions, Notifications, Generic accessors) plus everything that has no feature at all:
fromZipwith nested_sde.yaml, optional-peer errors, English-locale extraction, ID-suffix renaming, lazy FK index build,getVersionandclose,./sde/memoryparity with./sde, and the ingestion pipeline (download, extract, build, transforms). - Done when.
spec:coverage:sdereports 0 uncovered and the baseline is empty; features exist for0020-sde-loading,0021-sde-optional-peers,0022-sde-memory-entry,0023-sde-ingestion; ingestion scenarios mock only at the fs and network seam (lint:bdd-seamgreen);ears -- --only=sdegives every requirement a verdict. - Brief. Same conventions as Run 5. For loading and ingestion, the Given steps build a temp directory or an in-memory ZIP with
adm-zipfromSdeTestDataFactoryrecords; nothing touches the network (SdeDownloadergets a stubbedfetch). Optional-peer scenarios usejest.doMockto make the peer unresolvable and assert theSdeErrormessage names the install command. One Rule per transform iningestion/transforms.ts(ID renaming, locale extraction, nested metadata) with a scenario each. A parity scenario: every export name from./sdeexceptSdeDataProviderand the ingestion classes is exported from./sde/memory. - PR.
test(sde): specify remaining lookups, loading, optional peers and ingestion - Marker. baseline empty and
0023-sde-ingestion.featureexists. - Landed 2026-09-28. Features
0013to0023: the eleven remaining families over a reference data set that extends Run 5's (openReferenceProviderintests/bdd/support/sde.ts), then loading, optional peers, the memory entry point and ingestion over files a newtests/bdd/support/sdeFiles.tswrites to a temp directory and archives withadm-zip; 71 Rules and 96 scenarios, 58 new step files. The downloader's fetch is answered by the transport seam (queueResponsewith a URL match), so no scenario touches the network; a missing peer is made unresolvable withjest.doMock, which reaches the entry point'screateRequireunder Jest; the database builder's output is read back withbetter-sqlite3.spec:coverage:sdereports 101 of 101 methods covered and the baseline is empty, so from here the gate fails on any new provider method without a Rule. The World gained acleanupslist the After hook runs, for temp directories and lifted mocks.
Wave C · Depth tiers
Run 7 · Properties and model-based fuzz · 1 night · test
- Goal. A fast-check tier for the SDE that the nightly property run drives to 10,000 runs.
- Done when.
tests/fuzz/sde-transforms.property.test.ts(ID renaming is idempotent and reversible on the suffix; locale extraction never returns a non-string for a localised map; nested metadata is found at any depth),tests/fuzz/sde-provider-model.property.test.ts(a generated data set loaded intoMemorySdeProvideranswers every lookup identically to a naive oracle over the same arrays, FK-index queries after lazy build included) andtests/fuzz/sde-schema-fuzz.test.ts(generated records round-trip throughschemas.ts; a record missing a required key is rejected) exist;npm run fuzz:propertiesandnightly-properties.ymlpick them up without workflow changes; each honoursFC_NUM_RUNSandFC_SEEDpertests/fuzz/AGENTS.md. - Brief. Read
tests/fuzz/AGENTS.mdandetag-cache-model.property.test.tsas the model-based pattern. Arbitraries live intests/fuzz/support/sde.tsand generate internally consistent data (every FK points at an existing parent) plus a controlled fraction of dangling FKs. Keep the PR-time run under 30 seconds at 100 runs. A counter-example is a bug: write the EARS scenario, fix it, note it in the PR body; never weaken the property. - PR.
test(fuzz): property and model-based tests for the SDE provider and transforms - Marker.
tests/fuzz/sde-provider-model.property.test.tsexists. - Landed 2026-09-28. The three files, registered through
describePropertywhere they are model-based, so each property fails against its known-bad subjects: the field-name transform against a first-match-only, an inside-word and a lower-casing rename; locale extraction against a map value passed through, the English entry always read and the fallback ignored; the record transforms against a top-level-only rename, a forgotten ID and nested values stored as objects; the metadata parser against a top-level-only read and the top level winning; both providers against a dropped last record, an ignored search limit, an off-by-one ID and a foreign-key index that answers every query from its first.tests/fuzz/support/sde.tsgenerates the data set (28 families, IDs unique per table, one foreign key in ten dangling), the query sequences and the oracle; the SQLite route writes the set as YAML to a temp directory and loads it withfromDirectory, its oracle sorted by ID because a YAML map keyed by integers enumerates in that order. The schema fuzz derives an arbitrary from each zod definition insrc/sde/schemas.ts, so a new field or schema is covered the day it lands;BlueprintActivitiesSchemais the one schema with no required key and is listed as such. The whole fuzz tier runs in about 22 s with four workers; the SDE files at 2,000 runs in about 60 s. No counter-example surfaced against the code. Two edges the oracle deliberately did not decide, because no Rule did: a search limit below 1 (both providers answer one record for a limit of 0) and the order of a whole-table answer; the generator asks for limits of 1 or more, and since Leigh's decision of 2026-09-28 the oracle expects ID order of both providers.
Run 8 · Type tests and the type-mutation ratchet · 1 night · test
- Goal. Both SDE entry points get tsd coverage and a non-zero type-mutation floor.
- Done when.
tests/typetests/sde.test-d.tsandsde-memory.test-d.tsassert everyIStaticDataProvidermethod's return type (nullable against array), thatMemorySdeDatakeys match the entity type names, thatSdeDataProvideris absent from./sde/memory, and that the error guards narrow;npm run test:type-mutationrun locally for./sdeand./sde/memoryandconfig/mutation/type-thresholds.jsonraised from 0 to the measured score minus 2;scripts/package/export-coverage-baseline.jsonshrunk for the two SDE sub-paths. - PR.
test(types): tsd coverage for ./sde and ./sde/memory; seed type-mutation floors - Marker.
./sdeabove 0 inconfig/mutation/type-thresholds.json. - Landed 2026-09-28.
tests/typetests/sde.test-d.tsandsde-memory.test-d.ts: construction (fromDirectory,fromZip, the options,close), every one of the 99 provider methods on the port and on both providers (nullable single lookups against arrays, the search limits, the genericgetAllandgetById), the identifying field of every entity type,MemorySdeDatakeys against the entity type names, the error hierarchy and its guards narrowing,SdeVersionInfo, and./sde/memoryexporting the factory and the memory provider withoutSdeDataProvider; the two SDE sub-paths leftscripts/package/export-coverage-baseline.json. Measured on the ratchet's sample (seed 1, 500 mutants):./sde34.1,./sde/memory27.2, so the floors are 32 and 25. The same run scored./schemasat 4.1 against its floor of 6 and./testingat 11.7 against 0: the sample is drawn over every entry point's declarations, so the./cliententry point joiningdist/(Phase 2 PR 11) reshuffled it after those floors were seeded. No floor was lowered; the next nightly type-mutation run says whether./schemasholds on CI, and a re-seed of the pre-existing floors is Leigh's call.
Run 9 · Real-data nightly and registry drift · 1 to 2 nights · ci
- Goal. Load CCP's actual latest export every night, run the real-data suite, and open an issue when the export drifts from what the module knows.
- Done when.
.github/workflows/nightly-sde.yml(cron15 5 * * *, afternightly-audit, beforenightly-spec-drift;timeout-minutes: 60; concurrency groupnightly-sde; every action SHA-pinned; zizmor clean) resolves the current build fromSDE_LATEST_BUILD_URL, restores the ZIP fromactions/cachekeyed on that build number (download only on a miss), runssde:ingesttosde-data/, thentests/integration/sde;scripts/sde/sde-drift.ts(offline, over an extracted directory; pure logic insde-drift-core.ts) compares the ZIP's file list withSDE_FILE_REGISTRYand each file's observed top-level keys with the Zod schema for that table, writesreports/sde-drift.jsonand a step summary naming the build;tests/tdd/scripts/sde-drift.test.tsuses a fixture directory with one unknown file and one record with an extra key; on drift or failure the workflow opens or comments on one fixed-title issue and closes it after the next green night (thenightly-spec-drift.ymlpattern);sde-real-data.test.tsfails instead of skipping whenSDE_REQUIRE_DATA=1andsde-data/is missing, and gains per-family smoke queries so every family in the README's API reference is exercised against real data; the fourexamples/sde-*.tsrun in this workflow throughexamples:nightly(thesdetier runs whensde-data/is present, otherwise skips); QUALITY-GATES.md gains the GATE-07 row and CLAUDE.md's CI Workflows list names the workflow. - PR.
ci(sde): nightly real-data run and export drift report - Marker.
.github/workflows/nightly-sde.ymlexists. - Landed 2026-09-28.
nightly-sde.ymlat 05:15 UTC: the build number from CCP's feed, the ZIP fromactions/cachekeyedsde-zip-<build>(downloaded on a miss and saved),sde:ingest --from-zip(new flag;--keep-ziptoo),tests/integration/sdewithSDE_REQUIRE_DATA=1,sde:drift, thenexamples:nightly -- --tier sde; areportjob keeps one issue titled "sde: nightly real-data run failed or the export drifted" on scheduled runs.scripts/sde/sde-drift-core.tsmaps every registered file to its schema (SCHEMA_BY_FILE), compares the export's file list with the registry and each file's record keys, aftertransformRecordNative, with the schema's: a key on any record the schema does not declare is new, a key the schema requires (neither optional nor nullable) that no record carries is gone.tests/tdd/scripts/sde-drift.test.tsruns it overfixtures/sde-drift(one unknown file, one category withsortOrder), checks every registered file has a schema and every injected ID is declared, and pins the workflow to the module's URLs.sde-real-data.test.tsfails on a missing export when required and queries one table per family (50 tables, 19 families) through the generic accessor and the family's own lookup.examples-core.tsgainedrunTiers: thesdetier joins the run when an export is on disk, and--tiernarrows a run. The container this landed from cannot reach CCP, so the first scheduled run is the first real-data proof; GATE-07 is Enforced in the charter on the strength of the workflow and its tests.
Run 10 · Benchmarks and heap soak · 1 night · test
- Goal. Measure the load-time and memory claims and put them under the same A/B and trend machinery as the pipeline.
- Done when.
tests/benchmark/sde.bench.tsmeasuresfromDirectoryload time on a generated 50k-type synthetic data set, p50 and p95 forgetType,searchTypesByNameand one FK-index query before and after index build, and peak heap;scripts/bench/bench-ab.tsincludes it inbench:compare;scripts/bench/soak.tsgains an SDE scenario (load, 100k random lookups, close, repeat; heap returns to baseline within a tolerance);nightly-sde.ymlruns the SDE bench and soak against the real export and publishes the numbers to the step summary;src/sde/README.md's "~500 MB" is replaced with the measured figure and the build it was measured on. - PR.
test(bench): SDE load, lookup and heap benchmarks with nightly trend - Marker.
tests/benchmark/sde.bench.tsexists. - Landed 2026-09-28.
tests/benchmark/sde.bench.tswrites a seeded 50,000-type export as CCP-shaped YAML once per process and adds sixsde/tasks to the catalogue:fromDirectoryload,getTypeby ID,getTypesByGroupwith the foreign-key index built and cold, andsearchTypesByNamewith a shared fragment and with one whole name;BenchTaskgainedminSamplesso a one-second operation is sampled eight times, not thirty. Because the tasks live intasks.ts,bench:abandbench:comparemeasure them on both sides of a pull request and the nightly trend records them.tests/benchmark/sde-soak.ts(npm run soak -- --sde) loads, answers 100,000 lookups, closes and repeats;analyseSdeSoakfails a cycle whose heap stays more than the larger of 8 MiB and 5% of the loaded heap above the baseline afterclose(), or a loaded heap that grows across cycles.nightly-sde.ymlgained aperformancejob that restores the night's archive and runs both against the real export, two rounds, into the step summary and ansde-performanceartifact; the report job counts its failure. The README's "~500 MB" is replaced with the measured synthetic figure (64 MiB heap, 348 MiB RSS, 0.7 s for 50,000 types on Node 22) and a pointer to the nightly table for the real build, which this container could not download.
Wave D · Write it down
Run 11 · Docs consolidation and charter closure · 1 night · docs
- Goal. Close the open charter items and make every guide describe the SDE as it now is.
- Done when.
src/sde/docs/*moved toguides/sde/withguides/SDE.mdas the index andsrc/sde/README.mdreduced to a pointer;docs-site/guide/sde.mdregenerated from that source (byscripts/docs/sync-docs.tsonce Phase 7 adds it, by hand until then, never rewritten separately);scripts/docs/doc-examples-core.tspaths updated andtest:docs-examplesgreen; CHARTER rows ARCH-10 (Run 1), TEST-10 (Run 4), GATE-07 (Run 9) at Enforced and the SDE lines of TEST-05 to TEST-08 and GATE-05 updated; a one-line note that the static-data port was considered and cut; AUDIT.md'ssrc/sdeline count, mutation and coverage numbers refreshed from the latest nightly; QUALITY-GATES.md's matrix has an SDE column; MUTATION-TESTING.md's "where the scores stand" includes the SDE directories; beadsesi-x3zand the SDE part ofesi-l38.8closed or updated. - PR.
docs(sde): move the SDE doc set to guides/ and record the enforced gates - Marker.
guides/sde/exists. - Landed 2026-09-28.
src/sde/docs/{ARCHITECTURE,USAGE,DEVELOPER_GUIDE,API_CONTRACTS}.mdand the module README (nowguides/sde/REFERENCE.md) moved with their history toguides/sde/;src/sde/README.mdis a pointer.scripts/docs/sync-docs.tsreads one folder belowguides/,slugForprefixes a subfolder guide (sde-usage) so it cannot shadowguides/USAGE.md, and the site gains a "Static Data Export" sidebar section holdingSDE.mdand the five;docs-site/guide/sde.mdis generated from that source as before.DOC_SOURCESinscripts/docs/doc-examples-core.tschecksguides/sdeinstead ofsrc/sde/docs. CHARTER: ARCH-10 records the static-data port considered and cut, TEST-05 to TEST-08 and GATE-05 name their SDE mechanism, TEST-10 states the provider half closed, GATE-07 stands Enforced from Run 9; the documentation tables mark the move done. AUDIT.md'ssrc/sdeline count and mutation rows and MUTATION-TESTING.md's "where the scores stand" carry the SDE directories (provisional floors, seeded locally until the nightly re-seeds). QUALITY-GATES.md's matrix has an SDE column and a real-export row.bdis not installed in the container this landed from, so beadsesi-x3zandesi-l38.8are left for a session with the tracker; both are noted here.
Run 12 · Clean architecture for src/sde · 1 night · refactor
- Goal. Give the SDE module the structure the core has, so a reader finds a port, a domain, a provider or an ingestion step by its folder rather than by its file name. Asked for by the maintainer on 2026-09-27 ("as clean an architecture as possible"), and placed here, after every other run, so the moves land on files the earlier runs have finished specifying, mutating and documenting.
- Done when.
src/sde/isports/(IStaticDataProviderand any other type-only contract, importing nothing),domain/(the 1,100-linetypes.tsand 1,000-lineschemas.tssplit one file per SDE domain: types, universe, market, dogma, and so on, each schema next to the type it validates),providers/(sqlite/forSdeDataProviderandmemory/forMemorySdeProvider, each with its query and index helpers beside it),ingestion/as today,testing/forSdeTestDataFactory, witherrors.ts,version.ts,optionalPeers.tsand the two entry files at the root;config/eslint/layers.rules.cjsgains rules inside the module (ports import nothing; domain imports only ports; providers import domain, ports and the peers; ingestion never imports a provider; testing is imported by nothing undersrc/) andtests/tdd/layers/layers-lint.test.tscovers each;./sdeand./sde/memoryexport exactly the names they export today, checked bynpm run api-reportshowing no line lost, so the change isrefactor:and not a break; every SDE test, feature, fuzz, type test and benchmark still passes;guides/sde/andguides/SDE.md's C4 L3 diagram describe the new layout; mutation and coverage floors unchanged. - PR.
refactor(sde): split the module into ports, domain, providers, ingestion and testing - Marker.
src/sde/ports/exists. - Landed 2026-09-28.
src/sde/is nowports/IStaticDataProvider.ts;domain/with one folder per SDE domain (universe,types,dogma,industry,market,characters,corporations,skins,content,ui, eachtypes.tsbeside itsschemas.ts,version/schemas.ts, andtypes.ts/schemas.tsbarrels so the two entry files export the same names);providers/withyaml/SdeDataProvider.ts,memory/MemorySdeProvider.tsand the sharedorder.ts(yaml/, not thesqlite/written above: the production provider reads YAML, and the folder is named for what it does);ingestion/unchanged;testing/SdeTestDataFactory.ts;errors.ts,version.ts,clock.ts,optionalPeers.tsand the two entry files at the root.config/eslint/layers.rules.cjsgainedSDE_LAYERS: ports import domain only, domain imports domain (andzod), providers import ports, domain, providers and ingestion, ingestion imports ingestion, testing imports ports, domain and providers, and nothing undersrc/sdeimportsindex.tsormemory.ts;tests/tdd/layers/layers-lint.test.tscovers 21 crossings and the allowed edges, andguides/SDE.mdcarries the table.npm run api-reportlost no line, so the change isrefactor:. Re-seeded floors from local runs on the new layout: unitsrc/sde93.7 /src/sde/ingestion73.8 /src/sde/providers89.5 over 838 mutants (floors 85.6 kept, 71.8 kept, 87.5 new), BDDsrc/sde31.2 /src/sde/ingestion58.0 /src/sde/providers58.5 over 1,007 mutants (floors 29.2, 56.0, 56.5);src/sde/domainyields no valid mutants andsrc/sde/testingis excluded in both runs. The BDD seed only became possible with #523: the step-library specs bind at run time, so--findRelatedTestshad selected no SDE scenario since Run 3 and thesdeBDD shard scored nothing. The unit Stryker sandbox needs a realnode_modulesdirectory in a worktree (a symlink is skipped), noted in MUTATION-TESTING.md.
Run M · Kill SDE mutants, one directory per night · repeat · test
Once Runs 2, 3 and 5 to 7 are merged, the SDE has enough test surface for the same loop as the core attack plan. Schedule it for any night nothing else is queued, until every SDE directory sits at 90 or carries documented equivalents.
- Input. The latest
mutation-reportnightly artifact. - Done when. The chosen directory's score is up by at least 10 points or at 90; every remaining survivor has a verdict (real gap, no coverage, equivalent with a one-line reason, dead code deleted); the floors in
config/mutation/unit-thresholds.jsonandconfig/mutation/bdd-thresholds.jsonraised to the nightly-comparable score; one directory per PR. - Brief. Pick the SDE directory with the highest points-per-kill (100 / valid mutants) below 90. Follow "Mutation testing" in
guides/TESTING.md: NoCoverage first, then survivors grouped by function, boundary tests at n-1, n, n+1, exact-value assertions instead oftoBeDefined, side-effect assertions for emptied blocks. Prefer one sharp scenario in the EARS feature over a unit test where the behaviour is caller-visible; unit tests for internals such as index building. Delete redundant tests you make obsolete. Never touchtests/mutation-fixture. - PR.
test(sde): raise <directory> mutation score to <n> - Marker. any SDE directory below 90 in the latest mutation report.
- Status. Done 2026-10-08 (#458): every SDE directory at 90 or above on both the unit and the BDD-only run, in #607, #608 and #610 to #612; scores, floors and the survivors left with a reason are in sde/TESTING.md.
Nightly CI after the programme
| Workflow | UTC | SDE coverage added by |
|---|---|---|
nightly-mutation | 02:00 | Run 2: sde shards in both the unit and BDD runs, both ratcheted |
nightly-no-retry | 03:00 | Already runs the SDE suites; grows with Runs 3 to 6 |
nightly-properties | 03:30 | Run 7: three SDE property files at 10k runs |
nightly-examples | 04:15 | Unchanged: the sde tier stays type-check only here and runs inside nightly-sde |
nightly-benchmarks | 04:30 | Run 10: SDE bench in the bench:compare trend |
consumer-matrix-nightly | 04:45 | Already probes the optional peers |
nightly-sde (new) | 05:15 | Runs 9 and 10: real export cached by build, real-data tests, drift issue, bench and soak numbers |
Cadence and the weekly task
One run a week, Sunday night into Monday at 00:54 Europe/Dublin, as a scheduled task that starts a fresh session against the repository with the preamble above. The SDE is a value-add, not a core use case, so it takes one night a week and no more; the core phases keep the other nights. The task keeps no queue by hand: it checks the repository for each run's marker and for an open pull request whose branch matches */sde-*, and takes the first run that is neither done nor already in review. If the previous week's pull request is open with review comments, it addresses those instead of starting the next run. Runs 1, 2, 7, 8, 9 and 10 do not depend on the spec work, so if a week's run is blocked on review the task may take the next independent one rather than idle. Run 12 is the exception: it waits until Runs 1 to 11 are merged, because it moves the files they finish.
| Week | Run | Marker the task checks |
|---|---|---|
| 1 | Run 1 | an sde rule group in config/eslint/layers.rules.cjs |
| 2 | Run 2 | an sde shard in config/mutation/bdd-shards.json |
| 3 | Run 3 | no sde/ entries in legacyStepFiles |
| 4 | Run 4 | scripts/sde/sde-spec-coverage.ts exists |
| 5–6 | Run 5 | Types, Universe, Market, Dogma at 0 uncovered |
| 7–8 | Run 6 | baseline empty and 0023-sde-ingestion.feature exists |
| 9 | Run 7 | tests/fuzz/sde-provider-model.property.test.ts exists |
| 10 | Run 8 | ./sde type-mutation threshold above 0 |
| 11–12 | Run 9 | .github/workflows/nightly-sde.yml exists |
| 13 | Run 10 | tests/benchmark/sde.bench.ts exists |
| 14 | Run 11 | guides/sde/ exists |
| 15 | Run 12 | src/sde/ports/ exists |
| 16+ | Run M | any SDE directory below 90 in the latest mutation report |
Morning review per run is three things: is CI green on the pull request, does the PR body list a gate that could not be made green, and did any floor move down (it must never). Merge, or leave a review comment and re-queue the same brief with "address the review comments on PR #n first".
The 11.0.0 release gate
#405 merges when every line below is true. Each is checkable from the repository or CI; none is a matter of opinion.
The table is the gate. A row moves to Yes only by citing the merged pull request or the check that proves it, in the same pull request that makes it true; no checkbox syntax, so the tracker stays the tracker (PROC-01).
| Area | Condition | Holds today |
|---|---|---|
| Phase 2 | PRs 10b, 11 and 12 merged (#465, #472, #478); #183 and #432 closed. | Yes |
| Phase 3 | BASELINE in config/eslint/layers.rules.cjs is empty (#480); #295, #266, #298 closed (#470, #471); typecheck:isolated in CI (#482). | Yes |
| Phase 4 | #296, #268, #265 closed; sideEffects: false in package.json (#481, #486). | Yes |
| Phase 5 | GATE-01 to GATE-06, TEST-10, TEST-11 and PROC-06 at Enforced; #297, #271, #277, #382 closed (#526 GATE-05, #528 TEST-10, #531 TEST-11). | Yes |
| Phase 6 | SEC-04 and SEC-06 at Enforced. SEC-07 (branch protection without bypass, #270) and npm trusted publishing (#548) follow the release in 11.x: both are maintainer settings (decided 2026-09-28). (#256 may follow in 11.x.) | Yes |
| Phase 7 | feat!: require Node 22 on master with its BREAKING CHANGE: footer; docs site deployed by the release workflow; #264, #272, #273 closed; deprecations announced (#502, #506, #510, #529, #496). | Yes |
| Track S | Runs 1 to 12 merged (the markers table above all true; Run 12 #527); Run M, the mutation nights, follows the release. | Yes |
| Release mechanics | #383 and #378 closed: release-please's tag triggers release.yml and post-publish-canary.yml without a manual dispatch (#468, #474). | Yes |
| Release mechanics | etc/esi.ts.api.md regenerated on the release commit; every lost line covered by a type!: commit or an API-Compatible: trailer. (Current on master e4daf989: npm run api-report changes nothing, and the api-surface job proves that on every pull request; release-please's commit touches only package.json and CHANGELOG.md, so the report is the release commit's.) | Yes |
| Release mechanics | CHANGELOG.md has a ## [11.0.0] heading whose Breaking Changes section lists every ! commit with a migration line. (#405's body carries that heading with the six ! commits under Breaking Changes, each with its migration line; release-please writes it to CHANGELOG.md on merge.) | Yes |
| Release mechanics | SECURITY.md's support table lists 11.x as supported and 10.x with its end date. (#535: security fixes until 2027-03-31.) | Yes |
| Release mechanics | The CHARTER header numbers (clients, endpoints, requirements, scenarios, coverage) re-measured and revision 3 published. (#535, measured on master 114a485a.) | Yes |
| Release mechanics | npm run check:local -- --all green on the release commit. The nightlies are not on the release path (decided 2026-09-28): their open issues follow in 11.x. (On master e4daf989, 2026-09-28: 31 of 33 tiers green in one run, lint:workflows needs GitHub, which a container cannot reach, and is proven by the Workflow Security (zizmor) job on every pull request, and test:type-mutation passes re-run alone, 500 mutants, exit 0. Release-please's commit touches only package.json and CHANGELOG.md.) | Yes |
Mutation is not on the release path (decided 2026-09-28 by the maintainer). The nightly mutation runs keep ratcheting and filing issues, but 11.0.0 does not wait for them. These follow the release, in 11.x:
- Every directory in
config/mutation/unit-thresholds.jsonat 60 or above and every directory inconfig/mutation/bdd-thresholds.jsonat 20 or above,src/schemasexcepted in the BDD file (CHARTER TEST-07). Below today: unitsrc/core/logger; BDDsrc,src/clients,src/core/pagination,src/core/rateLimiter. mutation-prback inci.ymland needed byci-success(advisory inmutation-pr.ymlsince 2026-09-27).Every SDE directory at 90, or with every survivor carrying an equivalence reason (Run M).Done 2026-10-08.
Executing this plan
For a session that picks up a phase or a run:
- Read this file, then CHARTER.md Part 1 (posture) and the guide the phase's requirement IDs point at. Do not re-derive the plan from the conversations; if this file is wrong, fix it in the pull request.
- Take the smallest open unit: one Phase 2 PR, one numbered Phase 3 to 7 item, or one Track S run. One pull request per unit, titled as a conventional commit, body in the Before / After / How shape the repository's pull requests use, listing every gate run and its result.
- Every behaviour change starts with the EARS rule and scenario (
ears-gherkin-dev), classifies itself per SEMVER.md before the commit, and never lowers a floor. - Stop and write up rather than decide when: a breaking change is needed outside Phase 7's
feat!:commit; an existing floor would have to fall; a public export would be renamed or removed; a run's rule reveals wrong behaviour; anything wants the core to import the SDE; a step needs a repository secret, a setting or a token only the maintainer holds. - On finishing a unit, update this file's phase table and the release gate in the same pull request, so the next session starts from the truth.
Tracking
Every open item above has a GitHub issue carrying release::11.0 and its phase label; Track S runs carry phase::s-sde. The beads tracker mirrors them: file each with bd create from a checkout that has bd, one bead per issue, titled [Run n] <PR title> or [Phase n] <item>, labelled with the phase, and close it when the pull request merges. The scheduled weekly task and the sessions read markers in the repository, not the tracker, so a missing bead never blocks a run.
Revision rule. This roadmap is revised by pull request like any other file. A phase moves to Done only by citing the merged pull request or the check that proves it; a run's "done when" is not softened to fit a night. Adding a phase or run is a reviewed change; removing one needs a bead explaining why.
