# SDE testing scorecard

What protects `src/sde`, measured. The Static Data Export module ships as its own sub-paths (`./sde`, `./sde/memory`), shares no code with the HTTP pipeline, and is scored, ratcheted and restored on its own in every tier the core has. This page is the SDE's copy of the core's [Where the suite stands](../TESTING.md#where-the-suite-stands) and [Where the scores stand](../TESTING.md#where-the-scores-stand): one table per tier, the command that reproduces it, and the floor that holds it.

The first three suite rows, the coverage and the specification counts were re-measured on 2026-10-08 after Run M merged; the mutation section says where each score comes from, and the remaining rows were measured on `e4daf989` on 2026-09-28. The SDE's own docs, the module's role and its isolation rule are in [SDE.md](../SDE.md).

## Where the suite stands

| Command                                                       | Config                               | Suites | Tests | Result                                                                                                                                                  |
| ------------------------------------------------------------- | ------------------------------------ | -----: | ----: | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `npx jest --config config/jest/unit.config.cjs tests/tdd/sde` | `config/jest/unit.config.cjs`        |     17 |   609 | All pass but one, about 7 s; the `./sde/memory` bundle test skips until `npm run build` has made `dist/`                                                |
| `npm run bdd:sde`                                             | same                                 |     23 |   227 | All pass, about 9 s; every scenario executes (`bdd:report`)                                                                                             |
| `npx jest --config config/jest/unit.config.cjs sde`           | same                                 |     43 | 1,001 | The two above plus `sde-bench`, `sde-drift` and `sde-spec-coverage` under `tests/tdd/`, about 22 s                                                      |
| `npx jest --config config/jest/fuzz.config.cjs sde`           | `config/jest/fuzz.config.cjs`        |      3 |   249 | All pass, about 18 s; 100 runs per property on a pull request, 10,000 in `nightly-properties.yml`                                                       |
| `npx jest --config config/jest/integration.config.cjs sde`    | `config/jest/integration.config.cjs` |      1 |   114 | Skip without `sde-data/`; all 114 pass against CCP's current export in [`nightly-sde.yml`](https://github.com/lgriffin/ESI.ts/actions/runs/36404108879) |
| `npm run test:types`                                          | tsd                                  |      2 |     — | `sde.test-d.ts` and `sde-memory.test-d.ts`, 801 and 603 lines of `expectType` / `expectError` assertions over the two entry points                      |
| `npm run test:consumer`                                       | `tests/consumer/`                    |      2 |     — | Steps 7 and 8: `./sde` loads without `js-yaml` and `adm-zip` and names the missing one; with them installed it reads real YAML and ZIP, CJS and ESM     |

The unit suites are one per module: the `IStaticDataProvider` contract (94 tests over the 96 methods, null and empty returns and the typing of each), `MemorySdeProvider`, `SdeDataProvider`, `SdeTestDataFactory`, `schemas`, `errors`, `clock`, `optionalPeers`, `order`, `index`, the `./sde/memory` bundle test, and five under `ingestion/` (`SdeDownloader`, `SdeExtractor`, `transforms`, `registry`, `index`).

| Coverage over `src/sde/**` (excluding `src/sde/testing/`) | Unit suites alone | Unit and BDD together | Floor (`config/jest/unit.config.cjs`) |
| --------------------------------------------------------- | ----------------: | --------------------: | ------------------------------------: |
| Statements                                                |            98.45% |                99.88% |                                   90% |
| Branches                                                  |            98.26% |                98.26% |                                   80% |
| Functions                                                 |            97.28% |                97.28% |                                   75% |
| Lines                                                     |            98.65% |               100.00% |                                   90% |

Reproduce with `npx jest --config config/jest/unit.config.cjs --coverage --collectCoverageFrom='src/sde/**/*.ts' --collectCoverageFrom='!src/sde/testing/**' tests/tdd/sde tests/bdd/specs/sde`. The floors are the repository's; the SDE has no separate ones, and its numbers sit above the core's on every line. Coverage says a line ran; the mutation table below says whether a test would notice it changing.

| Specification (`npm run spec:audit:verbose`, `features/sde/`) | Count                                                                                                                                                                      |
| ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Feature files                                                 | <!-- metric:featureFilesSde -->23<!-- /metric -->, all on the step library (`tests/bdd/specs/sde/`, `tests/bdd/steps/`); none on legacy `defineFeature` files              |
| `Rule:` requirements                                          | <!-- metric:requirementsSde -->156<!-- /metric -->, one EARS `shall` each                                                                                                  |
| Scenarios                                                     | <!-- metric:scenariosSde -->223<!-- /metric -->, a `Scenario Outline` counted once                                                                                         |
| Audit result                                                  | 23 of 23 pass; `scripts/spec/spec-audit-exceptions.json` lists no SDE file                                                                                                 |
| Method coverage (`npm run spec:coverage:sde:ci`)              | 96 of 96 `IStaticDataProvider` methods reached by a Rule or a bound step; `scripts/sde/sde-spec-coverage-baseline.json` is empty, so a method that loses its Rule fails CI |

The feature files run from `0001-static-data-lookup` to `0023-sde-ingestion`: the entity families in the interface's order (types, universe, market, dogma, characters, NPC organisations, presentation tables, industry, mission content, generic access), then loading, the optional peers, the `./sde/memory` entry and ingestion. The largest are `0023-sde-ingestion` (33 Rules, 43 scenarios), `0014-sde-npc-organisations` (16, 25) and `0020-sde-loading` (14, 19).

## Where the scores stand

Three mutation runs score the SDE, each with its own shard and its own floors, ratcheted the way the core's are (`guides/TESTING.md`, "Where the scores stand"): a complete nightly raises a floor to that night's score and never lowers one, and a pull request cannot lower or remove one.

Track S Run M ([#458](https://github.com/lgriffin/ESI.ts/issues/458), 2026-10-07 and 08) took every SDE directory to 90 or above on both the unit and the BDD-only run, one directory per pull request. Each pull request gave every surviving mutant a verdict: a new test or scenario kills it, or it is equivalent and carries `// Stryker disable next-line <Mutator>: <reason>` in the source, or it is listed below with the reason no test should name it. The scores are local runs with the nightly's own configs (`config/mutation/stryker.config.mjs` and `stryker.bdd.config.mjs`, `--mutate` narrowed to the directory), except the unit `src/sde` row, which Run M did not need to touch and which comes from the [nightly of 2026-09-28](https://github.com/lgriffin/ESI.ts/actions/runs/36396694021).

| Directory           | Unit score | Killed / valid | Unit floor | Pull request | BDD-only score | Killed / valid | BDD floor | Pull request |
| ------------------- | ---------: | -------------: | ---------: | ------------ | -------------: | -------------: | --------: | ------------ |
| `src/sde`           |      93.8% |          15/16 |       85.6 | (nightly)    |         100.0% |          15/15 |        93 | #611         |
| `src/sde/ingestion` |     100.0% |        184/184 |         98 | #608         |          97.6% |        162/166 |        97 | #612         |
| `src/sde/providers` |     100.0% |          92/92 |         98 | #607         |          90.5% |          86/95 |        90 | #610         |

The `src/sde/ingestion` figures include `SdeDatabaseBuilder.ts` and the SQLite transforms, removed after 12.2.0; the next nightly scores the directory without them.

Before Run M the same directories scored 93.8 / 73.9 / 89.6 on the unit run and 50.0 / 59.0 / 58.6 BDD-only. The unit `src/sde` floor still holds its provisional seed, under its score, until a complete nightly raises it.

`src/sde` is the root files: `clock.ts`, `errors.ts` and `optionalPeers.ts`. `src/sde/domain` holds only type declarations and Zod schemas and yields no valid mutants; `src/sde/testing` is excluded from both runs like `src/testing`. A mutant that crashes the runner (a missing-file check removed, a transform recursing on itself) counts as a runtime error and in no score. A unit run takes 10 to 15 minutes a directory on four cores; a BDD-only run over all of `src/sde` takes about 50, because the BDD dry run cannot narrow to related tests and runs every spec (`guides/TESTING.md`, "How it works" under Mutation testing).

Run M changed one behaviour on the way: the database builder took its columns from the first 50 records of a file, so a field that first appeared later was dropped from the table. It then scanned every record (#609); the builder and that Rule were removed with the SQLite path after 12.2.0. Nothing else in the source changed except equivalent-mutant annotations; the scores rose because the tests and scenarios grew.

The BDD-only survivors, each left on purpose because unit tests pin it and no scenario has a reason to name it:

| File                                | Survivors | Why no scenario kills it                                                                                                                                    |
| ----------------------------------- | --------: | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `providers/order.ts`                |         8 | `compareIds` on mixed number and string IDs (no SDE table mixes them), and the `<=` / `>=` / `0` forms that only differ for equal IDs, which keys never are |
| `providers/yaml/SdeDataProvider.ts` |         1 | The null skip when indexing a foreign key: no lookup asks for null                                                                                          |
| `ingestion/transforms.ts`           |         1 | The array guard in `extractLocale` (an array has no `en` key); Run M's second survivor, in `toSqliteValue`, went with the SQLite path                       |

The unit runs have no survivors in `providers` or `ingestion`; `src/sde`'s one is in `optionalPeers.ts`.

| Type mutation ([job](https://github.com/lgriffin/ESI.ts/actions/runs/36396694021/job/108875328069)) | Score | Killed / valid | Invalid | Candidates | Floor (`config/mutation/type-thresholds.json`) |
| --------------------------------------------------------------------------------------------------- | ----: | -------------: | ------: | ---------: | ---------------------------------------------: |
| `./sde`                                                                                             | 29.5% |         36/122 |      20 |      1,906 |                                     32 (below) |
| `./sde/memory`                                                                                      | 28.0% |         30/107 |      11 |      1,608 |                                             25 |

Type mutation rewrites the built `dist/sde/*.d.ts` (a field made optional, a union widened, a parameter's type loosened) and runs tsd per mutant, on a sample of the candidates each night. `./sde` scored 29.5 against a floor of 32 that Run 8 seeded from a local sample; the nightly's sample is the one that counts, so the floor stays and the gap is the next tsd work, not a lowered ratchet. The core's entry points score between 4.1% (`./schemas`) and 30% (`./errors`) on the same night, so both SDE entry points sit at the top of the table.

## The other tiers

| Tier                     | What holds the SDE                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Proven by                                                                                                                                                                                        |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Properties and fuzz      | `sde-provider-model.property.test.ts` drives `MemorySdeProvider` and `SdeDataProvider` against a model of the port over generated universes; `sde-transforms.property.test.ts` holds the ingestion transforms to their invariants (field-name normalisation is idempotent and reversible, locale extraction always answers a string, the record and metadata transforms match a naive recursive model); `sde-schema-fuzz.test.ts` throws arbitrary and near-valid records at every domain schema | Each model-based property must fail against its registered known-bad implementations, checked on every run (`tests/fuzz/support/property.ts`)                                                    |
| Real data, nightly       | `nightly-sde.yml` at 05:15 UTC downloads CCP's current export (cached per build), runs the 114 real-data tests with `SDE_REQUIRE_DATA=1`, `sde:drift` (the file list against the registry, every record's keys against its Zod schema), the four `examples/sde-*.ts`, then the six `sde/` benchmarks and the heap soak on the real export                                                                                                                                                        | Keeps one issue ("sde: nightly real-data run failed or the export drifted") open while any job is red; last green run [36404108879](https://github.com/lgriffin/ESI.ts/actions/runs/36404108879) |
| Isolation and layers     | `npm run lint:layers`: `src/sde` imports nothing of the package but `src/core/ports`, nothing outside it imports the SDE, and inside the module each layer imports only the layers listed for it in `SDE_LAYERS` (`config/eslint/layers.rules.cjs`): `ingestion` knows nothing of the providers, `domain` nothing of either; `memory-entry-bundle.test.ts` proves the shipped `./sde/memory` carries no file-system, YAML, ZIP or SQLite code                                                    | `tests/tdd/layers/layers-lint.test.ts` lints a negative fixture at each real path; the bundle test checks itself against a known string                                                          |
| Determinism              | `npm run lint:determinism`: `src/sde/clock.ts` is the module's only reader of time, and the baseline lists no SDE exception                                                                                                                                                                                                                                                                                                                                                                      | `tests/tdd/determinism-lint/`                                                                                                                                                                    |
| Benchmarks and heap soak | `tests/benchmark/sde.bench.ts`: load, `getType`, `getTypesByGroup` cold and indexed, name search by fragment and by whole name, on a seeded 50,000-type export; `sde-soak.ts` answers 100,000 lookups per cycle for three cycles and requires the heap back within 8 MiB or 5% after each `close()`                                                                                                                                                                                              | An injected slowdown and an injected leak must be flagged (`tests/tdd/benchmark/sde-bench.test.ts`); the tasks ride `ci.yml`'s `benchmarks` job and `nightly-benchmarks.yml`                     |
| Package surface          | `npm run size`: `./sde` 45.4 kB ESM / 47.9 kB CJS and `./sde/memory` 20.7 kB / 22.5 kB budgets (`.size-limit.cjs`); `npm run test:docs-examples` type-checks every TypeScript block in `guides/sde/`, `guides/SDE.md` and `src/sde/README.md` against the packed package; `npm run api-report` tracks both entry points in `etc/esi.ts.api.md`                                                                                                                                                   | `size-limit` fails over budget; the doc-examples tier has negative fixtures in `tests/tdd/doc-examples/`; `api-report:semver` fails on a lost line without a breaking commit                     |
| Mutation on every PR     | `mutation-pr` in `ci.yml` mutates the changed `src/` files, SDE files included, plus any neighbour the nightly's incremental report does not cover, and refuses a pull request that lowers or removes a floor; the nightly's `mutation:ratchet` and `mutation:bdd:ratchet` hold the three SDE directories to the floors above                                                                                                                                                                    | `tests/mutation-fixture/` must leave survivors and kill at least one mutant                                                                                                                      |

## What is open

- `./sde` type mutation sits 2.5 points under its floor on the nightly sample. The floor does not move; the fix is more `expectError` assertions where the survivors cluster (`reports/type-mutation/type-mutation.md` from the run names them).
- The Run M floors come from local runs. The next complete nightly re-measures all six and raises any floor that sits under its score; a nightly score under a floor fails the ratchet and is the next piece of work.
- Two behaviours recorded by the programme and left for the maintainer, not the tests: an empty search fragment matches everything up to the limit, and a search limit below 1 answers one record (`ROADMAP.md`, Track S).

Every floor here is a one-way ratchet: the next complete nightly raises a floor to its score, a pull request cannot lower one, and a directory without a floor fails the ratchet.