The model proposed. The gates decided. refused

0 — What the model was allowed to see

A mapping proposal needs the shape of a value — is this an email, which date format, what prefix — never the value. So sensitive fields are replaced with type-preserving surrogates before the sample enters a prompt. The real file is read only by the deterministic layer, locally.

Surrogates are deterministic, so a value that repeats still repeats — cardinality survives, content does not. Dates, quantities, codes and company names pass through untouched, because those are the signal. This is not hypothetical: the Providence file carries a vendor_contct column of municipal suppliers' names and an e_mail_address column of their real addresses, and a test asserts none of those emails appears in the built prompt. The external results are unchanged at 5/5 with redaction on — the protection costs nothing, because the model never needed the values.

1 — The unknown source

incoming/warranty_registrations.txt · 1,100 rows · conventions the ETL was never taught

REG_NO|ITEM_SKU|BUYER_EMAIL|PURCHASED_ON|SALES_CH|COVER_YRS|ZONE
WR-202500001|SKU-P1005|diego.moreau267@example.com|14.03.2025|MKT|5|APJ
WR-202600002|SKU-P1097|anders.ferreira467@example.com|03.02.2026|WEB|2|LATM
WR-202500003|SKU-P1027|tomas.haddad270@example.com|11.06.2025|MKT|1|EMEA
WR-202500004|SKU-P1084|hugo.okafor224@example.com|21.12.2025|MKT|3|APJ
WR-202500005|SKU-P1105|freya.nilsson311@example.com|26.07.2025|MKT|1|LATM
WR-202500006|SKU-P1019|liam.barros756@example.com|15.06.2025|MKT|1| APJ 
WR-202500007|SKU-P1074|orders@lindqvist499.example.com|19.08.2025|BULK|1|APJ
WR-202500008|SKU-P1093|orders@kowalski533.example.com|22.05.2025|BULK|3|EMEA
WR-202500009|SKU-P1013|orders@andersen245.example.com|25.08.2025|BULK|1|EMEA
WR-202600010|SKU-P1001|orders@jensen148.example.com|23.06.2026|WEB|5|APJ

2 — The proposal (the voice)

Proposed via claude-cli · accepted on attempt 1 of 3 · 2026-08-30T19:24:44+00:00 · models in the CLI session: claude-haiku-4-5-20251001claude-opus-5[1m]

{
  "meta": {
    "backend": "claude-cli",
    "model": "claude-haiku-4-5-20251001/claude-opus-5[1m]",
    "created_utc": "2026-08-30T19:24:44+00:00",
    "source_file": "incoming/warranty_registrations.txt",
    "attempts": [
      {
        "attempt": 1,
        "model": "claude-haiku-4-5-20251001/claude-opus-5[1m]",
        "outcome": "accepted",
        "structural_problems": [],
        "failed_gates": []
      }
    ]
  },
  "proposal": {
    "delimiter": "|",
    "has_header": true,
    "columns": [
      {
        "source": "REG_NO",
        "target": "reg_id",
        "transforms": [
          "strip"
        ]
      },
      {
        "source": "ITEM_SKU",
        "target": "product_id",
        "transforms": [
          "strip",
          "upper",
          "strip_prefix:SKU-"
        ]
      },
      {
        "source": "BUYER_EMAIL",
        "target": "customer_email",
        "transforms": [
          "strip",
          "lower"
        ]
      },
      {
        "source": "PURCHASED_ON",
        "target": "purchase_date",
        "transforms": [
          "strip",
          "date:%d.%m.%Y"
        ]
      },
      {
        "source": "SALES_CH",
        "target": "channel",
        "transforms": [
          "strip",
          "upper",
          "value_map"
        ]
      },
      {
        "source": "COVER_YRS",
        "target": "warranty_years",
        "transforms": [
          "strip",
          "int"
        ]
      },
      {
        "source": "ZONE",
        "target": "region",
        "transforms": [
          "strip",
          "upper",
          "value_map"
        ]
      }
    ],
    "value_maps": {
      "channel": {
        "WEB": "Online",
        "SHOP": "Retail",
        "BULK": "Wholesale",
        "MKT": "Marketplace"
      },
      "region": {
        "AMER": "North America",
        "EMEA": "EMEA",
        "APJ": "APAC",
        "LATM": "LATAM"
      }
    },
    "notes": "Pipe-delimited export with day-first dotted dates, SKU- prefixed product ids, and short channel/region codes needing whitespace and case cleanup before mapping."
  },
  "approval": {
    "status": "approved",
    "by": "Burt Gardner",
    "at": "2026-09-02T19:01:34+00:00",
    "note": "verified against the Cobalt warranty spec",
    "proposal_fingerprint": "c0f37e69c2751f27"
  }
}

3 — The gates (the vote)

Deterministic, dependency-free Python — applied to the full file, not the sample the model saw. Recomputed live when this page was built.

4 — Measured across 50 unseen files

Each variant seed draws its own delimiter, date format, header vocabulary, prefix, code sets and column order — then the model faces it cold. Some are deliberately unmappable: the only correct outcome is refusal. Run with mapper/benchmark.py on claude-cli · 2026-08-30T20:57:33+00:00

5 — Real files, from real governments

Everything above uses files this project generates. These do not: they are fetched from public open-data portals in whatever format each publisher natively serves, and gated against a second contract whose ground truth is external fact — the actual list of US state codes, and the rule that one purchase-order number cannot belong to two vendors. 5/5 correct outcomes (2 accepted, 3 correctly refused).

Cost is measured, not estimated: this whole run — five files, 8 model calls including the retries on the refusals — came to $1.52 and 10,551 output tokens, recorded per call from the provider's own accounting.

The three refusals are the point: those files genuinely lack a line amount, a US state, or any notion of an order, and the model declined to invent one. Reproduce with python fetch_external_sources.py && python mapper/run_external.py.

Run it yourself

No API key needed: the default backend shells out to the Claude Code CLI on an existing subscription; any OpenAI-compatible endpoint (including a local Ollama) works via environment variables. CI never calls a model — it replays this recorded decision on every push.

git clone https://github.com/bgard68/BI-Simulator
python generate_sources.py
python generate_unknown_source.py
python mapper/propose_mapping.py     # live model call, records on success
python mapper/validate_mapping.py    # the gates, and the verdict