Six runs, and what each one settled

Result

A workspace with two languages mixed and templates cloned two to four times. Quarantine fell from thousands of nodes to 81 across a single type, and the whole run cost 141 LLM calls.

What it also showed

Nine defects surfaced. The most serious: the tenant bootstrap silently reset the evolved ontology on every pass, so the graph kept relearning types it had already earned, producing seven multi-parent concepts and a cycle in the taxonomy. It was caught by reading a health report that did not look right, fixed in three places, and the damaged meta-graph was repaired in place.

What was decided

Health output is read as a gate rather than as a report, so a silent regression in the governance layer surfaces on the next pass.

Result

The LLM proposes, a deterministic validator disposes. Unknown types quarantine on arrival and promote only by recurring past a threshold; on promotion the model proposes a parent and the validator sends a bad proposal to the root, so a promoted type is never left flat or dangling. Multi-level taxonomies came out of the material on their own: Thing › Product › Feature › Attribute, and Event › Process › Automation. A skeptic reviewer judged the band below the threshold and rejected six proposals as noise: Area, CRM Object, Element, Form, MarketingChannel, Vendor.

What it also showed

When the arc closed, the uncertain band had no reviewer: below-threshold types simply sat in quarantine, and the report records that in its open items. The skeptic agent was built afterwards as a separate change and verified end to end against a live model.

What was decided

Strict is not the same as complete-up-front. The schema is allowed to be slower than the data.

Result

Scoring is lexical-dominant by construction, 0.6 lexical against 0.4 semantic, so an embedding can confirm or lift a merge and never drives one alone. Conflicting types block a merge outright. On the real corpus the first large page minted 508 canonical entities against 427 merged variants; by the third pass a cloned template produced 11 new against 223 merged.

What it also showed

The measurement that shaped the design was a negative one. Bare-name embeddings could not separate AuraDB/Aura at 0.69 from Aurora/Aura at 0.70, a gap of one hundredth between a correct merge and a catastrophic one. Word-token matching rather than substring matching is what stops "RAG" being swallowed by "GraphRAG".

What was decided

Name-plus-context embeddings, scoring 0.865 against 0.650, were adopted because the measurement forced them. An independent review added the rule that an embedding may lift a pair to review but never merge it.

Result

Two hundred queries run twice: a cold pass opening a session per query, and a warm pass reusing one. Recall cleared the target comfortably at 0.992. Latency did not.

What it also showed

The p95 target was missed by 75ms. The two-pass design is what made that interpretable: warm and cold latency came out identical, which rules out session setup and TLS handshakes and isolates a cross-region network floor of roughly 205ms. The index is not the bottleneck; the distance between the client and a free-tier instance is.

What was decided

Recorded with the root cause attached rather than re-run in a closer region, because the root cause names what to change: the distance between client and instance, not the index.

Result

Valid-time records when a fact was true in the world; transaction-time records when the system learned it. On contradiction a fact is invalidated rather than deleted, so the window closes and the history stays queryable, and a backdated fact answers differently on the two axes. Retrieval threads a point-in-time filter that drops superseded edges, so an answer reflects one coherent moment.

What it also showed

The specification contradicted itself. One section named reified Event/Interval fact-nodes; another described superseded edges being invalidated. Two incompatible mechanisms had both been written down as decided, and resolving that came before any of it could be built.

What was decided

Edge-property bitemporality, which is Graphiti’s documented EntityEdge model replicated in-graph. That makes Graphiti a like-for-like benchmark rather than a runtime dependency. The trade-off is that edge properties cannot hold a reason for change.

Result

Each person’s employer exists only on a graph edge; the source text never names it. Graph completion projects that fact into context, while flat RAG sees chunk text alone. Both run through the same retrieval, grounding and answering LLM, with the projection stripped for the baseline, so the only variable is the graph fact. The margin clears zero on a paired bootstrap CI.

What it also showed

The scope is narrow by design: edge-only facts on a small set, which makes this a mechanism test rather than a benchmark. Scoring is deterministic token EM/F1 with no LLM judge, because a judge inside the loop makes the gate unfalsifiable. Public benchmarks plug into the same harness and are still to be run.

What was decided

Build the evaluation in-house rather than rely on vendor numbers. A validation pass had already caught a widely repeated vendor score that was wrong.