Chapter 03 · the measurement

The convergence measurement

The extractor was run five times on each of three documents and all fifteen runs produced a different graph. Measured against generation two's expectation that runs should converge, that was zero convergence. In the lens terminology the project adopted afterwards, each run is a lens, lenses are expected to differ, and identical ones would mean something was broken. What is still unmeasured is convergence under the new definition.

What was measured

Three documents: one Addis Fortune English article, one BBC Amharic article, one Ethiopian Reporter Amharic article. Five extraction runs each, on 2 August 2026. Assertions were canonically labelled for comparison: class, participant roles and labels, verbalized text; NFC-normalized, casefolded, whitespace collapsed; byte offsets and cited text excluded so that two runs citing the same fact from different spans would still match.

What came back

Every document produced five unique graph hashes. Per assertion class, the core (the intersection across all five runs) was zero in every class.

Fifteen runs, fifteen different graphs.

commit 3fdb48ab · the convergence artifact, deleted with the corpus, survives only in git history · retrieved 2026-08-21

What it meant at the time

Generation two expected runs to converge: the design treated an extraction as an attempt at the one graph a document contains, so five runs should have agreed on a core and the core would have been the graph. Against that expectation the result was zero convergence, and the record carries its own caveat even then: this measures stability, not truth; a stable core can be consistently wrong. It became doctrine: never report convergence as a quality number. On 11 August 2026 the operator ruled the corpus "nowhere near worked" and the architecture was rewritten.

What it means in the lens terminology

The rewrite changed the expectation, not the data. In the lens-graph architecture, an extraction is a lens: one viewpoint on a document, defined by a closed vocabulary, yielding one small instance. One document under several lenses yields several instances, and they are supposed to differ, because they are looking for different things. Five runs of an open-vocabulary extractor are five uncontrolled lenses, so fifteen different graphs is the expected result, not a failure. The operator's clarification of 22 August 2026 is the plainest statement of it: it is completely unsurprising that they are all different, and if they were the same something would be broken.

What the measurement did establish is that an open-vocabulary extractor cannot be treated as if it produced the graph of a document, which is exactly why lenses have closed vocabularies. The question the first pass asked ("do runs agree?") was the wrong question; the lens architecture replaces it with two better ones: does a lens find what its vocabulary names, consistently, across runs (convergence under a closed vocabulary, anchored by a hand-built exemplar instance produced before any generated extraction); and what do different lenses on the same document disagree about, which is information rather than error.

Neither has been measured yet. That is the open item.

user-beadwork/plans/PLAN_lens-graph-similarity_2026-08-11.md (lenses, convergence redefinition) · the operator's clarification, 2026-08-22 · build report §3.3 · retrieved 2026-08-22

Published August 22, 2026 at 15:13 UTC. Last updated August 22, 2026 at 19:14 UTC. Times come from this page's revision history and can be checked against it.