REVIEWED PUBLIC SOURCE / HISTORICAL SNAPSHOT
REP-003: frozen H intervention
This is the exact UTF-8 source text at the recorded commit. The implementation repository is private. Older records may describe earlier access, hosting or release decisions; their original wording is preserved. Links and commands inside the record remain inert source text and do not require repository access to read this record.
- Original path
docs/HEAD_INTERVENTION_FINDINGS.md- Commit
e6d9e0584066ce0fb4cfb82088c1e7490a886e7b- SHA-256
a41d795a0f4c8db9b313b228da02143efacb71378f6f52a9827300b62dc9f34e- Size
- 8,682 bytes
Download the exact original bytes · Read the source
# REP-003: replacing H alone repairs the observed composition failures Replacing the context-sign-flip operation head, H, with its previously recorded training-only refit removed **all 2,062 length-16 errors**, with **zero new errors**, across five frozen Fourier models. Encoders, learned decoders and the other seven operation heads stayed unchanged. Every seed then answered every declared composition correctly at lengths 1, 2, 4, 8 and 16. This establishes a sufficient repair for these models and cases. It does not explain why training produced the original H matrices, establish a unique cause across possible interventions, or demonstrate general language understanding. ## Comparison and primary outcome The [frozen protocol](HEAD_INTERVENTION_PROTOCOL.md) implements R24 and R25 in the [specification](SPEC.md). Each of REP-002's five Fourier models was evaluated in four arms: native heads, H replaced alone, the other seven heads replaced, and all heads replaced. Replacements use the already saved least-squares matrices fitted only to the original training edges, cast from FP64 to FP32. There were **zero optimizer updates** and no new fit or decoder substitution. All twenty supervised RTX 5070 trials completed. The experiment reused the 98-state world and existing operation words. For length 16, each seed and arm has 3,136 cases: 98 initial states times 32 words. Percentages below measure exact equality of the complete decoded state, not individual coordinates. | Seed | Native | H only | Other seven only | All refitted | |---|---:|---:|---:|---:| | 0 | 94.23% | 100% | 93.53% | 100% | | 1 | 95.85% | 100% | 94.71% | 100% | | 2 | 92.06% | 100% | 94.23% | 100% | | 3 | 54.46% | 100% | 60.27% | 100% | | 4 | 97.64% | 100% | 98.31% | 100% | | Median | 94.23% | 100% | 94.23% | 100% | The identical marginal medians for native and other-seven-only conceal changes to individual answers. On the same 15,680 cases across the five seeds: | Intervention versus native | Correct stays correct | Correct becomes wrong | Wrong becomes correct | Wrong stays wrong | |---|---:|---:|---:|---:| | H only | 13,618 | **0** | **2,062** | 0 | | Other seven only | 13,433 | **185** | **398** | 1,664 | | All refitted | 13,618 | **0** | **2,062** | 0 | H-only repairs by seed were 181, 130, 249, 1,428 and 74. The other-seven-only arm improved three seeds, reduced accuracy in two, and left 1,849 errors in total. Its net gain of 213 answers does not erase its 185 regressions.  The [machine-readable summary](evidence/head-intervention-summary.json) retains all seeds, support strata, paired differences, all shorter lengths, H-count groups and relations. The registered factorial accuracy contrast `all_refit - h_only - others_only + native` has median -0.67 percentage points and mean -1.36 percentage points. It is descriptive and depends on this outcome scale; it does not identify a universal interaction mechanism. ## What the intervention preserved and what it repaired Checkpoint verification confirmed exact preservation of encoder buffers, decoder tensors and unselected heads. The native arm reproduced its saved parent arrays exactly. The all-refit arm reproduced the parent's refit metrics within the frozen tolerances. Each selected tensor matched its recorded FP32 refit, with the replacement mask retained in the checkpoint. Every retained H-free latent and decoder output stayed **bit-for-bit unchanged** between native and H-only execution within each backend. The independent review checked this on CUDA outputs and NumPy outputs separately, for all six saved case families and five seeds: 60 checks. The six families contain 42,630 H-free cases across the five seeds, including reconstruction. This is an isolation control, not another sample of new tasks. In the worst native seed, H followed by H had only 69.39% decoded agreement with the identity branch; H-only replacement restored 100%. The latent H-squared residual fell to approximately 7e-16 in every seed. These relations provide a mechanism-specific check alongside direct comparison with the true final state. Relation agreement alone would be insufficient, since two branches can agree while both are wrong. Exact decoded answers do not mean exact latent trajectories. At length 16, the median across seeds of mean normalized distance to the correct latent code fell from 0.26360 to 0.04614 with H-only replacement. With all heads refitted it was about 1.15e-8. Distances are Euclidean distances divided by sqrt(10). Thus H-only reaches the correct discrete answer despite residual numerical error from the other heads. The decoder's regression scores are not calibrated probabilities. In the proposed commuting-diagram language, the experiment identifies a particular operation implementation whose replacement restores every tested decoded consequence. It does not establish exact commutation everywhere. No additional latent dimension or context variable was needed for these cases: the ten supplied Fourier coordinates and learned decoder were already adequate for the observed repair. ## Verification and resources The [independent NumPy replay](evidence/head-intervention-replay.json) passed all twenty trials at the predeclared continuous tolerances of rtol=1e-4 and atol=1e-6, with exact categorical agreement: - All **331,240 saved decoded answers** matched; zero categorical disagreements. - All **8,631,840 scalar comparisons** passed; maximum absolute difference 7.15e-7 and maximum tolerance ratio 0.697. - All 75 paired answer-transition rows and 60 H-free checks passed. - All 320 relation comparisons passed at the aggregate level. Per-example CUDA relation arrays were not retained, so this is not a per-example relation replay. Replay used a separate NumPy inference implementation on the same host. It is not independent-host replication. REP-002's failed continuous replay for four learned-coordinate trials remains unchanged; REP-003's pass does not relabel it. The full suite passed **693 tests plus 23 subtests**, with zero skips and one expected Ripser warning. The [console log](evidence/head-intervention-tests.log) and [JUnit report](evidence/head-intervention-tests.xml) are retained. JUnit has 693 testcase elements and reports 716 cases including passing subtests. [Mathematical checks](evidence/head-intervention-math.json) also passed. The official worker ran for 19.49 seconds; supervised execution plus verification took 33.06 seconds. Peak allocated CUDA memory was 34.67 MiB, reserved CUDA memory 36 MiB, and host process-tree RSS about 1.25 GiB. This was actual deterministic FP32 execution on the RTX 5070 with TF32 disabled. It is small frozen-model inference, so its duration and memory use are not training benchmarks or GPU speed comparisons. There was one official attempt and no sizing probe. ## Evidence and next experiment Official attempt: `G:/RenigmaLLM_Artifacts/head-intervention-runs/rep-479f633a9e3c`. It retains 130 receipt-listed payloads, including the frozen protocol, source, parent inputs, checkpoints, numeric outputs and control results. - Runtime SHA-256: `f5c1049dcc2280dc40e1248641f02b1e1e66adce72b97dbe00a05c2eb8d4a8a9`. - Receipt SHA-256: `10c1a7ec00ddb79ed11a919c4962586bf7be80abd15e10b411e0352ca082330f`. - Parent receipt: `23470a50894fd6ec6bb3f1daefd2becf662a8722d2fa867c8bf43dd8dc13cfc1`. The [review ledger](evidence/head-intervention-review.json) binds the reviewed documents, scripts and evidence to these bytes. The local archive includes copied parent inputs and references the unchanged, separately retained REP-002 archive. Nothing has been publicly published. The next useful question is why fitting H left this error. A bounded follow-up should freeze the final encoder, decoder and non-H heads; initialize from each native H tensor; and compare the original prediction-plus-latent objective with latent-only fitting under identical, explicitly fresh optimizer states and fixed budgets. Native H and the recorded closed-form refit should remain zero-update references. Log training losses and gradients, then evaluate the same declared relations and compositions without selecting settings from those scores. Such a comparison can test recovery under each fitting rule. It cannot by itself separate the effects of extra optimization, an optimizer reset and the original joint training trajectory. A fresh-world replication remains necessary before claiming broader generalization or integrating the result into language tasks. This follow-up is proposed, not executed as part of REP-003.