[jonfroehlich]

Track 3.4 of #263: the control arm, scored. The released weights were fine-tuned on GSV only for 2,000 steps at global batch 16, with three seeds (stage_two/run_finetune_control.slurm, already on main). The three checkpoints and the released one were scored on all 14 reviewed splits with the #82 scorer. Full write-up: docs/control_arm_263.md.

Result

The extra GSV steps do not improve the released model off GSV, and they cost about 0.01 F1 on the US splits at 0.30. Each cell below is the three-seed mean minus released (paired pano bootstrap, 95%), with each seed's own contrast beside it. The seed-mean interval resamples panos only and leaves out seed variance. At 0.55 the seeds disagree in sign, so the 0.55 seed mean is not a result of the recipe.

pool	read	seed mean	s1	s2	s3
open-rig 4 (laurens_mapillary, bayonne, juneau, budapest_district5; 469 panos)	F1 @0.30	-0.007 [-0.020, +0.005]	-0.005 [-0.021, +0.012]	-0.001 [-0.016, +0.015]	-0.017 [-0.030, -0.004]
open-rig 4	F1 @0.55	-0.032 [-0.045, -0.018]	+0.020 [+0.004, +0.037]	-0.048 [-0.066, -0.029]	-0.066 [-0.085, -0.049]
open-rig 4	max-F1	-0.004 [-0.015, +0.004]	-0.005 [-0.018, +0.006]	+0.004 [-0.009, +0.014]	-0.012 [-0.025, -0.001]
US pooled basis (9 splits, 1,078 panos)	F1 @0.30	-0.011 [-0.017, -0.006]	-0.013 [-0.021, -0.006]	-0.009 [-0.015, -0.002]	-0.012 [-0.018, -0.006]
US pooled basis	F1 @0.55	-0.018 [-0.024, -0.013]	+0.009 [+0.002, +0.017]	-0.034 [-0.042, -0.025]	-0.030 [-0.039, -0.023]
US pooled basis	max-F1	-0.007 [-0.012, -0.002]	-0.006 [-0.012, -0.001]	-0.004 [-0.010, +0.002]	-0.011 [-0.016, -0.004]
manual_gold (1,000 panos)	F1 @0.30	+0.003 [+0.000, +0.007]	-0.002 [-0.006, +0.002]	+0.009 [+0.005, +0.013]	+0.003 [-0.001, +0.008]
manual_gold	max-F1	+0.005 [+0.002, +0.008]	+0.005 [+0.001, +0.009]	+0.007 [+0.003, +0.010]	+0.004 [-0.001, +0.007]

• Seed-consistent reads. The US-pool loss at 0.30 is seed-consistent. So is the laurens_mapillary max-F1 drop: 0.710 falls to 0.653–0.664 for every seed, each interval below zero (seed mean -0.050 [-0.073, -0.025]).
• The Laurens rig gap widens from the GoPro side. The change is +0.044 [+0.010, +0.079] F1 at 0.30 and +0.022 [-0.002, +0.051] at max-F1 (the max-F1 interval includes zero). At max-F1 the GSV arm does not move (laurens_gsv +0.004 [-0.013, +0.025]).
• What replicates Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82. The laurens_mapillary max-F1 drop (-0.050 here, -0.057 in Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82) and the small loss on the US splits replicate. The rig-gap widening replicates at 0.30 but only weakly at max-F1, which was Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's headline read: +0.028 [+0.003, +0.064] in Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82, +0.022 [-0.002, +0.051] here. These are three new runs, not three new seeds: seeds 1 and 2 are Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's seeds again.
• Seeds mostly move the operating point. At 0.55 the open-rig F1 runs from 0.571 to 0.658 across seeds. At max-F1 the seeds are within 0.017, though one pair is resolved: s3 - s2 is -0.017 [-0.028, -0.004] on open-rig 4.
• Same-seed reruns against Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's controls are confounded. They differ in dataset copy, commit and GPU, and Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's seed 2 was also resumed mid-run. Seed 1 lands close; seed 2 differs by +0.025 [+0.009, +0.041] at 0.55 on open-rig 4. This suggests, but does not show, that the seed sets the operating point more than run-to-run variation does.
• Proposed, not decided: read later arms against the same-seed control, with max-F1 and the per-seed contrasts beside every fixed threshold.
• Unbiased subset (top stratum dropped): same signs throughout. The released re-score is bit-identical to Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's klone caches on the 12 splits both scored.

Not run: lyon (no reviewed GT; review open on #159). Everything else in the 14 was run. The cross-GPU-type scoring difference was not measured; see Caveats in the doc.

What is in this PR

• scripts/analysis/control263_score.slurm: scoring launcher, a thin wrapper around aug82_score_ckpts.sh. That script gains a LABEL_PREFIX variable whose default aug82_ leaves every Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 call unchanged.
• scripts/analysis/control_arm_263.py: the read. It imports aug_finetune_82.py's statistics and adds per-seed and 3-seed-mean contrasts, seed spread, pools, the unbiased subset, the Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82-controls rerun read and a determinism check. --check passes.
• analysis_out/control_arm_263/:
  • 56 extract caches (4 checkpoints × 14 splits), plus the Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 controls' bayonne/juneau top-up
  • scoring logs and results.json / results.md
  • trainlogs/: the six training jobs' .out, .err heads and progress lines, plus checkpoint sha256s, in Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82's layout
• docs/control_arm_263.md: commands, inputs and hashes, numbers, caveats, gaps.
• Compute: 11 klone rows in analysis_out/compute_log.jsonl from the committed sacct -D dump docs/data/compute/sacct_klone_2026-10-22_control263.txt, all on ckpt-all, $0. Training took 34.20 GPU-h, including 3.48 GPU-h for the three runs that failed on node g3110 (ECC). Scoring took 3.10 GPU-h. Section added to docs/compute_cost.md; tests/test_slurm_usage.py extended.
• tests/test_control_arm_263.py.

The checkpoints are only on klone (/gscratch/makelab/jonf/control263/, sha256 in the doc). Everything here re-derives on CPU from the committed caches.

Refs #263, #82, #159.

🤖 Generated with Claude Code — Opus 5.5, claude-opus-5-5

[jonfroehlich]

Independent review of #271 (head b94006f)

Verdict: approve once the M items are fixed. Nothing blocking. The numbers are right. Every number in docs/control_arm_263.md and in the #263 results comment matches results.json. The read re-derives independently, the ledger is exact, and the aug82_score_ckpts.sh change is a no-op for #82. The M findings are about prose that claims more than the data shows, plus one replication gap (training logs not committed). None of them changes the main conclusion: GSV-only steps do not help off GSV, and laurens_mapillary max-F1 drops for every fine-tune.

What I verified (no finding)

• control_arm_263.py --check passes on Windows from a clean checkout of b94006f ("control arm results up to date"). Tests: pytest -q tests/test_control_arm_263.py tests/test_slurm_usage.py gives 64 passed. CI is green.
• Independent re-derivation. I wrote a separate script that uses only rampnet.detection_eval.score_pano and the raw cache JSON. It has its own micro-pooling, its own max-F1 (block ends of descending confidence), its own stratified pano bootstrap and its own RNG. The observed values match exactly, and the CIs agree to within 0.001.
  • laurens_mapillary max-F1: released 0.7098, s1 0.6638, s2 0.6529, s3 0.6639, Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 s1 0.6681, Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 s2 0.6375 (exact).
  • open-rig 4, seed mean minus released, dF1 @0.55: −0.0315 [−0.046, −0.017], against −0.032 [−0.045, −0.018] committed. dR @0.55: −0.034 [−0.049, −0.019].
  • US 9, dF1 @0.30: −0.0114 [−0.017, −0.006]. Unbiased subset: −0.0136 [−0.020, −0.008]. Both match.
  • manual_gold dF1 @0.30 +0.003 [+0.000, +0.007] and max-F1 +0.005 [+0.002, +0.008]. Rerun s1 on US @0.30: −0.0015 [−0.004, +0.001]. s3 − s1 on open-rig 4 @0.55: −0.086 [−0.107, −0.066].
• Pairing. The pano sets and the GT are identical, pano for pano, across released, all three controls, both Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 controls (the Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 directory and the bayonne/juneau top-up) and op_cache, on all 14 splits. So the paired contrasts compare like with like, and the pids asserts in observed_and_se / seed_mean_contrasts hold.
• Pools and subset. open-rig 4 is 94+125+125+125 = 469 panos, and US 9 is 1,078. The unbiased subset drops exactly 5 per split (449 and 1,033); manual_gold is unchanged. The seed mean averages per-seed metrics under one shared pano resample, as the doc says. The af.SEEDS swap around laurens_rig_effect is restored in finally.
• aug82_score_ckpts.sh. With the default LABEL_PREFIX=aug82_, --model-label is still aug82_<label>, and ${LABEL_PREFIX%_} prints aug82 in the driver.log header, so Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 calls are byte-identical. The top-up caches carry meta.model = aug82_control_s{1,2}, and nothing else in tests/ or scripts/ reads the changed lines.
• Ledger. I re-ran the documented sacct -X -D -P -n -j … on klone (read-only) today. Its sha256 is 43350ceb…, identical to the committed dump. All 11 rows are present, the 3 FAILED g3110 rows included.
  • GPU-h: 3.39 + 0.0444 + 0.0478 = 3.48 failed. 9.3956 + 9.3756 + 11.9422 completed, so training is 34.20.
  • Scoring: 0.7908 + 0.8131 + 0.8131 + 0.5281 + 0.1511 = 3.10. Total 37.29, $0. Every wall-clock in the compute_cost.md table matches ElapsedRaw.
• Checkpoints. sha256sum -c SHA256SUMS in /gscratch/makelab/jonf/control263/ gives OK for all three, and the hashes equal the doc's.
• Gaps. Lyon not run and checkpoints only on klone are both stated next to the numbers.

Findings

#	sev	where	finding	concrete failure scenario
1	M	docs/control_arm_263.md:29-31; PR body table; #263 comment table, "dF1 @0.55" column	The 0.55 seed-mean effect is not seed-consistent, but it is presented as a resolved loss ("−0.032 [−0.045, −0.018] … The 0.55 loss is recall"). Per-seed vs released on open-rig 4 @0.55: s1 +0.020 [+0.004, +0.037], s2 −0.048, s3 −0.066. On US 9 @0.55: s1 +0.009 [+0.002, +0.017], s2 −0.034, s3 −0.030. The seed F1 range at 0.55 (0.086 on open-rig 4) is about six times the pano half-width (0.014). The interval excludes zero only because seed variance is not in it. The Caveats section says that in general, but 0.55 is the one column where it flips the reading.	A later arm is judged against "the control loses 0.03 F1 at 0.55 on the open rigs", or someone repeats the claim in the paper. One of the three controls actually gains at 0.55. Fix: put the per-seed signs (or "not seed-consistent") next to every 0.55 number. The 0.30 US result (−0.013 / −0.009 / −0.012, each interval excluding zero) and the laurens_mapillary max-F1 drop (all five runs lower) are seed-consistent and can stay as stated.
2	M	docs/control_arm_263.md:38-39	"GSV training helps the GSV arm of Laurens (laurens_gsv +0.062 [+0.030, +0.094] F1 at 0.55) and not the GoPro arm" rests on the 0.55 cell alone. laurens_gsv is +0.008 [−0.018, +0.034] at 0.30 and +0.004 [−0.013, +0.025] at max-F1. By the doc's own rule (max-F1 beside every fixed threshold), the wider rig gap is a GoPro-arm loss (laurens_mapillary max-F1 −0.050 [−0.073, −0.025]), not a GSV-arm gain.	A reader concludes that GSV replay improves GSV-rig recall at Laurens and that the gap widens from both ends. At max-F1 the GSV side does not move. Fix: say that the gap widens because the GoPro arm drops, and give the GSV arm's max-F1.
3	M	docs/control_arm_263.md:49-50, 180-191; PR body bullet 3; #263 comment item 3	The same-seed rerun is confounded, and the prose overclaims from it. Besides the resume, #82's controls differ from these runs in three ways: • Dataset bytes. They trained on the old rampnet_dataset, whose byte provenance is "not checked" (aug_transfer_82.md:509-514). This arm trained on rampnet_dataset_v2, the raw parquet bytes, not q95 re-encodes (restage_263.md:37). Data order is the same (train.py:474 sorts). • Commit. 6e05943 / 81d3afa there, 20ea575 here. • GPU. #82 control_s1 ran on 4×A40 (job 41123606). This s1 ran on 4×L40S. #82's s2 ran on L40S in its first incarnation and on L40 in its last. The doc names only the resume as a possible cause of the seed-2 gap. And "the rerun below shows that this comes from the seed, not from run-to-run noise" rests on one pair. The same-seed s2 difference is +0.025 [+0.009, +0.041] @0.55 and +0.013 [+0.002, +0.021] at max-F1 on open-rig 4. The s1 rerun is also resolved at 0.55 on open-rig 4: +0.006 [+0.001, +0.011].	A reader takes "same-seed rerun" as a pure determinism test and treats run-to-run noise as negligible when planning same-seed pairing for later arms. On open-rig 4 at 0.55 the seed-2 rerun moved by about a third of the seed-to-seed range. Fix: list the dataset-copy, commit and GPU differences beside the rerun numbers, and soften "shows" to "suggests".
4	M	docs/control_arm_263.md:213-214	"The determinism check bounds cross-GPU scoring differences at about 1e-4" is not measured. Both references are A40-vs-A40: this released re-score ran on an A40 (caches/driver.log, job 42152649, g3046); #82's released ran on an A40 (aug_transfer_82/finetune/driver.log:3, g3076); op_cache is makelab2's A40. control_s3 was scored on an L40S (job 42152652), and no comparison anywhere crosses GPU types.	Someone later scores an arm on a different GPU type than its control, relying on this sentence, and a few near-threshold detections flip. Fix: drop the bound, or re-score one split of one checkpoint on the other GPU type (minutes on ckpt-all) and quote the measured difference.
5	M	docs/control_arm_263.md:69-71; replication rule in CLAUDE.md	The training-run provenance is only on klone. The claims "commit 20ea575, the one untracked path was logs/", "no preemption and no resume", the data root, the init hash, and the g3110 ECC failure are all backed only by /gscratch/makelab/jonf/RampNet263/logs/control263_42*.{out,err}. I read control263_42108717.out: restarts: 0, commit 20ea5753 dirty files: 1, data root rampnet_dataset_v2, init 024a987c, so the claims are true. #82 committed the equivalent files under analysis_out/aug_transfer_82/finetune/trainlogs/.	Someone checking from a clean clone cannot confirm the seed, commit, no-resume status or dataset of the three checkpoints every later #263 arm is read against. Fix: commit the six .out files plus err heads / progress tails, as #82 did. They are about 3 KB each, and the .err files are 1.5 MB of progress bars, so commit heads and tails only.
6	M	docs/control_arm_263.md:52-55; docs/README.md new row; PR body	The reading rule is marked "(proposed)" in the #263 comment, but it is stated as settled in the doc ("What this means for the later arms. Read each arm against the same-seed control… report max-F1 next to every fixed-threshold number"), in the README row ("so later arms are read against the same-seed control") and in the PR body.	A later arm's PR cites the doc as the agreed protocol before Jon has decided it. Fix: mark it "proposed" in the doc and the README too.
7	N	docs/control_arm_263.md:41; #263 comment item 1 ("with new seeds")	"Three new seeds" contradicts line 180 ("this recipe and these seeds (1 and 2)"). Seeds 1 and 2 are #82's seeds, so these are new runs.	Reads as a five-seed design when it is three seeds, two of them run twice.
8	N	PR body, "This repeats #82"; doc:41-43	#82's rig-gap headline was at max-F1: +0.028 [+0.003, +0.064]. Here, max-F1 gives +0.022 [−0.002, +0.051], so the interval now includes zero. The PR body cites only the 0.30 cell (+0.044) next to "This repeats #82". The laurens_mapillary max-F1 drop does replicate (−0.050 here, −0.057 in #82).	"Repeats" overstates the rig-gap part. Say which parts replicate.
9	N	doc:45-46; PR body bullet 2	"Seeds … agree within 0.017 at max-F1" is true as a range, but s3 − s2 at max-F1 is resolved: −0.017 [−0.028, −0.004] on open-rig 4 and −0.007 [−0.012, −0.000] on US. The seeds differ modestly in ranking too, not only in operating point.	Minor. "Mostly" already hedges this; one clause would make it exact.
10	N	tests/test_control_arm_263.py:31-42	The pano-set and meta test covers caches/ but not caches_aug82_controls/, which feeds every rerun and five-run row. I checked by script that those pano sets and GT match op_cache, so this is about locking it in, not a bug.	A short top-up cache would only surface as a pids assert in --check, not as a named test failure.
11	N	#263 comment, Compute bullet	"3.39 of it seed 1 building the env tarball": 3.39 GPU-h is the whole failed job. The tarball build was 3,004 s of its 3,051 s (about 3.34 GPU-h). The doc and compute_cost.md word it correctly.	Cosmetic.

Verdict

Approve after the M fixes (1-6). Nothing is blocking. The statistics, pooling, unbiased subset, max-F1, threshold handling, determinism check, caches, ledger and scorer change are all correct and reproducible. The fixes are wording and one small commit of training logs. Finding 4 needs either a rewording or about 10 GPU-minutes. I did not modify or merge the branch.

🤖 Generated with Claude Code — Opus 5.5, claude-opus-5-5

[jonfroehlich]

Review fixes applied (b83a4a7)

CI passes on b83a4a7 (checks, pytest 3.10, pytest 3.12). These fixes respond to the review at #271 (comment). No number changed. control_arm_263.py --check reports "up to date", and the targeted tests pass: 107 passed across test_control_arm_263.py, test_slurm_usage.py, test_docs_index.py and test_aug_finetune_82.py.

I updated the doc, the docs/README.md row and the PR body. I edited the #263 results comment in place (#263 (comment)) and added a dated correction note at the top.

#	sev	status	what changed
1	M	fixed	The Answer section, PR body and #263 comment now give each seed's interval next to the seed mean. On open-rig 4 at 0.55: s1 +0.020 [+0.004, +0.037], s2 −0.048, s3 −0.066. They also say the seed-mean interval leaves out seed variance and that the 0.55 seed mean is not a result of the recipe. "The 0.55 loss is recall" is removed. The doc has a new per-seed table (5 pools × F1 @0.30 / @0.55 / max-F1). The pooled table caption flags the 0.55 columns as not seed-consistent. The per-split caption notes that the signs disagree on 8 of 14 splits at 0.55. A test now locks the per-seed 0.55 cells into the doc.
2	M	fixed	The rig-gap widening is now attributed to the GoPro arm: at max-F1, laurens_mapillary −0.050 and laurens_gsv +0.004 [−0.013, +0.025]. laurens_gsv's +0.062 at 0.55 is labelled an operating-point effect (per seed +0.139 / +0.001 / +0.045; +0.008 at 0.30). The max-F1 change +0.022 [−0.002, +0.051] is marked as including zero.
3	M	fixed	The rerun section is retitled "confounded". It lists the four differences: dataset copy, commit (6e05943 / 81d3afa vs 20ea575), GPU (#82 s1 on 4×A40, job 41123606; #82 s2 on L40S then L40, job 41138715) and the resume. It now reports the s1 rerun's resolved 0.55 gap on open-rig 4 (+0.006 [+0.001, +0.011]), and the s2 rerun's 0.55 gap (+0.025) and max-F1 gap (+0.013). "Shows" is now "suggests … does not show that run-to-run variation is negligible", in the doc, PR body and #263 comment.
4	M	fixed (reworded)	The ~1e-4 bound is gone. The caveat now says the cross-GPU-type scoring difference was not measured, that both determinism references were A40, and what would close the gap. I did not re-score anything.
5	M	fixed	analysis_out/control_arm_263/trainlogs/ follows #82's finetune/trainlogs/ layout. For each of the six jobs it has the full .out, an .err head (first 60 lines; the whole ECC traceback for the g3110 jobs) and a progress line every 200 micro-batches, plus ckpt_SHA256SUMS.txt. Total size is about 55 KB. A new test reads seed, restarts: 0, commit 20ea5753, data root and the copied checkpoint sha256 from the committed .out files. The Inputs section cites them.
6	M	fixed	The reading rule is labelled "(proposed, not decided)" in the doc, the README row, the PR body and the #263 comment.
7	N	fixed	"three new seeds" is now "three new runs; seeds 1 and 2 are #82's seeds again, seed 3 is new".
8	N	fixed	A "What replicates #82" paragraph separates the parts. The laurens_mapillary max-F1 drop (−0.050 vs −0.057) and the small US loss replicate. The rig gap replicates at 0.30 (+0.044 vs +0.046) but only weakly at max-F1, #82's headline read (+0.022 [−0.002, +0.051] vs +0.028 [+0.003, +0.064]).
9	N	fixed	The resolved max-F1 pair (s3 − s2: −0.017 [−0.028, −0.004] open-rig 4, −0.007 [−0.012, −0.000] US) is now stated. The text says the seeds differ a little in ranking too.
10	N	fixed	New test test_the_82_controls_top_up_scored_every_pano. It checks pano sets against op_cache and the meta (fp32, no TTA, floor, model label aug82_control_s{1,2}) for bayonne and juneau.
11	N	fixed	The #263 comment's compute line now says 42104181 is 3.39 GPU-h, of which 3,004 s of 3,051 s was the tarball build.

Not merged.

🤖 Generated with Claude Code — Opus 5.5, claude-opus-5-5

[jonfroehlich]

Re-review of #271 at b83a4a7

Verdict: ready to merge. All eleven findings from the first review are fixed. I checked each fix against the branch, results.json, my own re-derivation and klone (read-only), not against the fix notes. The fixes broke nothing. One new nit (N12) is optional.

Checks run on b83a4a7

• --check: control_arm_263.py --check reports "control arm results up to date" from a clean detached checkout.
• Tests: pytest -q tests/test_control_arm_263.py tests/test_slurm_usage.py tests/test_docs_index.py tests/test_aug_finetune_82.py gives 107 passed. CI is green on b83a4a7.
• Numbers: b94006f..b83a4a7 touches only docs, tests, .gitignore and the new trainlogs/. No script or committed result changed, so every number I re-derived in the first review still stands.
• Training logs against klone (read-only):
  • All six committed .out.txt files are byte-identical to /gscratch/makelab/jonf/RampNet263/logs/control263_42*.out (same sha256).
  • The .err_head.txt files match: for the three g3110 jobs, the whole 50-line .err; for the three completed jobs, the first 60 lines of .err after \r→\n.
  • The three non-empty .progress.txt files equal the first line at each 200-step mark in the klone .err (same sha256). The three failed jobs never trained, so their progress files are empty.
  • ckpt_SHA256SUMS.txt equals klone's control263/SHA256SUMS, and the new provenance test reads seed, restarts: 0, commit 20ea5753, data root and copied sha256 from the committed files.
  • The ECC traceback ([rank3]: RuntimeError: CUDA error: uncorrectable ECC error encountered) and the 3,004 s tarball build are in the committed 42104181 files.

Per-finding status

#	sev	status	verified against
M1	M	fixed	The per-seed table in the doc (lines 161-177; 5 pools × F1 @0.30 / @0.55 / max-F1) matches the contrasts block of results.json cell for cell, and my independent re-derivation from the first review. The Answer section, PR body and #263 comment all show each seed beside the seed mean and say the 0.55 seed mean is not a result of the recipe. "The 0.55 loss is recall" is gone everywhere. The sub-claims hold. "8 of 14 splits disagree in sign at 0.55": I count 8. The laurens per-seed values at 0.55 are +0.045 / −0.030 / −0.020 and +0.139 / +0.001 / +0.045. "Unbiased subset gives the same signs": I checked all 5 pools × 3 reads × 3 seeds, and every sign matches. "Seed-consistent at 0.30 on GSV tier and US pool": all six intervals are below zero. A test now locks the 0.55 per-seed cells into the doc.
M2	M	fixed	The widening is attributed to the GoPro arm in the doc, PR body and comment. laurens_gsv max-F1 is +0.004 [−0.013, +0.025], and its +0.062 at 0.55 is labelled an operating-point effect. The max-F1 interval is marked as including zero.
M3	M	fixed	The rerun section is titled "(confounded)" and lists dataset copy, commit (6e05943 / 81d3afa vs 20ea575), GPU (job 41123606 on A40; job 41138715 on L40) and the resume. All four match the #82 trainlogs. It reports the resolved s1 0.55 gap (+0.006 [+0.001, +0.011]) and the s2 gaps (+0.025, +0.013). "Shows" is now "suggests … does not show" in the doc, PR body and comment.
M4	M	fixed (reworded)	The ~1e-4 bound is gone from the doc and the comment. The caveat now says cross-GPU-type scoring was not measured, that both references were A40, and what would close the gap. That matches the driver logs.
M5	M	fixed	analysis_out/control_arm_263/trainlogs/ follows #82's layout and matches klone byte for byte (above). The .gitignore whitelist and the existing analysis_out/control_arm_263/** -text attribute cover it. The doc's Inputs section cites the files.
M6	M	fixed	The rule is "(proposed, not decided)" in the doc (line 75), "Proposed, not decided" in the docs/README.md row, and proposed in the PR body and the #263 comment. A grep for "later arms are read" and "should be read" finds nothing left in the doc, README or PR body.
N7	N	fixed	"three new runs, not three new seeds; seeds 1 and 2 are #82's seeds again" appears in the doc, PR body and comment.
N8	N	fixed	The "What replicates #82" paragraph is right. The #82 numbers it quotes match aug_transfer_82.md: rig gap F1 @0.30 +0.046 [+0.009, +0.084], max-F1 +0.028 [+0.003, +0.064], laurens_mapillary −0.057, US7 F1 @0.30 −0.011.
N9	N	fixed	The resolved s3 − s2 max-F1 pair (−0.017 [−0.028, −0.004] open-rig 4; −0.007 [−0.012, −0.000] US) is stated in the doc, PR body and comment.
N10	N	fixed	test_the_82_controls_top_up_scored_every_pano covers bayonne and juneau for both #82 controls (pano set, fp32, no TTA, floor, min distance, aug82_ label).
N11	N	fixed	The comment's compute line now reads "Job 42104181 is 3.39 GPU-h of that, and 3,004 s of its 3,051 s went to building the env tarball", matching the committed .out.
N12 (new)	N	open, optional	docs/control_arm_263.md:158 says seed 1 at 0.55 is "resolved on open-rig 4, open-rig 3 and the US pool". It is also resolved on manual_gold: +0.009 [+0.005, +0.014], in the doc's own table. Nothing wrong is claimed; the list is just incomplete.

Anything the fixes broke

Nothing found.

• The new doc text quotes only existing results.json cells, and I checked every new number above.
• The RampNet 2.0: open-rig detector campaign, phase 0 (miss buckets, Lyon review, training plumbing) #263 comment has a dated correction header that says no number changed, which is true.
• No claude.ai session links appear in the PR body, the doc or the comment.
• The docstring of control_arm_263.py still calls the Training augmentation is a single horizontal flip: measure downscale / photometric jitter as a transfer lever (downstream of #84) #82 comparison a "same-seed rerun", but it already names the earlier dataset copy and commit, so I am not re-raising it.

Verdict: ready to merge. N12 is a one-word nit and can go in with the merge or be left as is. Nothing on the branch was modified, and the branch was not merged.

🤖 Generated with Claude Code — Opus 5.5, claude-opus-5-5