# H1 real-text pilot v13 protocol (larger model, v11 data)

Written before execution. Code: `h1_realtext_v13.py`, entry point `run_realtext_v13()`.

## Question
v10, v11 and v12 (tiny model: 2 layers, width 64) showed the hierarchy code very slightly better than the mean of the scrambled (mixed) codes at 200-400 updates (gaps of about -0.0002 to -0.0013 nats per byte), always below the between-permutation range (a non-material tendency). Does the gap hierarchy - mixed codes GROW with the size of the model, everything else being identical?

## Data
Identical to v11: `SetFit/20_newsgroups`, frozen revision `f1b91292074e7cfb69be58b642d583ec262f30ed`, `train.jsonl` only (official test unused), whitespace-normalised UTF-8, first 96 bytes, 8 categories (`comp.graphics`, `comp.os.ms-windows.misc`, `comp.sys.ibm.pc.hardware`, `comp.sys.mac.hardware`, `rec.autos`, `rec.motorcycles`, `rec.sport.baseball`, `rec.sport.hockey`), 300 training (idx < 300) and 80 validation (idx 300..379) messages per category. `load_data()` is unchanged, so `selected_data_sha256` must equal the v11 value. Hierarchical code `category // 4`, `category % 4`.

Training size: `SIZES = (100,)` per category only (nested subset via `subsample_train`); validation identical (640 messages).

## Model and training
The ONLY change versus v11 is the architecture: causal Transformer with `N_LAYERS = 4`, `D_MODEL = 128`, 4 heads, `FF = 256` (v11: 2 layers, 64, 4 heads, 128). Unchanged: batch 24, AdamW lr 0.001, 800 updates, `DROPOUT = 0.2` (training only, eval mode at every evaluation, train mode restored), `WEIGHT_DECAY = 0.1`, TF32 off, 2-token prefix, batch sequence `torch.randint(len(train), (800, 24), generator=torch.Generator().manual_seed(seed + 30000))` identical for all conditions of a seed, identical initial weights for all conditions of a seed. The regularisation is NOT retuned for the larger model.

Seeds `SEEDS = (13121, 14129, 15137, 16141, 17149)`. Permutation seeds `PERMUTATION_SEEDS = (11011, 22022, 33033, 44044, 55055)`, fallback `(66066, 77077, 88088, 99099)`. No overlap with v10 (seeds 2027, 3037, 4049, 5051, 6067; permutations 31415, 27182, 16180, 14142, 17320 + fallback 11235, 13579, 24680, 97531), v11 (7079, 8081, 9091, 10093, 11099; 42424, 55555, 66666, 77777, 88888 + 99991, 12121, 23232, 34343) or v12 (12101, 13103, 14107, 15109, 16111; 10101, 20202, 30303, 40404, 50505 + 60606, 70707, 80808, 90909); checked by `out13/test_v13_logic.py`. (14107 is a v12 seed and was therefore avoided.) Same rejection rule as v11: a permutation is rejected if categories 0-3 all fall in the same code//4 group; a rejected primary seed is replaced by the next unused fallback seed, in order.

Checkpoints 100, 200, 400, 800, all reported; no checkpoint selection.

## Conditions (2-token prefix)
`none` [256, 256]; `hierarchy` [265 + category // 4, 267 + category % 4]; `scrambled_1..5` = hierarchy code of permutation `perm[category]`. 7 conditions x 5 seeds = 35 models.

## Pre-registered cells
Two cells: size 100 x checkpoint {200, 400} (keys `100_200`, `100_400`), taken from v10/v11 (chosen AFTER seeing v10). `compute_replication_cells` reports `mean_paired_diff` (hierarchy - mean of the 5 scrambled, mean over seeds), `seeds_hierarchy_better` (of 5), `mean_between_permutation_range`, `sign_negative`, `wins_at_least_4_of_5`, `material` (`mean_paired_diff < -mean_between_permutation_range`), plus `n_cells_sign_negative`, `n_cells_wins_at_least_4_of_5`, `n_cells_material`. Outputs: `summary`, `summary_label_use`, `replication_cells`, `reference_v10_v11_n100`. No significance test, no data-savings percentage.

Frozen reference (constant `REFERENCE_V10_V11_N100`, copied from `the run summary`, section "Pooled v10+v11", exploratory, 10 seeds): n=100 per category, mean paired diff hierarchy - scrambled mean: 200 updates -0.0006, 400 updates -0.0006.

## Predictions declared in advance
- P1 (sign): mean gap hierarchy - scrambled mean negative in BOTH cells. Prior: about 45%.
- P2 (regularity): hierarchy beats the scrambled mean on at least 4 seeds out of 5 in BOTH cells. Prior: 25%.
- P3 (materiality): at least 1 cell is `material`. Prior: 10%.
- P4 (growth): the mean gap at 400 updates (`100_400`) is more negative than the reference v10+v11 value at n=100 (-0.0006). Prior: 35%. Reading: the comparison is UNPAIRED (other model, other seeds) and purely indicative.
- P5 (sanity): the mean `label_use_gap` of `hierarchy` at 100 per category and 800 updates (`summary_label_use['100']['800']['hierarchy']`) is > 0. If not, the larger model does not use the label and v13 is uninformative.
- Reading rule: no update of confidence in H1. A sign that reproduces but is not material is reported as a "non-material tendency".

## Limitations
One larger model only (4 layers, width 128): not a scaling law. A single regularisation setting (dropout 0.2, weight decay 0.1), not retuned, which may under- or over-regularise the 4-layer model. Five seeds, five permutations, two cells that share seeds across checkpoints and are not independent. The comparison with the v10+v11 reference is unpaired (other model, other seeds) and indicative only. No significance test and no percentage of data saved. Labels are present at labeled evaluation. Cells chosen AFTER seeing v10. One historical text dataset. Not a pretrained-LLM or general data-savings result.

## Budget and running
1 size x 5 seeds x 7 conditions = 35 models, each about 4-8x the per-step compute of v11 (about 6x in FLOPs, see report), so roughly 35/70 x (2-6) of the v11/v12 time. GPU guard 600 s, abort at 585 s without partial result. Fail-fast: after the first finished seed, the total is projected as elapsed x seeds / seeds_done; if it exceeds 585 s the run aborts at once. Both guards are disabled in custom (smoke-test) mode (`smoke_test_not_protocol`). Call `run_realtext_v13()` on a CUDA machine; smoke tests: `run_realtext_v13(device='cpu', steps=..., seeds=..., sizes=...)`.
