# H1 real-text pilot v14 protocol (larger model, v12 categories)

Written before execution. Code: `h1_realtext_v14.py`, entry point `run_realtext_v14()`.

## Question
v13 (4 layers, width 128, comp/rec categories) showed a weak effect of the hierarchy code against the scrambled codes. Is the weak effect of v13 due to the size of the model or to the categories? v14 keeps the v13 model and changes only the categories, to those of v12 (sci vs politics/society).

## Data
Identical to v12: `SetFit/20_newsgroups`, frozen revision `f1b91292074e7cfb69be58b642d583ec262f30ed`, `train.jsonl` only (official test unused), whitespace-normalised UTF-8, first 96 bytes, 300 training (idx < 300) and 80 validation (idx 300..379) messages per category.
- group 0 (codes 0-3, "sci"): `sci.crypt`, `sci.electronics`, `sci.med`, `sci.space`
- group 1 (codes 4-7, "politics/society"): `talk.politics.guns`, `talk.politics.mideast`, `talk.politics.misc`, `soc.religion.christian`

`load_data()` is unchanged from v12, with the same eligibility check (>= 380 messages of >= 96 bytes per category; eligible counts on the frozen revision, as in the v12 protocol: sci.crypt 556, sci.electronics 542, sci.med 560, sci.space 547, talk.politics.guns 507, talk.politics.mideast 513, talk.politics.misc 427, soc.religion.christian 570). `selected_data_sha256` must equal the v12 value. Hierarchical code `category // 4`, `category % 4`.

Training size: `SIZES = (100,)` per category (nested subset via `subsample_train`); validation identical (640 messages).

## Model and training
Identical to v13: causal Transformer with `N_LAYERS = 4`, `D_MODEL = 128`, 4 heads, `FF = 256`, batch 24, AdamW lr 0.001, 800 updates, `DROPOUT = 0.2` (training only, eval mode at every evaluation, train mode restored), `WEIGHT_DECAY = 0.1`, TF32 off, 2-token prefix, batch sequence `torch.randint(len(train), (800, 24), generator=torch.Generator().manual_seed(seed + 30000))` identical for all conditions of a seed, identical initial weights for all conditions of a seed. The ONLY change versus v13 is the category set.

Seeds `SEEDS = (18163, 19169, 20173, 21179, 22189)`. Permutation seeds `PERMUTATION_SEEDS = (12012, 23023, 34034, 45045, 56056)`, fallback `(67067, 78078, 89089, 91919)`. No overlap with v10 (seeds 2027, 3037, 4049, 5051, 6067; permutations 31415, 27182, 16180, 14142, 17320 + fallback 11235, 13579, 24680, 97531), v11 (7079, 8081, 9091, 10093, 11099; 42424, 55555, 66666, 77777, 88888 + 99991, 12121, 23232, 34343), v12 (12101, 13103, 14107, 15109, 16111; 10101, 20202, 30303, 40404, 50505 + 60606, 70707, 80808, 90909) or v13 (13121, 14129, 15137, 16141, 17149; 11011, 22022, 33033, 44044, 55055 + 66066, 77077, 88088, 99099); checked by `out14/test_v14_logic.py`. Same rejection rule as v11-v13: a permutation is rejected if categories 0-3 all fall in the same code//4 group; a rejected primary seed is replaced by the next unused fallback seed, in order.

Checkpoints 100, 200, 400, 800, all reported; no checkpoint selection.

## Conditions (2-token prefix)
`none` [256, 256]; `hierarchy` [265 + category // 4, 267 + category % 4]; `scrambled_1..5` = hierarchy code of permutation `perm[category]`. 7 conditions x 5 seeds = 35 models.

## Pre-registered cells
Two cells: size 100 x checkpoint {200, 400} (keys `100_200`, `100_400`), taken from v10/v11/v12 (chosen AFTER seeing v10). `compute_replication_cells` reports `mean_paired_diff`, `seeds_hierarchy_better` (of 5), `mean_between_permutation_range`, `sign_negative`, `wins_at_least_4_of_5`, `material` (`mean_paired_diff < -mean_between_permutation_range`), plus `n_cells_sign_negative`, `n_cells_wins_at_least_4_of_5`, `n_cells_material`. Outputs: `summary`, `summary_label_use`, `replication_cells`, `reference_v12_n100`, `p5_vs_v12_indicative`. No significance test, no data-savings percentage.

Frozen reference (constant `REFERENCE_V12_N100`, copied from `the earlier run output`, `replication_cells['cells']['100_200'|'100_400']['mean_paired_diff']`, equal to `summary['100'][step]['paired_diff_hierarchy_minus_scrambled_mean']['mean']`; v12 = same categories, tiny model 2 layers / width 64, seeds 12101..16111): n=100 per category, 200 updates -0.0001591691970824982, 400 updates -0.0001796048482258783. Not recomputed.

## Predictions declared in advance
- P1 (sign): mean gap hierarchy - scrambled mean negative in BOTH cells. Prior: about 45%.
- P2 (regularity): hierarchy beats the scrambled mean on at least 4 seeds out of 5 in BOTH cells. Prior: 25%.
- P3 (materiality): at least 1 cell is `material`. Prior: 10%.
- P4 (sanity): the mean `label_use_gap` of `hierarchy` at 100 per category and 800 updates (`summary_label_use['100']['800']['hierarchy']`) is > 0. If not, the larger model does not use the label and v14 is uninformative.
- P5 (indicative, UNPAIRED): the mean gap at 400 updates (`100_400`) is more negative than the frozen v12 value at n=100/400 (-0.0001796). Other model and other seeds: comparison purely indicative (reported in `p5_vs_v12_indicative`).
- Reading rule: v14 against v13 (same model, other categories) and against v12 (same categories, other model): if v14 shows a larger gap than v13, the categories matter; if it stays small, the model size is not the lever either. No update of confidence in H1. No claim of significance. A sign that reproduces but is not material is reported as a "non-material tendency".

## Limitations
One larger model only (4 layers, width 128): not a scaling law. A single regularisation setting (dropout 0.2, weight decay 0.1), not retuned, which may under- or over-regularise the 4-layer model. The sci vs politics/society split may be more or less semantically clean than comp vs rec. Five seeds, five permutations, two cells that share seeds across checkpoints and are not independent. Comparisons with v12 and v13 are unpaired (other model or other categories, other seeds) and indicative only, so v14 alone cannot separate model size from categories. No significance test and no percentage of data saved. Labels are present at labeled evaluation. Cells chosen AFTER seeing v10. One historical text dataset. Not a pretrained-LLM or general data-savings result.

## Budget and running
1 size x 5 seeds x 7 conditions = 35 models of the v13 size. GPU guard 600 s, abort at 585 s without partial result. Fail-fast: after the first finished seed, the total is projected as elapsed x seeds / seeds_done; if it exceeds 585 s the run aborts at once. Both guards are disabled in custom (smoke-test) mode (`smoke_test_not_protocol`). Call `run_realtext_v14()` on a CUDA machine; smoke tests: `run_realtext_v14(device='cpu', steps=..., seeds=..., sizes=...)`.
