# H1 real-text pilot v6 protocol (longer training + label ablation)

Written before execution. Code: `h1_realtext_v6.py`, entry point `run_realtext_v6()`.

## Question
In v5, 200 updates did not separate the conditions, probably because the model had not yet learned to use the label prefix. Once trained longer (800 updates), does a tiny byte-level transformer (a) use the label prefix at all, and (b) does a hierarchical label prefix help more than a flat, scrambled or group-only prefix, in held-out validation NLL?

## Data
`SetFit/20_newsgroups`, frozen revision `f1b91292074e7cfb69be58b642d583ec262f30ed`, `train.jsonl` only (the official test split is unused). Same 8 categories (4 comp.*, 4 rec.*), same selection and encoding as v5: whitespace-normalised UTF-8, first 96 bytes, 300 training and 80 validation messages per category (640 validation messages). Selected-data SHA-256 is recorded in the result. Do not persist the messages; only aggregate numbers are saved.

## Model and training
Same randomly initialised 2-layer transformer as v5 (width 64, 4 heads, FF 128), AdamW lr 0.001, weight decay 0.01, batch 24, TF32 off. Seeds 2027, 3037, 4049 (paired). For a given seed, every condition gets the same initial weights and the same sequence of batches. 800 updates; checkpoints at 100, 200, 400, 800. Loss covers text bytes only.

## Conditions (2-token prefix)
- none: [256, 256]
- flat: [257 + category, 256]
- hierarchy: [265 + category // 4, 267 + category % 4]
- scrambled: hierarchy codes of a fixed permutation of categories (seed 31415)
- group_only: [265 + category // 4, 256] (comp vs rec only)

## Metrics
At each checkpoint, on the same 640 validation messages, in nats per byte (lower is better):
- `eval_nll_labeled`: prefix of the condition's own true label.
- `eval_nll_ablated`: prefix replaced by the dummy [256, 256] for every condition.
- `label_use_gap` = ablated - labeled. A gap clearly above 0 means the model uses the label; near 0 means it ignores it.
Per-seed values are reported, together with the paired structure; no significance test is run.

## Predictions declared in advance
1. If the labels are used, label_use_gap > 0 for flat, hierarchy, scrambled and group_only, growing with updates; for none it is about 0 by construction.
2. If the labels are used, hierarchy has lower labeled NLL than scrambled at 400 and 800 updates.
3. group_only lies between none and hierarchy.
4. A null result (gaps near 0, conditions not separated) is informative: it would mean 800 updates of this tiny model are still insufficient, or that prefixes of this kind do not help here. We will report it as such.

## Limitations
Three seeds, one permutation, short 2-token prefixes, one historical text dataset, a tiny model trained for 800 updates. Labels are present at labeled evaluation. No significance claim and no percentage of data saved are made. This is not a pretrained-LLM or general data-savings result. GPU budget is capped at 600 s; the run aborts rather than report a partial result.

## Running
Click "Run real-text v6 (longer + label ablation)" in the Space (CUDA required), then download the JSON.
