Models
Zyphra's open-source ZYFA FP8 attention kernels run 1.16 to 1.76 times faster than AMD's AITER on MI355X in Zyphra's tests; the 1.76x figure is one model, and only prefill is covered
Zyphra's open-source ZYFA FP8 kernels run 1.16 to 1.76x faster than AMD's AITER on MI355X, per Zyphra. 1.76x is one model; prefill only.
Zyphra released ZYFA on November 5, 2026, open-source FP8 and BF16 attention kernels for AMD's Instinct MI355X GPU, with a paper on arXiv. The headline figure is up to 1.76 times the FP8 throughput of AMD's AITER library, but that is the best of three models Zyphra tested, and the other two are around 1.16 and 1.17 times. All results are Zyphra's own, and we found no independent replication.
The essentials
- What it is: causal forward attention kernels for the prefill stage on the MI355X, with FP8 kernels written as hand-scheduled assembly generated by Python and BF16 kernels in HIP. The code is open source under Apache 2.0.
- The speedups: on activations captured from real models at 16K tokens and sustained clocks, 1.16x on Mistral-7B, 1.17x on Qwen3-8B and 1.76x on DeepSeek-V2-Lite, each measured on one layer. BF16 is roughly level at 1.01 to 1.02x.
- Accuracy: Zyphra reports FP8 numerical error 62 to 78% lower than AITER's at head dimension 128.
- Limits: it runs only on the MI355X with ROCm 7.2.3 and PyTorch 2.12, and covers prefill, not decode or paged KV.
Reading the 1.76x figure
The top number is one layer of DeepSeek-V2-Lite at 16K tokens. The Zyphra paper stresses that throughput on random inputs does not predict throughput on real activations, because the MI355X's sustained clock depends on operand values, so it benchmarked on captured model activations. On Gaussian inputs at 32K tokens it reaches 2,672 TFLOP/s, 53% of FP8 peak against 39% for AITER by Zyphra's measurement.
What it changes and what it does not
FP8 doubles matrix throughput but not the softmax between the two matrix products, so attention becomes softmax-bound. ZYFA computes each FP8 probability directly from its score and overlaps softmax with matrix work using a two-wave ping-pong schedule. These are kernel-level gains for long-context prefill. End-to-end inference speedups were not reported.
What is not established
- Independent replication of the benchmarks: not seen.
- End-to-end serving speedup: not reported.
- Decode, paged KV or training support: not covered.
- Other GPUs, including earlier AMD parts: not supported.
Summary table
| Point | What is stated as of November 7, 2026 | Source |
|---|---|---|
| Release | November 5, 2026, by Zyphra, with an arXiv paper | arXiv |
| Real-model speedup | 1.16x to 1.76x over AITER FP8, at 16K tokens and sustained clocks | MarkTechPost |
| Error | 62 to 78% lower FP8 error than AITER at head dimension 128 | daily.dev |
| Scope | MI355X only, prefill only, ROCm 7.2.3 and PyTorch 2.12 | MarkTechPost |
| License | Open source, Apache 2.0 | MarkTechPost |
FAQ
Is it always 1.76 times faster?
No. That is the best of three tested models; the other two are about 1.16 and 1.17 times.
Does it work on NVIDIA or older AMD GPUs?
No. It targets the AMD Instinct MI355X only.
Is it free to use?
The code is open source under Apache 2.0, per MarkTechPost.
Method and limits
Sources checked on November 8, 2026. We read the arXiv abstract and press coverage, not the full paper or code. All benchmark figures are Zyphra's and were not independently verified. This is not investment advice.
Sources
- arXiv: ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4 (verified 2026-11-08)
- MarkTechPost: Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster (verified 2026-11-08)
- FourWeekMBA: Zyphra Says Its AMD MI355X Attention Kernel Is 1.76x Faster (verified 2026-11-08)
- daily.dev: Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster (verified 2026-11-08)
Spotted an error in this article? Report it · Our corrections policy