Models

Zyphra's open-source ZYFA FP8 attention kernels run 1.16 to 1.76 times faster than AMD's AITER on MI355X in Zyphra's tests; the 1.76x figure is one model, and only prefill is covered

Zyphra's open-source ZYFA FP8 kernels run 1.16 to 1.76x faster than AMD's AITER on MI355X, per Zyphra. 1.76x is one model; prefill only.

By ·

Zyphra released ZYFA on November 5, 2026, open-source FP8 and BF16 attention kernels for AMD's Instinct MI355X GPU, with a paper on arXiv. The headline figure is up to 1.76 times the FP8 throughput of AMD's AITER library, but that is the best of three models Zyphra tested, and the other two are around 1.16 and 1.17 times. All results are Zyphra's own, and we found no independent replication.

The essentials

Reading the 1.76x figure

The top number is one layer of DeepSeek-V2-Lite at 16K tokens. The Zyphra paper stresses that throughput on random inputs does not predict throughput on real activations, because the MI355X's sustained clock depends on operand values, so it benchmarked on captured model activations. On Gaussian inputs at 32K tokens it reaches 2,672 TFLOP/s, 53% of FP8 peak against 39% for AITER by Zyphra's measurement.

What it changes and what it does not

FP8 doubles matrix throughput but not the softmax between the two matrix products, so attention becomes softmax-bound. ZYFA computes each FP8 probability directly from its score and overlaps softmax with matrix work using a two-wave ping-pong schedule. These are kernel-level gains for long-context prefill. End-to-end inference speedups were not reported.

What is not established

Summary table

Summary table
PointWhat is stated as of November 7, 2026Source
ReleaseNovember 5, 2026, by Zyphra, with an arXiv paperarXiv
Real-model speedup1.16x to 1.76x over AITER FP8, at 16K tokens and sustained clocksMarkTechPost
Error62 to 78% lower FP8 error than AITER at head dimension 128daily.dev
ScopeMI355X only, prefill only, ROCm 7.2.3 and PyTorch 2.12MarkTechPost
LicenseOpen source, Apache 2.0MarkTechPost

FAQ

Is it always 1.76 times faster?

No. That is the best of three tested models; the other two are about 1.16 and 1.17 times.

Does it work on NVIDIA or older AMD GPUs?

No. It targets the AMD Instinct MI355X only.

Is it free to use?

The code is open source under Apache 2.0, per MarkTechPost.

Method and limits

Sources checked on November 8, 2026. We read the arXiv abstract and press coverage, not the full paper or code. All benchmark figures are Zyphra's and were not independently verified. This is not investment advice.

Sources

Spotted an error in this article? Report it · Our corrections policy

OrigoTech logo

New: the app iPhone & iPad

OrigoTech is now an iPhone and iPad app

Read the articles natively, with “New” markers on unread ones, local search and offline reading of what you have already loaded. Free, no account, iOS 17 or later; ads appear between articles.

Rekha icon: a house drawn in a single stroke

From the same author Self-promotion

An Eulerian path, drawn with your finger

Rekha, the iPhone game by OrigoTech's author: connect every dot in a single stroke without tracing the same link twice, across 500 drawings that are all solvable.

The OrigoTech newsletter

A sourced weekly digest of AI tool news.

Read next