The desk
Die Brief

DIE.BRIEF

Hardware desk
Models MULTI-SOURCE

RTX 3090 inference stack lifts Qwen3.8-27B to 381 tokens per second

The Die Brief Desk, after x.com

A single RTX 3090 reportedly lifted Qwen3.8-27B from 82 to 381 tokens per second using inference optimizations, while ordinary chat stayed around 133 tokens per second.

Picture from wccftech.com
Picture from wccftech.com
RTX 3090
baseline82 tokens/sec
optimized MTP114 tokens/sec
DFlash2 + context lookup138 tokens/sec
optimized lookup verification381 tokens/sec
ordinary chat133 tokens/sec
document task260 to 382 tokens/sec
time to first token22.4s to 0.56s
VRAM24GB

The X post puts Qwen3.8-27B at 381 tokens/sec on one RTX 3090 after inference-stack changes. Wccftech’s article calls the same run a "near-5x gain" while also naming 381 tokens/sec. That post is the document I trust, and 381 tokens/sec is the number that decides it.

The post’s ladder is specific: baseline 82 tokens/sec, optimized MTP 114, DFlash2 plus context lookup 138, and optimized lookup verification 381. On a 25,000-token document task it reports 260 to 382 tokens/sec, while ordinary chat stays around 133. Prefix caching drops time to first token from 22.4 seconds to 0.56 seconds, but that is a repeated-prefix condition, not a general speed claim.

The constrained part is the verification path, not the silicon. The 381 tokens/sec figure only appears when the model is checking blocks of up to 16 tokens and accepting about 15, so it is a reuse workload, not a chat benchmark. For a buyer, the RTX 3090 becomes useful for document-heavy, repeated-prefix tasks; for a fab, nothing changes, because the gain is software-side and does not prove a new memory or compute limit.

What is still unmeasured is power draw, thermal headroom, and whether the same 381 tokens/sec survives outside the developer’s stack. The document does not name a benchmark suite, a second GPU, or a reproducible dataset large enough to separate luck from architecture.

The document does not measure power draw, thermal limits, or a second GPU under the same stack.

The gain comes from speculative verification and context reuse, so the RTX 3090’s 24GB VRAM is not the limiting factor in the peak case. Buyers should treat 381 tokens/sec as a narrow document-reproduction result, not a general chat throughput.

After x.com. We did not report this. The pictures, if any, are theirs.

October 18, 2026 · 2 min
Copy link Share to X