Qwen4-Max arrived on November 16 with a million-token context window and a redesigned architecture. The model many people want to run at home is the 27B. As of November 18, 2026, Alibaba has promised smaller open-weight Qwen4 models, and there is public development evidence for a dense Qwen4-27B. The final 27B specs, licence and release date are still unpublished. Alibaba’s Qwen4-Max announcement · llama.cpp development discussion
Our best guess is a dense model in the 27-billion-parameter class, with a native context around 262K and a possible path to one million tokens. The strongest clues come from the previous 27B model, Qwen’s public preview of the Qwen4 architecture, and GPU kernels that reference the new 27B directly.
Qwen4-27B vs Qwen3.8-27B
| Spec | Qwen3.8-27B (released) | Qwen4-27B (estimated) |
|---|---|---|
| Parameters and routing | 27B dense language model with a vision encoder | About 27B and likely dense. Whether the headline count includes any extra embedding or prediction weights is unknown. |
| Native context | 262,144 tokens | 262,144 tokens is our baseline estimate. |
| Extended context | 1,000,000 tokens with YaRN in a compatible runtime | Around 1 million with scaling is plausible. Official support is unknown. |
| Attention | Gated DeltaNet and gated full attention in a 3:1 pattern | Gated DeltaNet likely returns. Qwen Sparse Attention is plausible in the other layers. The final pattern is unknown. |
| Layers and model width | 64 layers, 5,120 hidden width | 5,120 width is supported by kernel shapes. Layer count is unknown. Keeping 64 is plausible. |
| Attention heads | 24 query heads and 4 key/value heads in the full-attention layers | Kernel shapes suggest 24 query and 4 key/value heads for the tested attention path. |
| Feed-forward width | 17,408 | 14,336 is suggested by kernel tuning shapes. The final configuration could differ. |
| Residual connections | Conventional single residual stream | Four gated residual streams are likely. Kernel shapes point to a rank-320 bottleneck, matching Flash-Next. |
| Embeddings | 248,320 padded token slots | A separate N-gram table is possible. Its inclusion and size are unknown. |
| Multi-token prediction (MTP) | Included | Likely. Kernel testing suggests an MTP path. The final head layout and speedup are unknown. |
| Input and output | Text, images and video in, text out | Text output with image input is likely. Video support is plausible. |
| Thinking and output limits | Thinking can be switched off. Serving guidance suggests up to 131,072 final-answer tokens within the extended window. | Thinking and direct-answer modes are likely. The output budget is unknown and will depend on the runtime. |
| Weights and licence | Downloadable under Apache 2.0 | Downloadable weights expected. Licence and compatible runtimes are unconfirmed. |
The released specs come from Qwen’s Qwen3.8-27B model card. The Qwen4-27B shape estimates come from an analysis of Intel GPU kernels in the llama.cpp tracker. The underlying pull request was unavailable when checked, so these details describe the development evidence, and the public checkpoint may differ. Qwen describes the related architecture in its Flash-Next model card.
What changed between Max and 27B last time?
| Spec | Qwen3.8-Max and its text backbone | Qwen3.8-27B | Qwen4-Max |
|---|---|---|---|
| Parameters | 2.4T total, 95B active per token for the text checkpoint | 27B dense, with no expert routing | 2.4T total and 71B active per token, with an additional 328B N-gram embedding parameters as reported by Alibaba |
| Attention | Gated DeltaNet plus gated full attention in the text backbone | Gated DeltaNet plus gated full attention | Gated DeltaNet plus Qwen Sparse Attention |
| Depth and width | 92 layers, 8,192 hidden width in the text backbone | 64 layers, 5,120 hidden width | Undisclosed in the launch materials |
| Context | 1M through the API. The text checkpoint has 262,144 native and extends to 1.01M. | 262,144 native, up to 1M with YaRN | 1M through the API. Native versus scaled context is undisclosed. |
| Vision | The hosted Max accepts images and video. The downloadable 2.4T checkpoint is text-only. | Image and video input | Image and video input |
| Availability | Hosted Max. Related text weights ship under a Qwen licence. | Apache 2.0 weights | Hosted model. Smaller open-weight Qwen4 releases announced. |
Sources for the baseline are Qwen3.8-Max API specifications, the Qwen3.8-2.4T-A95B model card, the Qwen3.8-27B model card, Alibaba’s Qwen4-Max architecture notes, and the Model Studio API comparison.
Qwen3.8’s move from Max to 27B changed expert routing, width and depth while keeping the same mix of recurrent and full-attention layers. The smaller model also kept vision. It matched the Max API’s headline context with scaling. That precedent gives us more reason to expect Qwen4’s attention and residual changes to carry over than to expect its expert count, embedding size or benchmark results to carry over.
Will Qwen4-27B have a million-token context?
One million is plausible as an extended setting. Qwen3.8-Max offered it through the API, while Qwen3.8-27B shipped with a native 262,144-token window and documented a route to one million using YaRN, a method for scaling the model’s positional encoding. Qwen3.8-Flash-Next, which Qwen explicitly describes as a preview of the architecture behind Qwen4, uses the same 262,144 native and one-million extended split. Qwen4-Max’s hosted limit also stayed at one million. Together those give us a reason to expect a similar split for the 27B. Qwen3.8-27B context guidance · Qwen3.8-Flash-Next context guidance · Qwen4-Max API limits
The context window is the space shared by the prompt, conversation history, tool results and generated tokens. A local runtime may set a lower limit, and how reliably the model finds and uses details near the end of a long prompt needs testing at release.
What could improve on local hardware?
Sparse attention could make a bigger difference on a long coding session or document collection than on a short chat. Qwen Sparse Attention selects blocks of past tokens for attention, cutting the work done by those layers on long prompts. The kernel analysis found signs of this path but said the developers lacked the 27B sparse-attention weights to test it. Gated residual connections are a firmer clue, since the reported tensor shapes match Flash-Next’s four-stream design. Flash-Next also previews N-gram embeddings, which store learned representations of short token sequences. A separate table could matter for download size and memory even if the headline model size stays at 27B. Kernel analysis · Qwen’s architecture preview
If the model has 27 billion parameters, the raw weights alone work out to about 13.5 GB at 4-bit or 54 GB at BF16. A real quantized file may include scales, higher-precision tensors and separate vision or prediction weights. Add runtime buffers and memory for the context. Whether a 16 GB card can hold the whole model at a useful context length depends on the actual quantization and runtime. Even if sparse attention ships, its effect on cache memory will depend on how the runtime implements it.
Will the weights be downloadable?
That is the expectation. Alibaba’s November 16 announcement confirmed smaller open-weight Qwen4 models, and the previous 27B shipped under Apache 2.0. The 3.8 launch also gives a useful timeline. Alibaba announced Max on August 3, and Qwen published the 27B on August 14. That gap is one example and gives us little reason to predict a specific date. For Qwen4-27B, the release to watch for is an official weight download with a licence, model configuration and runtime support. Once those are available, local inference should be possible without an Alibaba API. Qwen4 release announcement · Qwen3.8-Max launch · Qwen3.8 release history · Official Qwen downloads
