How to Run Qwen3.8 Flash Next Locally: GGUF, Hardware and Benchmarks

In this guide, you'll learn:

• What Qwen3.8 Flash Next is and how it differs from Qwen 3.8 27B and Qwen 3.8 Max
• What hardware you need to run it
• How to run Qwen3.8 Flash Next locally with Atomic Chat or llama.cpp

If you want the full Qwen lineup from 0.8B up, see our guide to running Qwen models locally. This article is only about Qwen3.8 Flash Next.

What is Qwen3.8 Flash Next?

Qwen3.8 Flash Next is an open-weight Mixture-of-Experts model developed by Alibaba's Qwen team, and an early preview of the architecture behind Qwen4. Alibaba published the weights on August 26, 2026, twelve days after Qwen 3.8 27B. The hosted Qwen3.8-Flash on Alibaba's API is the served version of the same model; Flash Next is the checkpoint you download.

Qwen3.8 Flash Next main specs:

Flash Next activates 6B of its 125B parameters per token, so compute per generated token is close to a much smaller dense model, while memory still has to hold the full file. The download is bigger than 125B, though: the repository reports about 180B parameters. The extra 55B is a lookup table of n-gram embeddings plus the 4B Multi-Token Prediction head. The table stores ready-made representations for frequent two- and three-token sequences; reading it costs almost no compute, and Qwen notes it is easier to offload than the expert weights. We'll put numbers on this in the hardware section.

The Multi-Token Prediction head lets the model draft several tokens per forward pass. That's the same mechanism behind speculative decoding.

The Qwen 3.8 lineup now spans four different models, and they're easy to confuse:

• Qwen 3.8 Max is the 2.4T-parameter MoE flagship with 95B active parameters, served through the Alibaba Cloud API.
• Qwen3.8-2.4T-A95B is the downloadable version of Max. It's text-only, reasons on every request, and at 2.4 trillion parameters it takes a multi-GPU datacenter cluster to serve, the same class of hardware as Kimi K3.
• Qwen 3.8 27B is the dense vision-language model that fits on a 24 GB GPU or a 32 GB Mac. We cover it in a separate guide to running Qwen 3.8 27B locally.
• Qwen3.8 Flash Next is the new-architecture MoE this guide is about: too big for a gaming GPU, and a comfortable fit for a workstation or a Mac with 128 GB of memory.

Qwen3.8 Flash Next benchmarks

The table below puts Flash Next next to its dense sibling and the local models in its class. Every score is the one that model's own vendor publishes:

On Qwen's own launch table, Flash Next also comes out ahead of Claude Opus 4.6 Max on eight of the nine language benchmarks where both report scores. An Opus-level model now runs on a 128 GB workstation.

Every lab runs its own eval setup, so small gaps between columns don't mean much. A hyphen means the vendor didn't publish that number for its latest build: DeepSeek reports only agentic benchmarks on the 0731 card, and Laguna's label for its 59.4 is SWE-Bench Pro on the public dataset. The DeepSeek and Laguna GGUF sizes are the unsloth 4-bit builds; the 27B size is our AD-Q4_K_M.

The direct rival is DeepSeek V4 Flash: a 284B MoE with 13B active parameters, released a month earlier, and the model Qwen's own card measures against. In the table above, DeepSeek keeps Terminal Bench 2.1 and NL2Repo-Bench; Flash Next leads on the other rows. Flash Next is also the smaller download at every quality tier: around 110 GB at 4-bit against DeepSeek's 155 GB.

Flash Next has no direct predecessor in the open lineup: it previews the architecture the Qwen4 family will be built on, and Qwen says this early release is deliberate, so the community can examine the changes before Qwen4 lands. Against the hosted Qwen3.7-Plus, the card claims training cost about one ninth as much. If your machine is a 24 GB card, the dense 27B remains the stronger pick for local coding; Flash Next is the better model when you have the memory for it.

Qwen3.8 Flash Next hardware requirements

For Qwen3.8 Flash Next, the system requirement to check is memory. The BF16 shards total 360 GB, so quantization is what brings the model onto a workstation: a 4-bit build lands around 110 GB, and the 1-bit builds land around 73 GB. Those sizes include the 51B n-gram table. The table below shows the total usable RAM plus VRAM you need, or unified memory on Apple Silicon:

Note: if you are choosing between two neighbouring files, take the larger one. It costs a few gigabytes more and picks the wrong token noticeably less often, especially at the bottom of the ladder, where quality falls fastest.

How much memory does the context window add?

The figures above only cover the model weights. On top of that, the engine allocates a KV cache: the longer you chat, the more context the model holds in memory, and the more gigabytes that takes. The same goes for pasting big documents or codebases into the conversation.

The Qwen4 architecture keeps this cost small. Of the 48 layers, 36 use Gated DeltaNet with a fixed-size state that does not grow with the conversation, and the 12 Qwen Sparse Attention layers keep just 2 KV heads each. The attention cache works out to about 25 KB per token, a tenth of what the dense 27B stores:

On the dense 27B, the same window needs about 67 GB of cache; here it fits next to the weights on a 96 GB or 128 GB machine.

What hardware can run Qwen3.8 Flash Next?

• 128 GB unified-memory Macs (M4/M5 Max, Mac Studio) and 128 GB compact workstations (NVIDIA DGX Spark, AMD Strix Halo): can run the 4-bit build with the full 262K context. On a Mac, raise the default GPU memory limit first, since macOS caps it at roughly 75 percent of unified memory. The GGUF builds run natively through Metal; see our GGUF vs MLX guide if you're choosing between formats.
• 96 GB GPUs (RTX PRO 6000): can run a 2-bit build entirely in VRAM, or a 4-bit build with the expert weights split out to system RAM.
• 96 GB Macs and 96 GB RAM desktops: can run the 1-bit build with a moderate context; on a Mac this also needs the GPU memory limit raised. Quality at the bottom of the ladder drops fast; check the quant table before settling here.
• 24 GB GPUs (RTX 3090 / 4090 / 5090) with 96 GB or more of system RAM: can run the 4-bit build with the experts held in system RAM and the attention layers on the GPU. Only the 6B active parameters are read from system RAM on each token, so generation stays usable.
• Phones: no. For a model that runs on a phone, use the smaller members of the Qwen family from our Qwen guide.

How to run Qwen3.8 Flash Next locally with Atomic Chat

Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.

Here's how to run Qwen3.8 Flash Next with Atomic Chat:

Step 1: Install Atomic Chat

Download Atomic Chat from atomic.chat and install the build for your platform:

• macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
• Windows: an .exe installer for x64
• Linux: a self-contained .AppImage for x86_64, which needs no root
• iOS: through the App Store
• Android: through Google Play

On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.

Step 2: Find our Qwen3.8 Flash Next GGUF

Open the Models tab and search for:

AtomicChat/Qwen3.8-Flash-Next-GGUF

The catalog is backed by Hugging Face, so you'll also see builds from other publishers as they appear. Choose the result published by AtomicChat, then expand the Download Options picker to list the available quantizations.

Note: fake Qwen3.8-Flash-Next repositories existed on Hugging Face before the weights did, including empty "GGUF" and "MLX" repos with zero files behind them. If you're downloading from anywhere other than the app catalog, see the Troubleshooting section for how to check what you're pulling.

Step 3: Pick a quant for your memory

Use the hardware table above. For the most common configurations:

• 128 GB Mac or workstation: download the 4-bit build.
• 96 GB: download the 3-bit build.
• 192 GB and up: download the 6-bit build or larger.

The picker lists each build under its short tag, without the AD prefix, and a double name shows up under the tag of its smaller half. The sizes shown in the picker differ a little from the file sizes in our tables; when two rows share a name, go by size. Select the largest quant that fits with context headroom. If you're not sure what the quant names mean, see our guide to what GGUF is and how quantization works.

Step 4: Set the context, thinking, and sampling

The model declares a 262K context window, and on this architecture the cache for it is cheap: about 6.5 GB at the full window. If your build fits with that headroom, you can set the maximum straight away; otherwise start with 8,192 tokens for chat or 32,768 for code and document work.

Context size lives in the model's own settings, not in the chat window. Open Settings → Model Providers → Llama.cpp, find the Flash Next build in the model list, and click the gear icon on its row. Set these there:

• Context Size: set it to 8192, 32768, or your target window. A value of 0 reads the maximum from the GGUF, which for this model means the full 262K.
• Auto Increase Context Size: on by default. It grows the context when a conversation runs out of room, which can push a build that only just fit past your memory limit. Turn it off when you're working close to that limit.
• GPU Layers: -1 offloads every layer. Use it when the whole file fits in VRAM or unified memory.
• Keep all Experts in CPU and Number of MoE weights in the CPU: the MoE offload controls. They move expert weights off the GPU so a quant larger than your VRAM still loads. Expect slower generation: the GPU waits for the expert weights read from system RAM on every token.

Changing Context Size or GPU Layers restarts the model, so set them before a long conversation.

Thinking is on by default, with xhigh, medium and low effort levels, and you can turn it off entirely: the lightbulb icon in the chat switches it on and off. Atomic Chat can show or hide the reasoning output.

Sampling is a separate panel: click the sliders icon next to the model name at the top of the chat. For thinking mode, Qwen recommends temperature 1.0, top_p 0.95, and top_k 20; with thinking off, 0.7 and 0.80. These settings are saved on the assistant, your chat profile, not on the model, so they stay the same when you switch models.

Step 5: Chat locally

After the download completes, Atomic Chat loads the model and opens it in the built-in chat.

Since Flash Next is a native vision-language model, you can attach an image to the chat and ask about it. The image, the weights, and your prompts stay on your machine.

Atomic Chat also exposes an OpenAI-compatible API server at http://localhost:1337/v1. Any tool that speaks the OpenAI API, including coding agents like Claude Code and Cline, can use the local model as a drop-in replacement for a cloud one.

How to run Qwen3.8 Flash Next with llama.cpp

You might prefer to run the model directly through llama.cpp if you need:

• An OpenAI-compatible local endpoint with explicit flags
• Precise control over GPU offload
• A reproducible server configuration

The Qwen4 architecture is new to llama.cpp: Gated DeltaNet carries over from earlier Qwen models, and the sparse-attention indexer and the n-gram tables landed with this release.

Step 1: Build current llama.cpp

For NVIDIA CUDA:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-cli llama-server

For Apple Silicon, Metal is enabled by default:

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server

Step 2: Run the model

The following command downloads the 4-bit build from Hugging Face, offloads all layers to the GPU, applies Qwen's recommended sampling, and limits the context to 8K:

./build/bin/llama-cli \
  -hf AtomicChat/Qwen3.8-Flash-Next-GGUF:AD-Q4_K_M \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192

Replace the quant name if your system has a different memory capacity. If the file is larger than your VRAM, keep the attention layers on the GPU and move the experts to system RAM:

./build/bin/llama-cli \
  -hf AtomicChat/Qwen3.8-Flash-Next-GGUF:AD-Q4_K_M \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 --n-cpu-moe 48 \
  -c 8192

Step 3: Expose a local OpenAI-compatible API

Replace llama-cli with llama-server:

./build/bin/llama-server \
  -hf AtomicChat/Qwen3.8-Flash-Next-GGUF:AD-Q4_K_M \
  --alias qwen3.8-flash-next \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

Test it with:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "qwen3.8-flash-next", "messages": [ { "role": "user", "content": "Explain what an n-gram embedding table does." } ] }'

Bind the server to 127.0.0.1 unless other machines on your network need access.

Running vision through llama.cpp

Image input needs one extra file, the vision projector, which is shared by all quants. Download the mmproj file from the repository once and pass it alongside whichever quant you picked:

llama-mtmd-cli \
  -m Qwen3.8-Flash-Next-AD-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-F16.gguf \
  --image your-photo.jpg \
  -ngl 99 -c 8192 \
  -p "What is in this image?"

Running it with Ollama or LM Studio

Both apps run llama.cpp underneath and accept GGUF files you point them at.

Troubleshooting

The GGUF you downloaded is not the real model

Placeholder repositories claimed the Qwen3.8-Flash-Next name on Hugging Face before the weights existed, including "abliterated" and "jailbreak" variants with zero files behind them. Before downloading from an unfamiliar repo, check that the publisher is either the official Qwen organization or a quantizer you recognize, that the model card exists, and that the file list actually contains GGUF shards of a plausible size. A 125B model at 4-bit cannot weigh 400 MB.

llama.cpp says the model architecture is unknown

The qwen4_exp architecture is new, and builds from before the support landed cannot load the GGUFs at all. Update to a current build first. The same applies to Ollama and LM Studio: both bundle their own llama.cpp, so wait for their updates rather than pointing an old version at the files.

The model loads but generation is very slow

The file fits, but each token takes seconds. That usually means part of the model landed in the wrong memory: expert weights swapped out to disk, or everything on CPU because GPU Layers is 0. Check that your total free RAM plus VRAM exceeds the file size with a few gigabytes of headroom, set GPU Layers to -1, and use the MoE offload controls rather than letting the OS swap.

Thinking mode won't turn off

The model supports switching thinking off with enable_thinking: false, but the switch only works when the chat template passes it through, and fresh Qwen releases have broken that template handling more than once. If the model keeps reasoning after you turn thinking off, update llama.cpp first, and check the repository README: we document the working flags there.

You run out of memory at long context

The weights fit, then a long conversation crashes the model. That's the KV cache growing past your headroom, or Auto Increase Context Size raising the window on its own. On this model the cache itself is small, about 25 KB per token, so the usual culprit is the second one: set an explicit Context Size and turn off Auto Increase when you're near the limit.

Frequently asked questions

The questions that come up most often about running Qwen3.8 Flash Next on your own hardware.

How much RAM does Qwen3.8 Flash Next need?

About 128 GB of combined RAM and VRAM, or unified memory, for the 4-bit build with full context headroom. The practical minimum is around 80 GB for the 1-bit builds. The BF16 release itself is 360 GB, which is why quantization is the difference between a datacenter and a workstation here.

Can I run Qwen3.8 Flash Next on an RTX 4090?

No, not on the card alone. 24 GB of VRAM does not hold any build of a 125B model. With 96 GB or more of system RAM next to it, yes: keep the attention layers on the GPU and move the expert weights to RAM. Only 6B parameters are active per token, so the penalty for offloading is far smaller than it would be for a dense model of this size.

Can you run Qwen3.8 Flash Next on a Mac?

Yes, from 96 GB of unified memory up, and comfortably at 128 GB. macOS caps GPU-addressable memory at roughly 75 percent of unified memory by default, so a 128 GB Mac gives the model about 96 GB to work with: that runs the 2-bit build as is, and the 4-bit build after raising the limit with sysctl iogpu.wired_limit_mb. The GGUF builds run natively through Metal.

Is Qwen3.8 Flash Next the same as Qwen 4?

No. Qwen describes it as an experimental preview of the architecture that will underpin Qwen4: the same building blocks, shipped early so the community can examine them before the Qwen4 family is built on top. What you download today is a Qwen 3.8-generation model built on the new architecture.

Has Qwen3.8 Flash Next been released yet?

Yes. The weights landed in the official Qwen organization on Hugging Face on August 26, 2026, under the Qwen Community License 1.0, together with an official FP8 build. Our GGUF builds are in the AtomicChat repository and in the Atomic Chat catalog.

What is the difference between Qwen3.8 Flash Next and Qwen 3.8 27B?

The difference is size, architecture, and the hardware they need. The 27B is a dense model that fits on a 24 GB GPU; Flash Next is a 125B MoE that needs 80 GB at minimum and 128 GB for the 4-bit build. On the card's language benchmarks Flash Next is ahead on every row, with the biggest gaps on agentic work: DeepSWE 58.7 against 42.2, and JobBench 55.7 against 33.4. If your machine is a 24 GB card, run the 27B; if you have workstation memory, Flash Next is the better model.

How does Qwen3.8 Flash Next compare to DeepSeek V4 Flash?

They're the closest pair in the big-MoE class: DeepSeek V4 Flash holds 284B total with 13B active, Flash Next 125B with 6B active. On Qwen's launch table, Flash Next leads on eight of the nine shared rows, and DeepSeek keeps NL2Repo-Bench. Flash Next is the smaller download at every quality tier, around 110 GB at 4-bit against DeepSeek's 155 GB, so a 128 GB machine runs it at a quality level the DeepSeek model reaches only on bigger hardware. We cover that model in our DeepSeek V4 Flash guide.

What are the 51B n-gram embedding parameters?

A lookup table of token pairs and triples that the model reads once, early in the network. Looking up rows is far cheaper than running them through matrix multiplications, which is how the checkpoint scales to 180B stored parameters while activating only 6B per token. The card notes it is easier to offload than the transformer weights.

Does Qwen3.8 Flash Next support vision locally?

Yes. Flash Next is a native vision-language model that accepts image and video input, and the vision encoder ships inside the checkpoint. In Atomic Chat, you attach an image to the chat and ask about it, with nothing leaving your machine. Through llama.cpp, image input uses the mmproj file from the GGUF repository.

Can I turn off thinking mode?

Yes. Thinking is on by default, with xhigh, medium and low effort levels, and enable_thinking: false switches it off entirely; with thinking off, Qwen recommends temperature 0.7 and top_p 0.80. In Atomic Chat, the lightbulb icon in the chat toggles it.

Is Qwen3.8 Flash Next free for commercial use?

Yes, for most products. The Qwen Community License 1.0 allows commercial use, modification, and redistribution. Two carve-outs: a product with more than 100 million monthly active users or 20 million dollars in monthly revenue must display the model name in its interface, and building a model-as-a-service API or a coding or office assistant product on it requires a separate license from Qwen. Unlike the 27B, this release does not use Apache 2.0.

Bottom line

If your machine has 128 GB of memory, Qwen3.8 Flash Next is the strongest model you can run on it today: download the 4-bit build through Atomic Chat and set the context you actually need, since even the full 262K window adds only about 6.5 GB of cache. The llama.cpp route serves the same files behind a local OpenAI-compatible API.

Key takeaways:

• Qwen3.8 Flash Next is a 125B MoE that activates 6B parameters per token, and the first open model on the Qwen4 architecture.
• The weights landed on August 26, 2026 under the Qwen Community License 1.0, which allows commercial use for most products.
• A 4-bit build lands around 110 GB: a 128 GB Mac or workstation runs it with the full 262K context.
• Long context is cheap on this architecture: the entire 262K window costs about 6.5 GB of cache.
• Between neighbouring quants, the larger one is worth the extra gigabytes; error grows fastest at the bottom of the ladder.