How to Run Qwen 3.8 27B Locally: A Complete Setup Guide

In this guide, you'll learn:

• What Qwen 3.8 27B is and how it differs from Qwen 3.8 Max
• What hardware you need to run it
• How to run Qwen 3.8 27B locally with Atomic Chat or llama.cpp

If you want the full Qwen lineup from 0.8B up, see our guide to running Qwen models locally. This article is only about Qwen 3.8 27B.

What is Qwen 3.8 27B?

Qwen 3.8 27B is an open-weight dense language model developed by Alibaba's Qwen team, the smaller sibling of Qwen 3.8 Max. Alibaba launched the Max on August 3, 2026, announcing open weights for both models. The 2.4T-parameter Max weights followed on August 12, and the 27B weights landed on August 14.

Qwen 3.8 27B main specs:

The 27B is multimodal, meaning it can process images and video alongside text. It also inherits the hybrid attention design of Qwen3.6 27B: 48 of the 64 layers use Gated DeltaNet, a form of attention whose memory use stays the same no matter how long the conversation gets. Only the other 16 layers keep a regular KV cache, the part that normally grows with every token. In practice, that means long chats and big documents cost far less memory here than on a typical 27B model. The 3.8 config.json confirms the layout carried over unchanged. We'll put numbers on this in the hardware section.

The model is also trained for Multi-Token Prediction, so it can draft several tokens per forward pass. That's the same mechanism behind speculative decoding, and the prediction head ships inside the checkpoint; llama.cpp runs it with --spec-type draft-mtp.

The Qwen 3.8 release includes three different models, and they're easy to confuse:

• Qwen 3.8 Max is the 2.4T-parameter MoE flagship with 95B active parameters, served through the Alibaba Cloud API, and currently the top open-weight scorer on Artificial Analysis's Intelligence Index.
• Qwen3.8-2.4T-A95B is the downloadable version of Max, published on August 12. It's text-only, reasons on every request, and at 2.4 trillion parameters it takes a multi-GPU datacenter cluster to serve, the same class of hardware as Kimi K3.
• Qwen 3.8 27B is the dense vision-language model this guide is about, and the only one of the three that runs on an everyday machine: a desktop with a 24 GB GPU, or a MacBook with 32 GB of unified memory.

Qwen 3.8 27B benchmarks

Alibaba's launch numbers, from the model card, compare the 27B with Qwen3.6 27B, Muse Glimmer 30B, and Claude Opus 4.6 Max:

The 27B comes out ahead of Claude Opus 4.6 Max on four of the seven benchmarks. An Opus-level model now runs on a single 24 GB card.

Here's the same class of local models, each scored by its own vendor:

All scores in this second table come from each vendor's own model card or release blog, and every lab runs its own eval setup, so small gaps between columns don't mean much. A hyphen means the vendor didn't publish that number, and the Qwen3.6 35B card labels its score plain GPQA rather than GPQA Diamond.

The direct rival here is Muse Glimmer 30B: Meta's return to open weights, released the same week, and a dense vision-language model of almost the same size. On launch numbers the 27B scores higher on GPQA Diamond (89.2 vs 83.5) and Humanity's Last Exam (30.8 vs 22.0), while Glimmer has the best published SWE-bench Verified score in the table above.

The more useful reference for the 27B is its direct predecessor. In Qwen's published results, Qwen3.6 27B scores 86.2% on MMLU-Pro and 77.2% on SWE-bench Verified. Since its release in April, it has been one of the strongest dense models for local coding you can fit on a 24 GB card. The 3.8 replaces it at the same parameter count and dimensions, so the hardware footprint is identical.

Qwen 3.8 27B GGUF: we quantized it ourselves

We built the AtomicChat Qwen 3.8 27B GGUF repository from Qwen's original BF16 weights, then measured every build against the unquantized reference. The repository holds sixteen builds from 8.5 to 28.9 GB, plus the vision projector, a separate file shared by all of them; the multi-token prediction head is inside every file and needs no extra download. The AD prefix stands for Atomic Dynamic, and a double name like AD-Q5_K_M-Q4_K_M says what the two largest tensor groups got; when both match, the name collapses to one.

How to read the table: KL divergence measures the distance between the quant's output distribution and the original weights. Lower values are better, and zero means identical. Top-1 match is the percentage of positions where the quant selects the same next token as the reference. For a single summary measure, use top-1 match. In the next section we'll explain which file fits your hardware.

We measured every number in the table ourselves. The reference is the original BF16 weights, the calibration corpus is public, and the raw logs are in the metrics repo, so you can check any of it yourself. The full recipe is on the model card.

How our GGUFs compare at matched sizes

We downloaded the builds from unsloth, ggml-org, and lmstudio-community and measured them on the same harness as ours, against the same BF16 reference:

At every size in the table, the AtomicChat build has the lower KL divergence.

Three publishers ship a file called Q4_K_M for this model, and they are not the same file:

A quant name only says which recipe the publisher requested. Each publisher makes its own layout choices under the same label, which is why one name comes out at three different sizes. When you pick a build, go by file size and measured quality.

Qwen 3.8 27B hardware requirements

For Qwen 3.8 27B, the system requirement to check is memory. The table below shows the total usable RAM plus VRAM you need, or unified memory on Apple Silicon:

Note: if you are choosing between two neighbouring files, take the larger one. It costs one or two gigabytes more and picks the wrong token noticeably less often, especially below 14 GB, where quality falls fastest.

How much memory does the context window add?

The figures above only cover the model weights. On top of that, the engine allocates a KV cache: the longer you chat, the more context the model holds in memory, and the more gigabytes that takes. The same goes for pasting big documents or codebases into the conversation.

In a conventional dense transformer, all 64 layers would store attention data for every token. Here, 48 of the 64 layers use Gated DeltaNet with a fixed-size state, and the 16 gated-attention layers use just 4 KV heads. Measured on our builds, the model keeps 256 KB of attention cache per token, about a quarter of what a standard transformer of this shape would store:

The full 262K window needs server hardware, since the cache alone is about 67 GB, more than twice the model file.

What hardware can run Qwen 3.8 27B?

• RTX 3090 / RTX 4090 (24 GB VRAM): can run AD-Q5_K_M-Q4_K_M entirely in VRAM with room for about 16K of context.
• RTX 5090 (32 GB VRAM): can run AD-Q6_K entirely in VRAM with around 24K of context, and at that quant the model behaves close to the original.
• 16 GB GPUs (RTX 4060 Ti, RTX 5060 Ti): can fit AD-IQ3_S with around 8K of context, or a 4-bit build with a few layers offloaded to system RAM. The 27B is dense, so offloading costs more here than it does on the MoE models we've covered recently: the GPU waits for the slow system-RAM layers on every generation step. A smaller quant that fits entirely in VRAM is usually the better trade.
• MacBook Pro (24 GB unified memory): can run AD-IQ3_S with an 8K context. macOS caps GPU-addressable memory at roughly 75 percent of unified memory by default, so a 24 GB Mac gives the model about 18 GB to work with.
• MacBook Pro M4/M5 Max, Mac Studio (36-64 GB): can run Q5 or Q6 with room to spare, and Q8_0 from 48 GB up. On Apple Silicon the GGUF builds run natively through Metal; see our GGUF vs MLX guide if you're choosing between formats.
• Phones: no. For a model that runs on a phone, use the smaller members of the Qwen family from our Qwen guide.

Measured throughput

We measured AD-Q4_K_M on two RTX 5090s with full GPU offload:

Generation stays at 72-77 tokens per second as the context grows from 8K to 32K. On a single 24 GB card, expect lower numbers.

How to run Qwen 3.8 27B locally with Atomic Chat

Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser, manages split GGUF downloads, and provides a chat interface without requiring a manual llama.cpp build.

Here's how to run Qwen 3.8 27B with Atomic Chat:

Step 1: Install Atomic Chat

Download Atomic Chat from atomic.chat and install the build for your platform:

• macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
• Windows: an .exe installer for x64
• Linux: a self-contained .AppImage for x86_64, which needs no root
• iOS: through the App Store
• Android: through Google Play

On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.

Step 2: Find our Qwen 3.8 27B GGUF

Open the Models tab and search for:

AtomicChat/Qwen3.8-27B-GGUF

The catalog is backed by Hugging Face, so you'll also see builds from other publishers as they appear. Choose the result published by AtomicChat, then expand the Download Options picker to list the available quantizations.

Note: fake Qwen3.8-27B repositories existed on Hugging Face before the weights did. If you're downloading from anywhere other than the app catalog, see the Troubleshooting section for how to check what you're pulling.

Step 3: Pick a quant for your memory

Use the hardware table above. For the most common configurations:

• 24 GB GPU: download AD-Q5_K_M-Q4_K_M.
• 32 GB: download AD-Q6_K.
• 48 GB and up: download Q8_0.

The picker lists each build under its short tag, without the AD prefix, and a double name like AD-Q5_K_M-Q4_K_M shows up under the tag of its smaller half: on a 24 GB card, the file to take is the Q4_K_M row at 18.2 GB. The sizes shown in the picker differ a little from the file sizes in our tables; when two rows share a name, go by size. Select the largest quant that fits with context headroom. If you're not sure what the quant names mean, see our guide to what GGUF is and how quantization works.

Step 4: Set the context, thinking, and sampling

The model declares a 262K context window, but configuring the maximum allocates the KV cache up front. Start with 8,192 tokens for chat or 32,768 for code and document work, and increase it only when the workload requires it.

Context size lives in the model's own settings, not in the chat window. Open Settings → Model Providers → Llama.cpp, find the Qwen 3.8 build in the model list, and click the gear icon on its row. Set three things there:

• Context Size: set it to 8192 or 32768. A value of 0 reads the maximum from the GGUF, which for this model means the full 262K window.
• Auto Increase Context Size: on by default. It grows the context when a conversation runs out of room, which can push a build that only just fit past your memory limit. Turn it off when you're working close to that limit.
• GPU Layers: -1 offloads every layer, which is what you want with a dense model that fits in VRAM.

Changing Context Size or GPU Layers restarts the model, so set them before a long conversation.

Thinking is on by default at xhigh effort, with medium and low available, and you can turn it off entirely: the lightbulb icon in the chat switches it on and off. Atomic Chat can show or hide the reasoning output.

Sampling is a separate panel: click the sliders icon next to the model name at the top of the chat. For thinking mode, Qwen recommends temperature 1.0, top_p 0.95, and top_k 20; with thinking off, 0.7 and 0.80. These settings are saved on the assistant, your chat profile, not on the model, so they stay the same when you switch models.

Step 5: Chat locally

After the download completes, Atomic Chat loads the model and opens it in the built-in chat.

Since the 27B is a native vision-language model, you can attach an image to the chat and ask about it. The image, the weights, and your prompts stay on your machine.

Atomic Chat also exposes an OpenAI-compatible API server at http://localhost:1337/v1. Any tool that speaks the OpenAI API, including coding agents like Claude Code and Cline, can use the local model as a drop-in replacement for a cloud one.

How to run Qwen 3.8 27B with llama.cpp

You might prefer to run the model directly through llama.cpp if you need:

• An OpenAI-compatible local endpoint with explicit flags
• Precise control over GPU offload
• A reproducible server configuration

The 27B uses the same hybrid Gated DeltaNet design as Qwen3.6, which upstream llama.cpp has supported since February 2026, so the release-day GGUFs run on a current build with no patches.

Step 1: Build current llama.cpp

For NVIDIA CUDA:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-cli llama-server

For Apple Silicon, Metal is enabled by default:

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server

Step 2: Run the model

The following command downloads AD-Q4_K_M from Hugging Face, offloads all layers to the GPU, applies Qwen's recommended sampling, and limits the context to 8K:

./build/bin/llama-cli \
  -hf AtomicChat/Qwen3.8-27B-GGUF:AD-Q4_K_M \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192

Replace AD-Q4_K_M with another quant name if your system has a different memory capacity.

Step 3: Expose a local OpenAI-compatible API

Replace llama-cli with llama-server:

./build/bin/llama-server \
  -hf AtomicChat/Qwen3.8-27B-GGUF:AD-Q4_K_M \
  --alias qwen3.8-27b \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

Test it with:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "qwen3.8-27b", "messages": [ { "role": "user", "content": "Summarize what Gated DeltaNet changes about attention." } ] }'

Bind the server to 127.0.0.1 unless other machines on your network need access.

Running vision through llama.cpp

Image input needs one extra file, the vision projector, which is shared by all quants. Download mmproj-Qwen3.8-27B-F16.gguf from the repository once and pass it alongside whichever quant you picked:

llama-mtmd-cli \
  -m Qwen3.8-27B-AD-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-F16.gguf \
  --image your-photo.jpg --image-min-tokens 1024 \
  -ngl 99 -c 8192 \
  -p "What is in this image?"

Keep --image-min-tokens 1024: llama.cpp warns that this family needs at least that many image tokens, and below it anything positional gets unreliable. Vision survives quantization well: every file from AD-IQ3_S upward transcribed our handwritten calligraphy test exactly, line breaks included.

Running it with Ollama or LM Studio

Both apps run llama.cpp underneath, and the release-day support is already live: ollama pull qwen3.8:27b works, and LM Studio lists the model in its catalog with GGUF and MLX builds. The AtomicChat GGUF builds load in both apps as well.

Troubleshooting

The GGUF you downloaded is not the real model

Placeholder repositories claimed the Qwen3.8-27B name on Hugging Face before the weights existed. Before downloading from an unfamiliar repo, check that the publisher is either the official Qwen organization or a quantizer you recognize, that the model card exists, and that the file list actually contains GGUF shards of a plausible size. A 27B model at 4-bit cannot weigh 400 MB.

Thinking mode won't turn off

The model supports switching thinking off with enable_thinking: false, but the switch only works when the chat template passes it through, and fresh Qwen releases have broken that template handling more than once. If the model keeps reasoning after you turn thinking off, update llama.cpp first, and check the repository README: we document the working flags there.

llama.cpp prints warnings while encoding an image

Lines about find_slot: non-consecutive token position and unused blk.64 tensors are expected. The first is how this family numbers image patches, with a multi-dimensional position encoding instead of one running index. The second is the prediction head, which a plain forward pass does not run.

You run out of memory at long context

The weights fit, then a long conversation crashes the model. That's the KV cache growing past your headroom. Set an explicit Context Size instead of 0, turn off Auto Increase Context Size when you're near the limit, and if you need more room, step one quant down: at 256 KB of cache per token, every gigabyte you free adds about 4K of context.

Frequently asked questions

The questions that come up most often about running Qwen 3.8 27B on your own hardware.

How much VRAM does Qwen 3.8 27B need?

About 17 GB for the 4-bit file. A 24 GB GPU is the practical minimum for mainstream quants with context headroom. On Apple Silicon, macOS caps GPU memory at about 75 percent of unified memory, so a 24 GB Mac runs the 3-bit build; 32 GB and up runs the 4-bit files with room for context.

Can I run Qwen 3.8 27B on 16GB of VRAM?

Yes, with tradeoffs. AD-IQ3_S (13.8 GB) fits in 16 GB with around 8K of context, at 92.41% top-1 agreement against the original. A 4-bit build needs partial CPU offload, which slows a dense model on every token. On 12 GB, AD-IQ2_S runs with short context only.

Can you run Qwen 3.8 27B on a Mac?

Yes. A 24 GB unified-memory Mac runs AD-IQ3_S with an 8K context, and 36-64 GB Macs run Q5 or Q6 natively through Metal, with Q8_0 from 48 GB up. A 16 GB Mac is too small for usable context; for that hardware, see our best local LLM for a 16GB Mac roundup.

Has Qwen 3.8 27B been released yet?

Yes. The Qwen 3.8 27B release date was August 14, 2026: Alibaba announced the family on August 3 and published the weights in the official Qwen organization on Hugging Face. Our GGUF builds are in the AtomicChat/Qwen3.8-27B-GGUF repository and in the Atomic Chat catalog.

What is the difference between Qwen 3.8 27B and Qwen 3.8 Max?

The difference is size and modality. Qwen 3.8 Max is a 2.4T-parameter MoE model; its open weights are text-only and always reason, and serving 2.4 trillion parameters needs datacenter hardware. Qwen 3.8 27B is a dense vision-language model that accepts images and video and fits on a single 24 GB GPU.

Is Qwen 3.8 27B better than Qwen3.6 27B?

Yes, by Alibaba's launch numbers: SWE-bench Pro goes from 53.5 to 61.7, Terminal Bench 2.1 from 63.4 to 73.0, and OSWorld-Verified from 63.9 to 84.3. The two models share the same size and dimensions, so the hardware math transfers one to one: if you run Qwen3.6 27B today, the 3.8 build fits the same machine.

How does Qwen 3.8 27B compare to Muse Glimmer 30B?

They're the closest pair in the local class right now: dense vision-language models around 30B, released the same week, and both fit a 24 GB card at 4-bit. The 27B leads on GPQA Diamond (89.2 vs 83.5) and Humanity's Last Exam (30.8 vs 22.0); Glimmer's strongest published number is 76.0 on SWE-bench Verified, a benchmark Qwen's card doesn't report. Both ship under Apache 2.0.

Does Qwen 3.8 27B support vision locally?

Yes. Qwen 3.8 27B is a native vision-language model that accepts image and video input. In Atomic Chat, you attach an image to the chat and ask about it, with nothing leaving your machine. Through llama.cpp, image input works from day one: every GGUF publisher ships the mmproj file alongside the quants. Video input isn't supported in llama.cpp.

Does Qwen 3.8 27B work with Ollama or LM Studio?

Yes, since release day: ollama pull qwen3.8:27b in Ollama, and a catalog entry with GGUF and MLX builds in LM Studio. Both apps also accept our files if you point them at the AtomicChat repo.

Can I turn off thinking mode?

Yes. Thinking runs at xhigh effort by default, with medium and low levels, and enable_thinking: false switches it off entirely; with thinking off, Qwen recommends temperature 0.7 and top_p 0.80. In Atomic Chat, the lightbulb icon in the chat toggles it.

Is Qwen 3.8 27B free for commercial use?

Yes. Qwen 3.8 27B ships under the Apache 2.0 license: commercial use, modification, and redistribution are all allowed. The custom Qwen license with commercial thresholds applies only to the Qwen 3.8 Max open weights, not to the 27B.

Bottom line

If your machine has a 24 GB GPU or 32 GB of unified memory, Qwen 3.8 27B is the Qwen 3.8 model to run locally: download AD-Q5_K_M-Q4_K_M through Atomic Chat, start at 8K context, and raise it as the work demands. The llama.cpp route serves the same files behind a local OpenAI-compatible API.

Key takeaways:

• Qwen 3.8 27B is a dense 27B multimodal model with hybrid attention, a 262K native context window, and Multi-Token Prediction.
• Alibaba announced the Qwen 3.8 family on August 3, 2026. The 27B weights landed on August 14 under Apache 2.0.
• Our GGUF builds run from 8.5 to 28.9 GB. For a 24 GB card, take AD-Q5_K_M-Q4_K_M.
• Long context costs memory on top of the file: about 2 GB at 8K and 8 GB at 32K.
• Between neighbouring quants, the larger one is worth the extra gigabyte; error grows fastest below 14 GB.