How to Run Qwen 3.8 27B Uncensored Locally: A Complete Setup Guide

In this guide, you'll learn:

• What an uncensored Qwen 3.8 27B is, and what abliteration changed in it
• Which of the community builds to run, and what hardware each one needs
• How to run Qwen 3.8 27B uncensored locally with Atomic Chat or llama.cpp

This article is about the uncensored builds. For the original model from Alibaba, see our guide to running Qwen 3.8 27B locally, and for the rest of the lineup, our guide to running Qwen models locally.

What is Qwen 3.8 27B Uncensored?

Qwen 3.8 27B Uncensored is not a release from Alibaba. Alibaba published Qwen 3.8 27B under Apache 2.0 on August 14, 2026, and within days several people edited the part of those weights that makes the model decline requests and republished the result. The license allows it. Every build in this guide is one of those community edits, and each one was made by a different person with a different method.

The underlying model is unchanged:

Abliteration edits weights, it does not retrain the model, so the parameter count, the layer count, the context window and the memory arithmetic all carry over from the original.

Abliteration touches neither the vision tower nor the multi-token prediction head, but only the vision tower reliably comes out the other side. Every build here keeps it. The prediction head is the one that gets lost, and not to the edit itself: an abliteration pipeline that round-trips through Hugging Face transformers drops that head, so it survives only if the publisher avoids that path or grafts the head back from the original checkpoint afterwards. The three builds below each handle that differently, and the table in the next section says how.

What abliteration does to Qwen 3.8 27B

Researchers found that refusal in many aligned models is governed by a single direction in the model's residual stream, the internal state it passes from layer to layer. Arditi et al. showed this across 13 open-weight chat models at NeurIPS 2024: erase that one direction and the model stops declining harmful instructions, add it back and the model starts declining harmless ones.

To find the direction, you run the model over matched sets of harmful and harmless prompts, record the activations, and take the difference in means between the two. Then you remove the model's ability to write to that direction, permanently, by orthogonalizing the weight matrices against it. No training data and no gradient descent are involved, which is why an abliterated 27B appears within days of the original rather than weeks.

The word abliteration does not appear in that paper. It is a community coinage for what the authors called directional ablation.

Abliterated and uncensored point at the same files here. Abliterated names the method, and every Qwen 3.8 27B build in this guide was made this way. Uncensored names the result, and it also covers models like Dolphin and Hermes that got there by training on extra instruction data instead. For the method on its own, across models and toolchains, see our explainer on what abliterated models are; for the wider category of builds, our roundup of the best uncensored LLMs to run locally.

What abliteration costs Qwen 3.8 27B

Two publishers measured the capability cost of their edit against the original weights. JonathanColetti ran the lighter edit, scoring both models on the same harness in the same session:

The numbers are 0-shot through lm-evaluation-harness, so they are not comparable to Alibaba's published few-shot scores, only to each other. Every one of these deltas sits inside its own reported standard error, so on those four tasks the edit did not measurably damage the model. The card is equally clear about what was not measured: no math, no code, no multilingual work, no vision, and no generative evaluation of any kind.

A harder edit costs more. The OBLITERATUS build of the same model ran MMLU over 5,700 questions and reports 82.33% against 84.46% for the original, a drop of 2.1 points that sits well outside its error bars. Their published log is the useful part, because it shows the same team trying three strengths of the same edit:

The cost is also not spread evenly. In their category breakdown, STEM loses 3.3 points while humanities lose 1.0, which fits a refusal direction that overlaps with structured reasoning. The original paper reports a cost of its own: accuracy on TruthfulQA drops consistently in orthogonalized models.

What the edit removes is not in dispute. Over-refusal and safety-refusal are the same direction in the weights. Remove one and the other goes with it, which is why every card in this space ships a responsibility clause. You are now the only guardrail in the loop.

Which uncensored Qwen 3.8 27B build should you run?

At least seven people have published a Qwen 3.8 27B abliterated build, and others have repackaged those. Three are worth your download, for different reasons.

JonathanColetti is the one to download if you want to know what you are getting. It is built with Heretic, which searches the ablation parameters with a two-objective optimizer: one objective counts refusals, the other measures how far the edited model's output distribution has drifted from the original. Two hundred trials produce a Pareto front, and the card publishes all 23 non-dominated points, so you can see the exact trade that was chosen. The published build sits at 12 refusals out of 100 with a KL divergence of 0.1191. The MTP head was copied back from the base checkpoint after the merge and then inspected in every shipped file, 65 blocks out of 65.

huihui-ai gets you running in one command. It is the most established name in abliteration. The method is older and simpler: the card describes its own tooling as a crude, proof-of-concept implementation, and it ablates only layers 18 to 51, which the card says is a deliberate choice to keep more of the original behavior. Watch the file names: the _L builds bump the ablated tensors up to Q8_0, so Q2_K_L weighs 17.2 GB, more than plain Q4_K at 16.8 GB.

orcarouter is where Mac owners should start. It is the only publisher shipping MLX, in 2, 4, 6 and 8-bit, with the vision tower and the normalization layers left in BF16 while only the language-model weights are quantized. The 4-bit build is mirrored at the repository root, so LM Studio and other tools that treat a repo as one model load it directly. Their own fidelity table puts 4-bit at 0.996 cosine against the BF16 source and 2-bit at 0.92, and the card says plainly that 2-bit collapses into repetition loops and garbled output at this size. Note that their GGUF repository is gated and needs a Hugging Face login; the MLX one is open.

Why the refusal numbers don't line up

Two of the three builds above publish a refusal rate, and so does OBLITERATUS. No two of those numbers were measured the same way.

The rows differ in the prompt set, the number of prompts, who or what decides that an answer counts as a refusal, and whether thinking was on. A response that opens with a warning and then delivers exactly what was asked scores as a refusal under one publisher's judge and as compliance under another's. OBLITERATUS counts that as a failure and calls it a soft deflection; the other two do not measure it. JonathanColetti's number was taken with thinking disabled, on the bf16 merge rather than on the file you download. orcarouter names no prompt set at all, and its figure comes from the FP8 build rather than from the GGUF or MLX it ships.

huihui-ai is missing from the table because it publishes no refusal number at all, which is its own kind of answer.

A refusal rate describes one publisher's own test, on their prompts and their judge. Run your own prompts against a build before you rely on it.

One cross-publisher measurement exists, on a different model. The Heretic project benchmarked its own output against huihui-ai's on gemma-3-12b-it and reports both reaching 3 refusals out of 100, at KL 0.16 for Heretic against 0.45 for huihui-ai. Both reach the same compliance, and Heretic drifts roughly three times less from the original. That is a result about the two methods on a different model, so treat it as a reason to prefer the newer tooling rather than a measurement of these files.

Qwen 3.8 27B uncensored hardware requirements

The system requirement to check is memory. The table below shows the total usable RAM plus VRAM you need. On Apple Silicon, compare it against 75 percent of unified memory, which is all macOS gives the model by default. Sizes are given as a range because the three publishers ship different files at each tier. Some tiers are missing from some repositories: JonathanColetti's ladder starts at IQ2_M and skips 3-bit entirely.

Note: if you are choosing between two neighbouring files and both still fit in memory, take the larger one. It costs one or two gigabytes more and picks the wrong token noticeably less often, especially below 14 GB, where quality falls fastest.

Low-bit builds carry a risk here that they do not carry on the original model. JonathanColetti's card recommends evaluating this model's behavior on Q6_K or Q8_0 rather than on the low-bit files, because behavior near the old refusal boundary is the least stable property of an abliterated model and 2-bit compounds exactly that. Nothing in any of these repositories measures the refusal boundary at 2-bit. orcarouter is blunter about its own 2-bit MLX build, calling it archival only, though on generation quality rather than on refusals.

How much memory does the context window add?

The figures above only cover the model weights. On top of that, the engine allocates a KV cache: the longer you chat, the more context the model holds in memory, and the more gigabytes that takes. The same goes for pasting big documents or codebases into the conversation.

In a conventional dense transformer, all 64 layers would store attention data for every token. Here, 48 of the 64 use Gated DeltaNet, whose memory use stays the same no matter how long the conversation gets, and the 16 gated-attention layers use just 4 KV heads. Measured on our builds of the original weights, the model keeps 256 KB of attention cache per token:

The full 262K window needs server hardware, since the cache alone is about 67 GB, four times the 4-bit file.

What hardware can run Qwen 3.8 27B uncensored?

• RTX 3090 / RTX 4090 (24 GB VRAM): can run a 4-bit build entirely in VRAM with room for about 20K of context.
• RTX 5090 (32 GB VRAM): can run Q6_K entirely in VRAM with around 24K of context, and at that quant quantization costs almost nothing on top of whatever the edit already cost.
• 16 GB GPUs (RTX 4060 Ti, RTX 5060 Ti): can fit a 3-bit build with around 8K of context. The 27B is dense, so offloading a 4-bit build costs more here than it does on a mixture-of-experts model: the GPU waits for the slow system-RAM layers on every generation step. A smaller quant that fits entirely in VRAM is usually the better trade.
• MacBook Pro (24 GB unified memory): can run a 3-bit build with an 8K context. macOS caps GPU-addressable memory at roughly 75 percent of unified memory by default, so a 24 GB Mac gives the model about 18 GB to work with.
• MacBook Pro M4/M5 Max, Mac Studio (36-64 GB): can run Q5 or Q6 with room to spare, and Q8_0 from 48 GB up. This is also where orcarouter's MLX builds are worth trying instead of GGUF; see our GGUF vs MLX guide if you're choosing between formats.
• Phones: no. For a model that runs on a phone, see the smaller entries in our uncensored roundup.

How to run Qwen 3.8 27B uncensored locally with Atomic Chat

Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.

Here's how to run Qwen 3.8 27B uncensored with Atomic Chat:

Step 1: Install Atomic Chat

Download Atomic Chat from atomic.chat and install the build for your platform:

• macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
• Windows: an .exe installer for x64
• Linux: a self-contained .AppImage for x86_64, which needs no root
• iOS: through the App Store
• Android: through Google Play

On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.

Step 2: Find an uncensored Qwen 3.8 27B GGUF

Open the Models tab and search for:

JonathanColetti/Qwen3.8-27B-Uncensored-GGUF

The catalog is backed by Hugging Face, so community repositories appear alongside official ones. Check the publisher name on the result before you download, then expand the Download Options picker to list the available quantizations.

Note: several repositories carry near-identical names, and one of them is gated. See the Troubleshooting section for what to check before you download.

Step 3: Pick a quant for your memory

Use the hardware table above. For the most common configurations:

• 24 GB GPU: download Q4_K_M, at 16.8 GB.
• 32 GB: download Q6_K, at 22.4 GB.
• 48 GB and up: download Q8_0, at 29.0 GB.

The picker lists each build under its short tag, and it shows every file in the repository, so expect surprises. Sizes run about 7 percent under the numbers above, because the picker counts in gibibytes. Each quant appears twice, once fused with the prediction head and once without it. The two small entries at the top of the list, an F16 at 884.6 MB and a Q8_0 at 2.9 GB, are the vision projector and the draft head rather than tiny models. Go by size, and take the larger of the two rows that share a name if you want the fused build. Select the largest quant that fits with context headroom. If you're not sure what the quant names mean, see our guide to what GGUF is and how quantization works.

Step 4: Set the context, thinking, and sampling

The model declares a 262K context window, but configuring the maximum allocates the KV cache up front. Start with 8,192 tokens for chat or 32,768 for code and document work, and increase it only when the workload requires it.

Context size lives in the model's own settings, not in the chat window. Open Settings → Model Providers → Llama.cpp, find the build in the model list, and click the gear icon on its row. Set three things there:

• Context Size: set it to 8192 or 32768. A value of 0 reads the maximum from the GGUF, which for this model means the full 262K window.
• Auto Increase Context Size: on by default. It grows the context when a conversation runs out of room, which can push a build that only just fit past your memory limit. Turn it off when you're working close to that limit.
• GPU Layers: -1 offloads every layer, which is what you want with a dense model that fits in VRAM.

Changing Context Size or GPU Layers restarts the model, so set them before a long conversation.

Thinking is on by default. The lightbulb icon in the chat switches it off.

Sampling is a separate panel: click the sliders icon next to the model name at the top of the chat. Qwen's recommended settings carry over from the original weights: temperature 1.0, top_p 0.95 and top_k 20 with thinking on, 0.7 and 0.80 with it off. These settings are saved on the assistant, your chat profile, not on the model, so they stay the same when you switch models.

Step 5: Chat locally

After the download completes, Atomic Chat loads the model and opens it in the built-in chat. Send it a prompt the original declines, to check that the build you downloaded is the edited one.

Since the 27B is a native vision-language model and the vision tower survives abliteration, you can attach an image to the chat and ask about it, once the separate projector file from the same repository sits next to the quant. The image, the weights, and your prompts stay on your machine.

Atomic Chat also exposes an OpenAI-compatible API server at http://localhost:1337/v1. Any tool that speaks the OpenAI API, including coding agents like Claude Code and Cline, can use the local model as a drop-in replacement for a cloud one.

How to run Qwen 3.8 27B uncensored with llama.cpp

You might prefer to run the model directly through llama.cpp if you need:

• A server you can start headless, with the host and port set by flag
• Precise control over GPU offload
• A reproducible server configuration

These builds use the same hybrid Gated DeltaNet architecture as the original, which upstream llama.cpp has supported since February 2026, so they run on a current build with no patches.

Step 1: Build current llama.cpp

For NVIDIA CUDA:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-cli llama-server llama-mtmd-cli

For Apple Silicon, Metal is enabled by default:

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server llama-mtmd-cli

Step 2: Run the model

The following command downloads Q4_K_M from Hugging Face, offloads all layers to the GPU, applies Qwen's recommended sampling, and limits the context to 8K:

./build/bin/llama-cli \
  -hf JonathanColetti/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192

Replace Q4_K_M with another quant name if your system has a different memory capacity.

Step 3: Expose a local OpenAI-compatible API

Replace llama-cli with llama-server:

./build/bin/llama-server \
  -hf JonathanColetti/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M \
  --alias qwen3.8-27b-uncensored \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

Test it with:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "qwen3.8-27b-uncensored", "messages": [ { "role": "user", "content": "Summarize what Gated DeltaNet changes about attention." } ] }'

Bind the server to 127.0.0.1 unless other machines on your network need access.

Running vision through llama.cpp

Image input needs one extra file, the vision projector, which is shared by all quants and is about 930 MB. Download it once from the same repository and pass it alongside whichever quant you picked:

./build/bin/llama-mtmd-cli \
  -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  --mmproj Qwen3.8-27B-Uncensored-vision-f16.gguf \
  --image your-photo.jpg --image-min-tokens 1024 \
  -ngl 99 -c 8192 \
  -p "What is in this image?"

Keep --image-min-tokens 1024: llama.cpp warns that this family needs at least that many image tokens, and below it anything positional gets unreliable. The projector file is named differently in each repository, so check the file list in yours before you copy the filename above.

Speculative decoding with the MTP head

The original checkpoint ships a multi-token prediction head, the mechanism behind speculative decoding, and the builds that kept it can draft the next token ahead of the main model. Add the flags:

./build/bin/llama-server \
  -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 1 \
  -ngl 99 -c 8192

MTP support landed in llama.cpp in PR #22673, merged in May 2026. Older builds load these files and silently ignore the MTP tensors, so if you see no speedup, update first. JonathanColetti measured the gain across draft lengths and reports 1.19x on prose and 1.28x on code at a draft length of 1, with the gain shrinking at every step above that and gone by 3 on prose. The draft head was trained against the unedited model, so acceptance may run a little lower on an abliterated build; every drafted token is still verified by the model itself, so output quality is unaffected.

Running it with Ollama or LM Studio

Two of these builds have Ollama tags:

ollama run huihui_ai/Qwen3.8-abliterated

ollama run orcarouter/Qwen3.8-27B-Uncensored

Each pulls a 17.7 GB download: a 16.8 GB 4-bit file plus the 0.9 GB vision projector, so image input works without a second download. Update Ollama first: Qwen 3.8 support landed in v0.32.12 on August 14, and these tags rely on Ollama's built-in Qwen 3.8 renderer rather than shipping a template of their own, so older builds will not load them.

LM Studio loads the GGUF files directly from JonathanColetti and huihui-ai. orcarouter's GGUF repo is gated, so LM Studio needs a Hugging Face read token before it can pull that one. It will also load orcarouter's 4-bit MLX build on Apple Silicon, since that is the one mirrored at the repository root.

Troubleshooting

The model still refuses

Abliteration reduces refusals, it does not eliminate them. The best-documented build here still declines 12 prompts out of 100 on its own test set. Check three things:

• Turn thinking off with the lightbulb icon. JonathanColetti's number was measured that way, and orcarouter never says which mode it used, so thinking is the variable those two leave open.
• Move up to Q6_K or Q8_0. The refusal boundary is the least stable part of an abliterated model, and JonathanColetti's card recommends judging behavior only at those two sizes.
• Confirm you loaded the build you meant to load. The original weights and the edited ones sit under very similar names in the app catalog.

The build you downloaded is not what it claims

Community re-uploads carry no publisher guarantee, and names collide. Before downloading from an unfamiliar repository, check that the model card exists and describes an actual method, that the file list contains GGUF shards of a plausible size, and that the publisher is one you can trace. A 27B model at 4-bit cannot weigh 400 MB. If a repository asks you to log in to see the files, that is a gated repository rather than a broken one; orcarouter's GGUF repo is gated.

Image input does nothing

The vision projector is a separate file, not part of the quant you downloaded. Fetch the mmproj file from the same repository and pass it with --mmproj, as in the vision section above.

The low-bit build produces nonsense

Repetition loops and garbled output at 2-bit are expected on a 27B model, and orcarouter says so on its own card. Step up one quant. If you cannot fit a 3-bit build, a smaller uncensored model will serve you better than a broken 27B; our uncensored roundup covers the options that fit under 12 GB.

You run out of memory at long context

The weights fit, then a long conversation crashes the model. That's the KV cache growing past your headroom. Set an explicit Context Size instead of 0, turn off Auto Increase Context Size when you're near the limit, and if you need more room, step one quant down: at 256 KB of cache per token, every gigabyte you free adds about 4K of context.

Frequently asked questions

The questions that come up most often about running an uncensored Qwen 3.8 27B on your own hardware.

What is the difference between abliterated and uncensored models?

Abliterated describes how the model was changed: one direction in the weights, the one that governs refusal, was removed. Uncensored describes the result, and it also covers models that got there through fine-tuning rather than weight editing. Every Qwen 3.8 27B build in this guide is abliterated, so in this case both words point at the same files.

Is Qwen 3.8 27B Uncensored an official Alibaba release?

No. Alibaba published Qwen 3.8 27B under Apache 2.0 and the uncensored versions are community edits of those weights, made by individual publishers. Alibaba neither produced nor endorsed them.

How much VRAM does Qwen 3.8 27B uncensored need?

About 17 GB for the 4-bit file, so a 24 GB GPU is the practical minimum with context headroom. A 3-bit build fits 16 GB with around 8K of context. On Apple Silicon, macOS caps GPU memory at about 75 percent of unified memory, so a 24 GB Mac runs the 3-bit build and 32 GB and up runs 4-bit with room for context.

Can I run Qwen 3.8 27B uncensored on a 16GB Mac?

Not usefully. A 3-bit build is 12.8 to 13.5 GB and macOS will only hand the model about 12 GB of a 16 GB machine, so it does not fit. Run a smaller uncensored model instead; our best local LLM for a 16GB Mac roundup covers what fits.

Which uncensored Qwen 3.8 27B build is best?

JonathanColetti's, unless you need something the other two have. It is the best documented of the three: a refusal count and a capability table, both measured against the original weights on the same harness. huihui-ai gives you a single Ollama command, and orcarouter is the only publisher shipping MLX for Apple Silicon.

Do abliterated models get dumber?

It depends how hard the edit was pushed. JonathanColetti scored his build against the original on MMLU, ARC-Challenge, HellaSwag and Winogrande and every difference landed inside the error bars. The OBLITERATUS build of the same model measured a 2.1 point MMLU drop over 5,700 questions, with STEM down 3.3 points, and its own log shows an aggressive pass costing 6 points. Nobody has published math, multilingual or vision numbers for any of them, and the only code result is OBLITERATUS's own 20-prompt pass count, so test the build on your own work before you trust it.

Is Qwen 3.8 27B uncensored free for commercial use?

Yes. Qwen 3.8 27B ships under Apache 2.0 and the uncensored builds inherit it: commercial use, modification, and redistribution are all allowed. What you generate with the model is your responsibility, and local law still applies.

Is it safe to download an uncensored Qwen 3.8 27B build?

The risk here is the wrong file rather than a malicious one. Several repositories carry near-identical names, so check that the card describes an actual method and that the file list holds GGUF shards of a plausible size. One of them is gated, and a login prompt there means a gated repository rather than a broken one.

Does Qwen 3.8 27B uncensored support vision locally?

Yes. The vision tower sits outside the part of the model that abliteration edits, and every build in this guide leaves it alone. In Atomic Chat you attach the image to the chat; through llama.cpp you pass the separate mmproj projector file with --mmproj.

Does Qwen 3.8 27B uncensored work with Ollama or LM Studio?

Yes. ollama run huihui_ai/Qwen3.8-abliterated and ollama run orcarouter/Qwen3.8-27B-Uncensored each pull a 17.7 GB download, a 16.8 GB 4-bit file plus the 0.9 GB vision projector. You need Ollama v0.32.12 or newer, the August 14 release that added Qwen 3.8 support. LM Studio loads the GGUF files from JonathanColetti and huihui-ai, and on Apple Silicon it also loads orcarouter's MLX build. orcarouter's GGUF repo is gated and needs a Hugging Face read token first.

Can I abliterate a model myself?

Yes. Heretic automates the whole search: you install it, point it at a model, and it runs 200 trials to find the ablation that removes the most refusals for the least drift from the original. A 4B model takes roughly half an hour on an RTX 3090. Running the search on a 27B needs more VRAM than a single 3090 has.

Bottom line

If you want an uncensored Qwen 3.8 27B and you have a 24 GB card, download JonathanColetti's Q4_K_M at 16.8 GB and start at 8K of context: of the three, its publisher documented both what the edit removed and what it cost. On a Mac with 32 GB or more, take orcarouter's 4-bit MLX build instead. If you would rather not pick a file at all, ollama run huihui_ai/Qwen3.8-abliterated gets you a working 4-bit model in one command. Nobody has published a refusal number for that one.

Key takeaways:

• Uncensored Qwen 3.8 27B builds are community edits of Alibaba's Apache 2.0 weights. Alibaba did not publish them.
• Abliteration removes the refusal direction from the weights without retraining, so the size and memory arithmetic are unchanged.
• The capability cost scales with how hard the publisher pushed, from nothing measurable up to 2.1 MMLU points on the builds you can download.
• Refusal rates from different publishers are measured on different prompt sets. They cannot be ranked against each other.
• Refusals are reduced, not eliminated. Low-bit quants make the remaining behavior less predictable.