In this guide, you'll learn:
- What an uncensored Inkling 2 Mini is, and what abliteration changed in it
- Which GGUF file to run, and what hardware each one needs
- How to run Inkling 2 Mini uncensored locally with Atomic Chat or llama.cpp
This article is about the uncensored build. For the same edit on a dense model of similar size, see our guide to running Qwen 3.8 27B uncensored locally, and for the wider category, our roundup of the best uncensored LLMs to run locally.
For a 24 GB GPU, start with our AD-Q4_K at 16.53 GB. On a 16 GB Mac, use AD-IQ2_M at 8.67 GB.
What is Inkling 2 Mini Uncensored?
Inkling 2 Mini Uncensored is not a release from Thinking Machines. Thinking Machines published Inkling 2 Mini under Apache 2.0 on November 20, 2026, and within a day we edited the part of those weights that makes the model decline requests and published the result. The license allows it. Other publishers, including OpenUncensored, released their own edits over the first two days.
The underlying model is unchanged:
| Specification | Inkling 2 Mini Uncensored |
|---|---|
| Total parameters | 26.4B |
| Active parameters per token | 3.4B, about 5.4B effective with the loop |
| Architecture | MoE looped transformer, 128 experts with 8 active per token |
| Layers | 24, the middle 12 run three times (48 passes per token) |
| Attention | 4 full-attention layers, 20 sliding-window layers of 1,024 tokens |
| Context window | 1,048,576 tokens |
| Modalities | Text and image input |
| Reasoning | On by default, effort from none to max |
| Speculative decoding | DSpark draft head in the original checkpoint, not in our GGUF files |
| Base model | thinkingmachines/Inkling-2-Mini, released November 20, 2026 |
| License | Apache 2.0, inherited from the base model |
Because the middle 12 layers run three times with the same weights, each token costs more compute than 3.4B active parameters suggest. Only 4 of the 24 layers keep a cache of the whole conversation. In practice, that means long chats and big documents cost less memory here than on a typical model of this size. We'll put numbers on this in the hardware section.
Abliteration edits weights without retraining the model, so the parameter count, the context window and the memory arithmetic all carry over from the original. Our edit ships on its own as a 44 MB LoRA adapter, and we also merged it into three GGUF files. The vision projector is the original's, unchanged. The original checkpoint also carries a DSpark draft head for speculative decoding, and our GGUF files leave it out.
Inkling 2 Mini benchmarks
Thinking Machines' launch numbers, from the model card, compare the Mini with Qwen3.8-27B, Muse Glimmer, Nemotron 3.5 Super VL, and Gemma 4 31B:
| Benchmark | Inkling 2 Mini | Qwen3.8-27B | Muse Glimmer | Nemotron 3.5 Super VL | Gemma 4 31B |
|---|---|---|---|---|---|
Params | 26B MoE (5B effective active) | 27B dense | 30B dense | 125B MoE (12B active) | 31B dense |
4-bit size | ~16.5 GB | ~17 GB | <20 GB | - | ~18 GB |
SWE-bench Verified Software engineering | 79.2 | 77.8 | 76.0 | 71.1 | 46.1 |
SWE-bench Pro (Hard) Harder engineering | 77.9 | 73.8 | 44.3 | 44.1 | 14.2 |
Terminal Bench 2.1 Terminal agents | 71.6 | 73.0 | 51.7 | 60.7 | 43.4 |
BrowseComp Deep search | 79.0 | 61.6 | 63.9 | 78.9 | 37.0 |
MCP Atlas MCP tools | 82.7 | 77.3 | 75.5 | 67.7 | 54.2 |
MMMU Pro Multimodal reasoning | 74.9 | 76.3 | 74.3 | 72.3 | 73.4 |
HLE-Diamond Expert questions | 15.2 | 15.3 | 9.1 | 8.8 | 7.2 |
LongBench v2 Long context | 66.9 | 52.3 | 33.1 | 62.1 | 52.9 |
The Mini comes out ahead of Qwen3.8-27B on five of the eight benchmarks. A Qwen3.8-27B-level model now runs on 3.4B active parameters per token.
All scores come from Thinking Machines' card and are for the original weights. Nobody has run these benchmarks on the uncensored build, and the closest check is the 2,000-question MMLU run in the next section, where the edit cost 1.5 points. A hyphen means we have no 4-bit figure for that model.
The direct rival here is Qwen3.8-27B: a dense vision-language model of almost the same size, released in August. On launch numbers the Mini scores higher on BrowseComp (79.0 vs 61.6) and LongBench v2 (66.9 vs 52.3), while Qwen scores higher on Terminal Bench 2.1 (73.0 vs 71.6), MMMU Pro (76.3 vs 74.9) and HLE-Diamond (15.3 vs 15.2). The two 4-bit files are about the same size, but the Mini needs about an eighth of Qwen's attention cache per token. For local coding on a 24 GB card, the Mini has the best SWE-bench Verified score in the table above.
Inkling 2 Mini is the smaller sibling of Inkling 2 Small, which has 594B parameters with 17B active and takes a multi-GPU server to run. Thinking Machines says the Mini is less capable across most of its evaluations, with the largest gaps on factual recall.
What abliteration does to Inkling 2 Mini
Researchers found that refusal in many aligned models is governed by a single direction in the model's residual stream, the internal state it passes from layer to layer. Arditi et al. showed this across 13 open-weight chat models at NeurIPS 2024: erase that one direction and the model stops declining harmful instructions, add it back and the model starts declining harmless ones.
To find the direction, you run the model over matched sets of harmful and harmless prompts, record the activations, and take the difference in means between the two. Then you remove the model's ability to write to that direction by orthogonalizing the weight matrices against it. The edit changes weights directly. It does not use training data or gradient descent.
The word abliteration does not appear in that paper. It is a community coinage for what the authors called directional ablation.
Abliterated and uncensored point at the same files here. Abliterated names the method. Uncensored names the result, and it also covers models like Dolphin and Hermes that got there by training on extra instruction data instead. For the method on its own, across models and toolchains, see our explainer on what abliterated models are.
The refusal direction lives in the residual stream, but Inkling 2 Mini needs a different edit from a model without a loop. The 12 looped layers share their weights across three passes, so one edit to those weights acts on 36 of the 48 passes. Our adapter changes 49 residual writers: the attention output and the routed experts of all 24 layers, plus the token embedding. In each layer it removes that layer's own refusal direction.
We took the direction from prompts with and without reasoning, because the model reasons even at the lowest effort setting and decides to refuse inside that reasoning. A direction taken without reasoning alone still left soft refusals in chat.
What abliteration costs Inkling 2 Mini
We measured the BF16 model with the adapter against the same model without it, on the same prompts, with greedy decoding:
| Benchmark | Inkling 2 Mini | Uncensored | Change |
|---|---|---|---|
Harmful prompts refused, English (81) | 100% | 0% | -100 pp |
Harmful prompts refused, Russian (100) | 91% | 0% | -91 pp |
Harmful prompts refused, effort high (30) | 26 of 30 | 0 of 30 | -26 |
XSTest safe prompts refused (250) | 4.0% | 0% | -4.0 pp |
Valid tool calls (20) | 18 of 20 | 20 of 20 | +2 |
MMLU, 2,000 questions | 80.65% | 79.15% | -1.50 pp |
The harmful prompts come from the validation and test splits of mlabonne/harmful_behaviors and its Russian translation; the adapter was built from the train split. The English and Russian rows were measured with reasoning off, and the effort-high row with a 3,072-token reasoning budget. At low effort, refusals fell from 28 of 30 to 0. The XSTest row counts over-refusal, safe prompts the model declines anyway.
The MMLU drop of 1.5 points sits outside its error bars, with a paired McNemar p value of 0.006. About half of it comes from editing at all: a random direction applied the same way costs 0.75 points. A harder edit costs more, and at strength 1.25 the same edit loses 2.2 points.
The edit removed over-refusal along with refusals of harmful prompts. This model will answer requests that the original refuses, and you are responsible for how you use it.
Inkling 2 Mini Uncensored GGUF: we built it ourselves
We built the AtomicChat Inkling 2 Mini Uncensored GGUF repository from our GGUF builds of the original weights. It holds three files with the edit merged in, plus the 44 MB LoRA adapter and the vision projector in F16 and BF16. The AD prefix stands for Atomic Dynamic: most of the compression goes into the experts, while attention and the loop controllers stay at Q8_0.
| File | Size | KL divergence | Top-1 match |
|---|---|---|---|
| AD-Q4_K | 16.53 GB | 0.0115 | 95.6% |
| AD-IQ3_S | 13.34 GB | 0.0235 | 93.8% |
| AD-IQ2_M | 8.67 GB | 0.132 | 85.7% |
How to read the table: KL divergence measures the distance between the quant's output distribution and the original weights. Lower values are better, and zero means identical. Top-1 match is the percentage of positions where the quant selects the same next token as the reference. Here the reference is the original model before the edit, so each number includes both the edit and the quantization. Against the same quants of the original weights in the next table, the edit adds a little at 4-bit, and at 2-bit the quantization accounts for nearly all of the distance.
We measured every number in the table ourselves, against the original BF16 weights on the same neutral corpus as the base repository: 87 chunks of 4,096 tokens in 30 languages. The calibration corpus is public, the importance matrix and the per-tensor recipes are in the metrics repo, and every refusal and quality check is on the model card.
More sizes with the Inkling 2 Mini uncensored LoRA
The adapter puts the same edit on 15 of the 17 builds in our original-weights repository, from the 6.88 GB AD-IQ1_S to the 52.2 GB BF16. Q4_K_M and Q5_K_M are still in our earlier layout and do not load on b12600 yet. These are the quality numbers for those files before the edit:
| File | Size | KL divergence | Top-1 match |
|---|---|---|---|
| Q8_0 | 27.74 GB | 0.0033 | 97.7% |
| AD-Q6_K | 23.79 GB | 0.0037 | 97.6% |
| Q6_K | 23.40 GB | 0.0047 | 97.3% |
| Q5_K_M | 19.81 GB | 0.0079 | 96.3% |
| AD-Q5_K | 19.07 GB | 0.0052 | 97.1% |
| Q4_K_M | 17.38 GB | 0.0165 | 94.7% |
| AD-Q4_K | 16.53 GB | 0.0080 | 96.3% |
| AD-IQ4_XS | 15.00 GB | 0.0101 | 95.8% |
| AD-IQ3_S | 13.34 GB | 0.0205 | 94.2% |
| AD-IQ3_XXS | 12.58 GB | 0.0271 | 93.3% |
| AD-IQ2_S | 11.56 GB | 0.047 | 91.3% |
| AD-IQ2_XXS | 10.43 GB | 0.084 | 88.5% |
| AD-IQ2_M | 8.67 GB | 0.130 | 85.7% |
| AD-IQ2_XXS-Q2 | 7.90 GB | 0.158 | 84.3% |
| AD-IQ1_M | 7.26 GB | 0.236 | 81.1% |
| AD-IQ1_S | 6.88 GB | 0.278 | 79.5% |
The adapter needs llama.cpp; the setup is in the llama.cpp section below.
The table holds four llama.cpp presets next to our files, Q8_0, Q6_K, Q5_K_M and Q4_K_M, and their names do not line up with quality. Q4_K_M is 0.85 GB larger than AD-Q4_K and has twice its KL divergence, and AD-Q4_K matches the Q5_K_M preset at 3.3 GB less. A quant name only says which recipe the publisher requested. When you pick a build, go by file size and measured quality.
Which uncensored Inkling 2 Mini build should you run?
At least eight repositories carried an uncensored Inkling 2 Mini within a day of the release, and two of them are GGUF conversions of another publisher's edit. Three are worth your download, for different reasons.
| Build | Method | Files | Vision | Runtime |
|---|---|---|---|---|
| AtomicChat | Rank-1 LoRA on all 49 residual writers | GGUF AD-Q4_K, AD-IQ3_S and AD-IQ2_M, plus a GGUF adapter for 15 of our 17 base builds | F16 and BF16 projector; base-repo Q8_0 checked on b12603 | llama.cpp b12600+, Atomic Chat as its client |
| OpenUncensored | Attention outputs only, layers 4 to 23 | Safetensors in the vendor's NVFP4 and MXFP8, about 20 GB, plus a 1 MB PEFT LoRA | Vision encoder unchanged in the checkpoint | SGLang, Transformers from source |
| JoaoZaokk | Heretic | GGUF UD-Q2_K_XL to UD-Q4_K_XL, five files, plus a DSpark draft | Q8_0 projector | llama.cpp, optional CUDA patch |
AtomicChat is the one to download if you want to know what you are getting. Its card measures refusals in English and Russian with reasoning off and at two effort levels with it on, plus over-refusal, tool calls and MMLU, all against the original on the same harness. It is also the only build here with a GGUF adapter, so llama.cpp can put the edit on our files from 6.88 GB to BF16.
OpenUncensored is where to start on a 32 GB GPU with SGLang. Its edit is the lightest of the three: only the 20 attention output projections change, and every expert stays byte for byte the original. Its card reports no MMLU loss. The checkpoint is about 20 GB, in the vendor's own NVFP4 format. It is a server build, and Atomic Chat loads GGUFs instead. Abiray and KookiesXy publish GGUF conversions of it, and KookiesXy says plainly that its own load tests are not a quality benchmark.
JoaoZaokk is the one to look at if you want a DSpark draft file next to the GGUF, and its card has throughput and long-context runs on a single RTX 3090. Its chosen build still refuses 20.0 percent of chat prompts and 58.3 percent with thinking on, against 100 percent for the original.
Why the refusal numbers don't line up
All three builds publish a refusal rate. No two of those numbers were measured the same way.
| Build | Refusal rate | Prompt set | Conditions |
|---|---|---|---|
| AtomicChat | 0 of 81, from 81 of 81 | mlabonne/harmful_behaviors test split, plus 100 Russian prompts | BF16 with the adapter, greedy, reasoning off; also 0 of 30 at low and high effort |
| OpenUncensored | 0 of 32 in chat and 0 of the 29 that finished thinking, from 32 of 32; 8 of 80 on a mixed set, from 68 | 32 harmful and 48 gray-area prompts of their own | Chat and thinking at effort 0.9, greedy, on the merged files |
| JoaoZaokk | 20.0% chat, 58.3% thinking, from 100% | Not named on the card | On the UD-Q4_K_M file |
The rows differ in the prompt set, the number of prompts, who or what decides that an answer counts as a refusal, and whether thinking was on. A response that opens with a warning and then delivers exactly what was asked scores as a refusal under one publisher's judge and as compliance under another's. OpenUncensored counts an answer as a refusal when it opens with phrases like "I can't" or "I'm sorry", and it classes 7 of its 8 remaining refusals as limits of what the model can know.
MMLU has the same problem. Our run lost 1.5 points on 2,000 questions, and OpenUncensored reports a gain from 78.95% to 79.47% on 1,140 questions scored by letter logits. The two results use different harnesses and are not comparable.
A refusal rate describes one publisher's own test, on their prompts and their judge. Run your own prompts against a build before you rely on it.
Inkling 2 Mini uncensored hardware requirements
The system requirement to check is memory. The table below shows the memory your GPU can use: VRAM on a graphics card, or the share of unified memory macOS gives the GPU, about two thirds on Macs up to 32 GB and about three quarters from 36 GB. The memory a file needs is its size, plus the KV cache and the compute buffer, plus about 1 GB of fixed buffers:
| Available memory | Recommended GGUF quant | File size | Leaves room for |
|---|---|---|---|
| 8 GB | AD-IQ1_S with the adapter, in llama.cpp | 6.88 GB | Short context, with some expert layers on CPU |
| 12 GB | AD-IQ2_M | 8.67 GB | Around 12K of context |
| 16 GB | AD-IQ3_S | 13.34 GB | Around 8K of context |
| 24 GB | AD-Q4_K | 16.53 GB | Around 40K of context |
| 32 GB | AD-Q4_K | 16.53 GB | Around 96K of context |
| 48 GB or more | Q8_0 with the adapter, in llama.cpp | 27.74 GB | Around 128K of context |
Note: if you are choosing between two neighbouring files and both still fit in memory, take the larger one. It costs a few gigabytes more and picks the wrong token noticeably less often, especially below 13 GB, where top-1 match falls from 93.8% to 85.7%.
Low-bit builds carry a risk here that they do not carry on the original model. Our refusal numbers come from the BF16 model with the adapter, and the merged AD-Q4_K passed the 8-prompt chat check too. Nothing in the repository measures refusals on AD-IQ3_S or AD-IQ2_M, so use AD-Q4_K or larger when you need to judge the edit itself.
How much memory does the context window add?
The figures above only cover the model weights. On top of that, the engine allocates a KV cache: the longer you chat, the more context the model holds in memory, and the more gigabytes that takes. The same goes for pasting big documents or codebases into the conversation.
Only 4 of the 24 layers use full attention. Two of them sit inside the loop, and every pass keeps its own cache, so the model keeps 32 KB of attention cache per token. The 20 sliding-window layers stop growing at about 0.25 GB. llama.cpp also reserves a compute buffer that grows with the context, about 96 KB per token at the default micro-batch (-ub) of 512, three times the cache. Both figures come from the configuration and llama.cpp's code:
| Context length | Attention cache | With the compute buffer |
|---|---|---|
| 8K (typical chat) | ~0.5 GB | ~1.3 GB |
| 32K (long document) | ~1.3 GB | ~4.5 GB |
| 128K (large codebase) | ~4.5 GB | ~17 GB |
| 256K | ~8.8 GB | ~35 GB |
| 1M (maximum context) | ~35 GB | ~138 GB |
The full 1M window needs server hardware: the cache alone is about 35 GB, twice the 4-bit file. The hardware table above counts the compute buffer and the fixed buffers too, and the troubleshooting section shows how to shrink the compute buffer.
What hardware can run Inkling 2 Mini uncensored?
- RTX 3090 / RTX 4090 (24 GB VRAM): can run AD-Q4_K entirely in VRAM with room for about 40K of context.
- RTX 5090 (32 GB VRAM): can run AD-Q4_K with about 96K of context, or AD-Q6_K with the adapter and about 48K.
- 16 GB GPUs (RTX 4060 Ti, RTX 5060 Ti): can fit AD-IQ3_S with around 8K of context. Inkling 2 Mini is a Mixture-of-Experts model, so keeping a few expert layers of AD-Q4_K in system RAM costs less here than it does on the dense Qwen 3.8 27B: each token reads only the 8 experts it routes to. On NVIDIA cards that split currently runs into an open llama.cpp bug, covered in the troubleshooting section, so AD-IQ3_S entirely in VRAM is the safer choice for now.
- 12 GB GPUs (RTX 3060 12 GB, RTX 4070): can run AD-IQ2_M entirely in VRAM with around 12K of context.
- 8 GB GPUs (RTX 4060, RTX 5060): can run AD-IQ1_S with the adapter in llama.cpp only with some expert layers on the CPU, and on NVIDIA cards that split can run into the same open bug. None of the three merged files fits, and a smaller uncensored model from our uncensored roundup is the better pick for now.
- MacBook Air, MacBook Pro (16-32 GB unified memory): macOS gives the GPU about two thirds of unified memory on these machines, so a 16 GB Mac can run AD-IQ2_M with around 8K of context, a 24 GB Mac AD-IQ3_S with around 8K, and a 32 GB Mac AD-Q4_K with around 32K.
- Macs with 36-64 GB (MacBook Pro M4/M5 Max, Mac Studio): can run AD-Q4_K with long context from 36 GB, and Q8_0 with the adapter from 48 GB up, since macOS gives the GPU about three quarters of unified memory from 36 GB. On Apple Silicon the GGUF builds run natively through Metal; see our GGUF vs MLX guide if you're choosing between formats.
- Phones: cannot run these files. For an uncensored model that runs on a phone, see the smaller entries in our uncensored roundup.
How to run Inkling 2 Mini uncensored in Atomic Chat through your own llama-server
Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.
As of November 21, 2026, the newest llama.cpp build approved for Atomic Chat is b12532, and Inkling 2 Mini needs b12600 or newer, so the bundled engine cannot load these files yet. Our engine manifest points the app at new llama.cpp builds, and the app pulls them without an app release. Once b12600 or newer is approved there, the model loads like any other GGUF in the app.
The model still runs in the app today, on a route that goes around the bundled engine: you start llama-server yourself and connect Atomic Chat to it as a client. The desktop app keeps the servers you run yourself on its Custom Endpoint page, and llama.cpp server is already listed there.
Here's how to run Inkling 2 Mini uncensored with Atomic Chat as a client:
Step 1: Install Atomic Chat
Download Atomic Chat from atomic.chat and install the build for your platform:
- macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
- Windows: an .exe installer for x64
- Linux: a self-contained .AppImage for x86_64, which needs no root
On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.
Step 2: Start llama-server on the model
Build llama.cpp b12600 or newer as shown in the llama.cpp section below, or take a prebuilt b12600 or newer binary for your platform from the llama.cpp releases page. Then start the server on our AD-Q4_K:
./build/bin/llama-server \ -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:AD-Q4_K \ --no-mmproj \ --alias inkling-2-mini-uncensored \ --jinja --temp 1.0 \ -ngl 99 -c 8192 \ --host 127.0.0.1 --port 8080
Leave it running. On a 16 GB card or a 24 GB Mac, replace AD-Q4_K with AD-IQ3_S, and on a 12 GB card or a 16 GB Mac with AD-IQ2_M. Any port works, and 8080 is only the value the app prefills.
Step 3: Configure llama.cpp server on the Custom Endpoint page
Click Custom Endpoint in the sidebar, then click the gear icon on the llama.cpp server row. The Base URL is already filled with http://localhost:8080/v1. Change the host or the port if your llama-server runs somewhere else, and keep the /v1 suffix. Leave the API key empty unless you started llama-server with --api-key. Click Save endpoint.
Step 4: Check the connection and load the model list
Click Ping on the llama.cpp server row. The app asks the server for its model list and shows the latency and the number of models it found. Saving the endpoint already opened its model list and reloaded it, so inkling-2-mini-uncensored, the --alias from Step 2, is listed there. If it is not, click Reload models, or click the + button next to it, type inkling-2-mini-uncensored into the Model ID field of the Add New Model dialog, and click Add Model.
Step 5: Chat locally
Open a chat, pick inkling-2-mini-uncensored in the model picker, and send. Send it a prompt the original declines, to check that the build you loaded is the edited one.
The effort slider works on this path: the app passes low, medium or high to llama-server as a chat template setting, and its two top positions send high as well. The lightbulb's off position sends a switch the Inkling 2 Mini template does not read, so the model keeps reasoning at its default, high. To get the lowest level from the app, start llama-server with --chat-template-kwargs '{"reasoning_effort":"none"}' and switch the lightbulb off, so the server's own setting applies. Thinking Machines notes that even none does not switch reasoning off completely.
Atomic Chat's Local API Server at http://localhost:1337/v1 sends requests for this model on to your llama-server, so coding agents like Claude Code and Cline that point at it reach the model too.
What you get on this path, and what you do not
On this path Atomic Chat is the client and llama-server runs the model, so part of what the app normally does for a local model is now yours to do:
- You start and stop llama-server yourself. The app shows no memory-fit estimate for this model, and Reload models finds nothing until the server is already up.
- The app's llama.cpp settings do not reach it. Context size, GPU layers and the micro-batch that sizes the compute buffer are llama-server flags on this path: -c, -ngl and -ub.
- Capabilities are not detected. Every model on this endpoint starts with text completion only: click the pencil icon on the model's row and turn on Vision and Tools, and image input also needs llama-server started with the projector from the vision section below.
- The app connects to llama.cpp server without a key at any address, so a server on another machine is open to anyone on that network. For a LAN box, start llama-server with --api-key and enter that key in the app.
How to run Inkling 2 Mini uncensored with llama.cpp
You might prefer to run the model directly through llama.cpp if you need:
- A server you can start headless, with the host and port set by flag
- Precise control over GPU offload
- The LoRA adapter on a file size we did not merge
Inkling 2 Mini uses the inkling2_loop architecture, which upstream llama.cpp has supported since release b12600 on November 20, 2026 (PR #32694). We re-uploaded the three model files and the adapter in that layout on November 21, so they load on b12600 or newer with no patches.
Step 1: Build current llama.cpp
For NVIDIA CUDA:
git clone https://github.com/ggml-org/llama.cpp cd llama.cpp cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON cmake --build build -j --target llama-cli llama-server
For Apple Silicon, Metal is enabled by default:
cmake -B build -DCMAKE_BUILD_TYPE=Release cmake --build build -j --target llama-cli llama-server
Step 2: Run the model
The following command downloads AD-Q4_K from Hugging Face, offloads all layers to the GPU, applies Thinking Machines' recommended temperature, and limits the context to 8K:
./build/bin/llama-cli \ -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:AD-Q4_K \ --no-mmproj \ --jinja \ --temp 1.0 \ -ngl 99 \ -c 8192
The --no-mmproj flag keeps llama.cpp from fetching the projector in this repository; the vision section below uses a different file. Replace AD-Q4_K with AD-IQ3_S or AD-IQ2_M if your system has less memory. If the file is larger than your VRAM, keep attention on the GPU and move the experts of the last six layers, which run only once per token, to system RAM. On NVIDIA cards, read the CUDA note in the troubleshooting section first:
./build/bin/llama-cli \ -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:AD-Q4_K \ --no-mmproj \ --jinja --temp 1.0 \ -ngl 99 -c 8192 \ -ot "blk\.(1[8-9]|2[0-3])\.ffn_.*_exps\.=CPU"
Step 3: Expose a local OpenAI-compatible API
Replace llama-cli with llama-server:
./build/bin/llama-server \ -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:AD-Q4_K \ --no-mmproj \ --alias inkling-2-mini-uncensored \ --jinja \ --temp 1.0 \ -ngl 99 \ -c 8192 \ --host 127.0.0.1 --port 8080
Test it with:
curl http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{ "model": "inkling-2-mini-uncensored", "chat_template_kwargs": { "reasoning_effort": "low" }, "messages": [ { "role": "user", "content": "Summarize what a looped transformer does differently." } ] }'The reasoning level goes in chat_template_kwargs: none, minimal, low, medium, high, xhigh or max, with high as the template's default. Bind the server to 127.0.0.1 unless other machines on your network need access.
Running vision through llama.cpp
Image input needs one extra file, the vision projector. Our edit leaves it unchanged, so the projector from our original-weights repository works with the uncensored files. We checked its 0.35 GB Q8_0 file on llama.cpp b12603. Download it next to the quant and pass both:
hf download AtomicChat/Inkling-2-Mini-Uncensored-GGUF \ Inkling-2-Mini-Uncensored-AD-Q4_K.gguf --local-dir . hf download AtomicChat/Inkling-2-Mini-GGUF \ mmproj-Inkling-2-Mini-Q8_0.gguf --local-dir . ./build/bin/llama-server \ -m Inkling-2-Mini-Uncensored-AD-Q4_K.gguf \ --mmproj mmproj-Inkling-2-Mini-Q8_0.gguf \ --alias inkling-2-mini-uncensored \ --jinja --temp 1.0 \ -ngl 99 -c 32768 \ --host 127.0.0.1 --port 8080
Then send the image through the API, attach it in the llama-server web UI at http://127.0.0.1:8080, or attach it in Atomic Chat after turning on Vision for the model. If image input does nothing, the projector is missing: pass the mmproj file with --mmproj.
Running the LoRA adapter on another file
The adapter holds the whole edit in 44 MB. Download Inkling-2-Mini-Uncensored-LoRA.gguf from the uncensored repository, pick an AD file or Q8_0 from our original-weights repository, and pass both:
./build/bin/llama-server \ -m Inkling-2-Mini-AD-Q6_K.gguf \ --lora Inkling-2-Mini-Uncensored-LoRA.gguf \ --mmproj mmproj-Inkling-2-Mini-Q8_0.gguf \ --jinja -c 32768 -ngl 99
We measured the adapter at strength 1.0, which is what --lora applies. Use the same command with Q8_0 on a 48 GB machine, or with AD-IQ1_S at 6.88 GB when nothing larger fits.
Running it with Ollama or LM Studio
Ollama's library tag, ollama run inkling-2-mini, is the original model from Thinking Machines: a 20 GB MLX build in the vendor's NVFP4 precision. The official Ollama library has no uncensored Inkling 2 Mini tag.
LM Studio lists the original weights through lmstudio-community's GGUF repository, published on release day.
Troubleshooting
The model still refuses
Our edit took refusals to zero on every test set, with reasoning off and at low and high effort. If you still get refusals, check three things:
- Confirm you loaded the build you meant to load. The original weights and the edited ones sit under very similar names on Hugging Face.
- Check that llama-server printed the LoRA file while loading, if you use the adapter. A run without --lora is the original model.
- Move up to AD-Q4_K. Refusals were measured on the BF16 model and checked on AD-Q4_K, and nothing in the repository measures them on the smaller files.
An answer that opens with a caution line and then does what you asked is expected from this build.
llama.cpp does not load the model
Loading stops with check_tensor_dims: tensor 'blk.24.attn_q.weight' not found. The file was built for a different pull request, #32695, which is still open and stores the loop as 48 layers, while the merged llama.cpp reads 24 layers plus the loop settings. We converted our files on November 21 under the same names, so if you downloaded one before 07:40 UTC that day, download it again. If llama.cpp instead says unknown model architecture: inkling2_loop, your build is older than b12600; update it.
The build you downloaded is not what it claims
Uncensored Inkling 2 Mini repositories appeared within hours of the release, and some of them are GGUF conversions of another publisher's edit, so check which edit a repository actually carries. Before downloading from an unfamiliar repository, check that the model card describes an actual method, that the file list contains GGUF files of a plausible size, and that the publisher is one you can trace. A 26B model at 4-bit cannot weigh 400 MB.
The model keeps reasoning and never answers
Thinking Machines reports that Inkling 2 Mini sometimes gets stuck in long reasoning loops, especially when debugging code. The edit does not change this: in our test, 2 of 30 harmless prompts at high effort did not finish reasoning within 3,072 tokens, the same count as the original. Lower the effort to low or medium and set a limit on output tokens.
Output turns into one repeated token on an NVIDIA card
llama.cpp has an open issue for this architecture on CUDA, issue #32710. When the experts of the looped layers sit in system RAM, evaluation returns NaN and llama-server answers with one repeated token: in one report from a 51-token prompt, in another only past about 7,400 tokens, and there the output stayed broken until the server restarted. A CPU-only run (-dev none) is correct, and -ngl 0 on a CUDA build is not. Until the fix lands, keep the whole model in VRAM, with a smaller quant if that is what fits, and restart llama-server if the output breaks.
You run out of memory at long context
The weights fit, then a long conversation crashes the model, or llama-server fails to start at a large context. On this model the usual cause is the compute buffer: llama.cpp reserves about 96 KB of it per token of context at the default -ub 512, three times the 32 KB of attention cache. Start llama-server with an explicit -c instead of the full window, and pass -ub 256 to halve the buffer, at some cost in prompt processing speed. If you need more room, step one quant down: every gigabyte you free adds about 7.5K of context.
Frequently asked questions
The questions that come up most often about running an uncensored Inkling 2 Mini on your own hardware.
What is the difference between abliterated and uncensored models?
Abliterated describes how the model was changed: one direction in the weights, the one that governs refusal, was removed. Uncensored describes the result, and it also covers models that got there through fine-tuning rather than weight editing. Every Inkling 2 Mini build in this guide is abliterated, so in this case both words point at the same files.
Is Inkling 2 Mini Uncensored an official Thinking Machines release?
No. Thinking Machines published Inkling 2 Mini under Apache 2.0, and the uncensored versions are edits of those weights made by other publishers, including us. Thinking Machines neither produced nor endorsed them.
How much VRAM does Inkling 2 Mini uncensored need?
About 17 GB for the 4-bit AD-Q4_K, so a 24 GB GPU runs it with around 40K of context. AD-IQ3_S fits a 16 GB card with around 8K, and AD-IQ2_M runs on 12 GB with around 12K. On Apple Silicon, macOS gives the GPU about two thirds of unified memory up to 32 GB, so a 16 GB Mac runs AD-IQ2_M and 32 GB and up runs AD-Q4_K.
Can I run Inkling 2 Mini uncensored on a 16GB Mac?
Yes. macOS gives the model about 11 GB of a 16 GB machine, and AD-IQ2_M at 8.67 GB fits with around 8K of context. Expect lower quality than at 4-bit: AD-IQ2_M picks the same next token as the original 85.7% of the time. For other models that fit this hardware, see our best local LLM for a 16GB Mac roundup.
Which uncensored Inkling 2 Mini build is best?
Ours, if you run GGUF in llama.cpp, on its own or with Atomic Chat connected to llama-server. It is the best documented of the three: refusals with and without reasoning, measured against the original on the same harness. The GGUF adapter puts the edit on other sizes, from 6.88 GB to BF16. OpenUncensored's edit is lighter and reports no MMLU loss, so it is the better choice on a 32 GB GPU with SGLang.
Do abliterated models get dumber?
It depends how hard the edit was pushed. Ours lost 1.5 MMLU points on 2,000 questions, and a stronger pass of the same edit lost 2.2. OpenUncensored, which edits only the attention outputs, reports no loss on a 1,140-question run with a different harness. No uncensored build has published SWE-bench or other standard coding and agent benchmarks, so test the build on your own work before you rely on it.
Is Inkling 2 Mini uncensored free for commercial use?
Yes. Inkling 2 Mini ships under Apache 2.0 and the uncensored builds inherit it: commercial use, modification, and redistribution are all allowed. Thinking Machines also publishes an acceptable use policy alongside its model card. What you generate with the model is your responsibility, and local law still applies.
Does Inkling 2 Mini uncensored support vision locally?
Yes. Our edit leaves the vision projector unchanged, so the Q8_0 projector from our original-weights repository works with the uncensored files. Start llama-server with it through --mmproj, then attach the image in the llama-server web UI or in Atomic Chat with Vision turned on for the model.
Does Inkling 2 Mini uncensored work with Ollama or LM Studio?
Not through their official catalogs. Ollama's inkling-2-mini tag and lmstudio-community's GGUF repository both carry the original model. For the uncensored build, use llama.cpp b12600 or newer with our GGUF files, on its own or with Atomic Chat as the client.
How does Inkling 2 Mini uncensored compare to Qwen 3.8 27B uncensored?
Both fit a 24 GB card at 4-bit, and on launch numbers the original Mini leads Qwen 3.8 27B on five of the eight benchmarks in our table above. The Mini runs 3.4B parameters per token against Qwen's 27B and holds up to 1M tokens of context against 262K. Their refusal numbers come from different test sets, so run your own prompts on both. The Qwen 3.8 27B uncensored guide covers the community builds of that model.
Where can I download Inkling 2 Mini Uncensored?
The uncensored GGUF files and the LoRA adapter are in our AtomicChat/Inkling-2-Mini-Uncensored-GGUF repository on Hugging Face. Thinking Machines does not publish an uncensored version.
Bottom line
If you want an uncensored Inkling 2 Mini and you have a 24 GB card or a 32 GB Mac, run our AD-Q4_K at 16.53 GB on llama-server b12600 or newer and start at 8K of context: its card documents both what the edit removed and what it cost. Atomic Chat connects to that server as a client today, and loads the file itself once its engine reaches b12600. On a 16 GB card take AD-IQ3_S, and on a 16 GB Mac AD-IQ2_M. For a size we did not merge, put the 44 MB adapter on an AD file or Q8_0 of our original-weights repository in llama.cpp.
Key takeaways:
- Inkling 2 Mini Uncensored is our abliterated edit of Thinking Machines' Apache 2.0 weights. Thinking Machines did not publish it.
- Abliteration removes the refusal direction from the weights without retraining, so the size and memory arithmetic are unchanged.
- Our edit took refusals to zero on every test set.
- The edit cost 1.5 MMLU points on 2,000 questions.
- Long context costs memory on top of the file: about 1.3 GB at 8K and 4.5 GB at 32K on current llama.cpp.
- Refusal rates from different publishers are measured on different prompt sets. They cannot be ranked against each other.

