In this guide, you'll learn:
- What Inkling 2 Mini is and how it differs from Inkling 2 Small
- What hardware you need to run it
- How to run Inkling 2 Mini locally with Atomic Chat or llama.cpp
If you want the dense 27B that Thinking Machines benchmarks it against, and our GGUF builds of it, see our guide to running Qwen 3.8 27B locally. This article is only about Inkling 2 Mini.
What is Inkling 2 Mini?
Inkling 2 Mini is a 26.4B open-weight mixture-of-experts model developed by Thinking Machines, the smaller sibling of the 594B Inkling 2 Small. Thinking Machines released the weights on November 20, 2026 under the Apache 2.0 license. It is the first looped transformer the lab has published.
Inkling 2 Mini main specs:
| Specification | Inkling 2 Mini |
|---|---|
| Total parameters | 26.4B, 3.4B active (5.4B effective with the loop) |
| Architecture | Looped mixture of experts, 8 of 128 experts per token, hybrid local and global attention |
| Layers | 24 stored; the middle 12 run three times, 48 layer passes per token |
| Context window | 1,048,576 tokens (1M) |
| Modalities | Text and image input |
| Reasoning | Thinking on by default at high effort, with levels from none to max |
| Multi-Token Prediction | Three-layer draft head included in the checkpoint |
| Original weights | NVFP4 and MXFP8, about 20 GB |
| Release date | November 20, 2026 |
| License | Apache 2.0 |
The Mini is multimodal, meaning it can process images alongside text. It is also a looped transformer: it stores 24 layers and runs the middle 12 three times with the same weights, so each token passes through 48 layers. Each token uses 3.4B of the 26.4B parameters, and with the loop it costs about as much compute as a 5.4B model. Five of every six layers look back over a 1,024-token window, and only the sixth keeps a regular KV cache. In practice, that means the attention cache for long chats and big documents is far smaller here than on a typical 26B model. We'll put numbers on this in the hardware section.
The model is also trained for Multi-Token Prediction, so it can draft several tokens per forward pass. That's the same mechanism behind speculative decoding, and the three-layer prediction head ships inside the checkpoint; SGLang runs it, and llama.cpp support is an open pull request.
The Inkling 2 release includes three related models, and they're easy to confuse:
- Inkling 2 Small is the 594B-parameter MoE flagship with 17B active parameters. Its main checkpoint needs a GPU cluster with at least 500 GB of VRAM, the same class of hardware as Kimi K3.
- TML-d24-varloop-research is a research checkpoint the size of the Mini, released alongside it, where you choose the number of loop passes at inference time. Thinking Machines has not tuned it for deployment.
- Inkling 2 Mini is the model this guide is about, the release version with a fixed three passes, and at 4-bit it runs on an everyday machine: a desktop with a 24 GB GPU, or a MacBook with 32 GB of unified memory.
Inkling 2 Mini benchmarks
Thinking Machines' launch numbers, from the model card, compare the Mini with Qwen 3.8 27B, Muse Glimmer 30B, Nemotron 3.5 Super VL, and Gemma 4 31B:
| Benchmark | Inkling 2 Mini | Qwen 3.8 27B | Muse Glimmer 30B | Nemotron 3.5 Super VL | Gemma 4 31B |
|---|---|---|---|---|---|
Params | 26.4B MoE (A3.4B) | 27B dense | 30B dense | 125B MoE (A12B) | 31B dense |
4-bit GGUF size | 16.5 GB | ~17 GB | <20 GB | - | ~18 GB |
SWE-bench Verified Software engineering | 79.2 | 77.8 | 76.0 | 71.1 | 46.1 |
SWE-bench Pro (Hard) Harder engineering | 77.9 | 73.8 | 44.3 | 44.1 | 14.2 |
Terminal Bench 2.1 Terminal agents | 71.6 | 73.0 | 51.7 | 60.7 | 43.4 |
BrowseComp Deep search | 79.0 | 61.6 | 63.9 | 78.9 | 37.0 |
MCP Atlas MCP tools | 82.7 | 77.3 | 75.5 | 67.7 | 54.2 |
MMMU Pro Multimodal reasoning | 74.9 | 76.3 | 74.3 | 72.3 | 73.4 |
LongBench v2 Long context | 66.9 | 52.3 | 33.1 | 62.1 | 52.9 |
The Mini comes out ahead of Qwen 3.8 27B on five of the seven benchmarks. At the same file size, the Mini computes with 5.4B effective parameters per token against 27B.
All scores in this table come from Thinking Machines' model card, and the lab ran some rows with its own agent setup for the Mini, such as BrowseComp with context compaction, so small gaps between columns don't mean much. The 4-bit sizes are our own GGUF builds for the Mini and Qwen 3.8 27B and each vendor's figure for the rest; a hyphen means the vendor didn't publish that number. The card's SWE-bench Pro row is the Hard subset of v2.
The direct rival here is Qwen 3.8 27B: a dense vision-language model of almost the same size, and one of the strongest dense models for local coding you can fit on a 24 GB card. On launch numbers the Mini scores higher on SWE-bench Pro (Hard) (77.9 vs 73.8) and LongBench v2 (66.9 vs 52.3), while Qwen leads on Terminal Bench 2.1 and MMMU Pro. Artificial Analysis's Intelligence Index, quoted on the card, puts both at 20.1.
The other reference for the Mini is its bigger sibling. Thinking Machines says the Mini is less capable than Inkling 2 Small across most of its evaluations, with the largest gaps on factual recall and knowledge-heavy questions.
Inkling 2 Mini GGUF: we quantized it ourselves
We built the AtomicChat Inkling 2 Mini GGUF repository from Thinking Machines' original weights, then measured every build against the BF16 reference. The lab ships the weights in NVFP4 and MXFP8, so our BF16 is that checkpoint dequantized. The repository holds fourteen quantized builds that load in upstream llama.cpp, from 6.88 to 27.74 GB, plus the vision projector, which is shared by all of them. The AD prefix stands for Atomic Dynamic: each tensor gets its own type. The experts hold 94% of the weights and get the low-bit types, while attention and the loop controllers stay at Q8_0.
| File | Size | KL divergence | Top-1 match |
|---|---|---|---|
| Q8_0 | 27.74 GB | 0.0033 | 97.7% |
| AD-Q6_K | 23.79 GB | 0.0037 | 97.6% |
| Q6_K | 23.40 GB | 0.0047 | 97.3% |
| AD-Q5_K | 19.07 GB | 0.0052 | 97.1% |
| AD-Q4_K | 16.53 GB | 0.0080 | 96.3% |
| AD-IQ4_XS | 15.00 GB | 0.0101 | 95.8% |
| AD-IQ3_S | 13.34 GB | 0.0205 | 94.2% |
| AD-IQ3_XXS | 12.58 GB | 0.0271 | 93.3% |
| AD-IQ2_S | 11.56 GB | 0.047 | 91.3% |
| AD-IQ2_XXS | 10.43 GB | 0.084 | 88.5% |
| AD-IQ2_M | 8.67 GB | 0.130 | 85.7% |
| AD-IQ2_XXS-Q2 | 7.90 GB | 0.158 | 84.3% |
| AD-IQ1_M | 7.26 GB | 0.236 | 81.1% |
| AD-IQ1_S | 6.88 GB | 0.278 | 79.5% |
How to read the table: KL divergence measures the distance between the quant's output distribution and the original weights. Lower values are better, and zero means identical. Top-1 match is the percentage of positions where the quant selects the same next token as the reference. For a single summary measure, use top-1 match. In the next section we'll explain which file fits your hardware.
We measured every number in the table ourselves. The reference is the BF16 dequantized from the lab's checkpoint, the calibration corpus is public, and the raw logs, the importance matrix, and the per-tensor recipes are in the metrics repo, so you can check any of it yourself.
Inkling 2 Mini hardware requirements
For Inkling 2 Mini, the system requirement to check is memory. The table below shows the memory your GPU can use: VRAM on a graphics card, or the share of unified memory macOS gives the GPU, which the Mac bullets further down work out:
| Available memory | Recommended GGUF quant | File size | Leaves room for |
|---|---|---|---|
| 8 GB | AD-IQ1_S | 6.88 GB | Short context, with some expert layers on CPU |
| 12 GB | AD-IQ2_M | 8.67 GB | Around 12K of context |
| 16 GB | AD-IQ3_S | 13.34 GB | Around 8K of context |
| 24 GB | AD-Q5_K | 19.07 GB | Around 24K of context |
| 32 GB | AD-Q6_K | 23.79 GB | Around 48K of context |
| 48 GB or more | Q8_0 | 27.74 GB | Around 128K of context |
Note: if you are choosing between two neighbouring files, take the larger one. It costs a few gigabytes more and picks the wrong token noticeably less often, especially below 12 GB, where quality falls fastest.
How much memory does the context window add?
The figures above only cover the model weights. On top of that, the engine allocates a KV cache: the longer you chat, the more context the model holds in memory, and the more gigabytes that takes. The same goes for pasting big documents or codebases into the conversation.
In a conventional transformer, every layer would store attention data for every token. Here, five of every six layers use a 1,024-token sliding window, so their cache stops growing at about 0.25 GB. Only the full-attention layers keep growing, and since the loop runs two of them three times, the model keeps 32 KB of attention cache per token. llama.cpp also reserves a compute buffer that grows with the context, about 96 KB per token at the default micro-batch (-ub) of 512, three times the cache:
| Context length | Attention cache | With the compute buffer |
|---|---|---|
| 8K (typical chat) | ~0.5 GB | ~1.3 GB |
| 32K (long document) | ~1.3 GB | ~4.5 GB |
| 128K (large codebase) | ~4.5 GB | ~17 GB |
| 256K | ~8.8 GB | ~35 GB |
| 1M (maximum context) | ~35 GB | ~138 GB |
The full 1M window needs server hardware: the cache alone is about 35 GB, more than the 27.74 GB Q8_0 file. The hardware table above counts the compute buffer and about 1 GB of fixed buffers too, and the troubleshooting section shows how to shrink the compute buffer.
What hardware can run Inkling 2 Mini?
- RTX 3090 / RTX 4090 (24 GB VRAM): can run AD-Q5_K entirely in VRAM with room for about 24K of context.
- RTX 5090 (32 GB VRAM): can run AD-Q6_K entirely in VRAM with around 48K of context, and at that quant the model scores within 0.1 points of Q8_0 on top-1 match.
- 16 GB GPUs (RTX 4060 Ti, RTX 5060 Ti): can fit AD-IQ3_S with around 8K of context, or AD-Q4_K with a few expert layers kept in system RAM through
--n-cpu-moe. The Mini is a MoE, so offloading costs less here than it does on a dense model like Qwen 3.8 27B: each token reads only the 8 experts it routes to. For other 16 GB options, see our best local LLM for 16GB roundup. - 8 GB GPUs (RTX 4060, RTX 5060): can run AD-IQ1_S with a short context and some expert layers on the CPU, at 79.5% top-1 agreement with the original. For that hardware, see also our best local LLM for 8GB roundup.
- MacBook Air, MacBook Pro (16-32 GB unified memory): macOS gives the GPU about two thirds of unified memory on these machines, so a 16 GB Mac can run AD-IQ2_M with around 8K of context, a 24 GB Mac AD-IQ3_S with around 8K, and a 32 GB Mac AD-Q4_K with around 32K.
- Macs with 36-64 GB (MacBook Pro M4/M5 Max, Mac Studio): can run AD-Q6_K from 36 GB and Q8_0 from 48 GB up, since macOS gives the GPU about three quarters of unified memory from 36 GB. On Apple Silicon the GGUF builds run natively through Metal; see our GGUF vs MLX guide if you're choosing between formats.
- Phones: no. The smallest file is 6.88 GB. For a model that runs on a phone, use the smaller members of the Qwen family from our Qwen guide.
How to run Inkling 2 Mini locally with Atomic Chat and llama-server
Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.
As of Atomic Chat 2.4.7, the bundled engine does not load Inkling 2 Mini: it runs llama.cpp b12532, and support for the looped architecture arrived in b12600. The model still runs in the app today, on a route that goes around the bundled engine: you start llama-server from llama.cpp b12600 or later yourself and connect Atomic Chat to it as a client. The app ships a built-in provider whose whole purpose is a llama-server you run yourself. When our engine moves to b12600 or later, the app pulls it without an app release.
Here's how to run Inkling 2 Mini with Atomic Chat as a client:
Step 1: Install Atomic Chat
Download Atomic Chat from atomic.chat and install the build for your platform:
- macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
- Windows: an .exe installer for x64
- Linux: a self-contained .AppImage for x86_64, which needs no root
On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.
Step 2: Start llama-server on the model
Build llama.cpp b12600 or later and start the server on one of our files. The llama.cpp section below has the build commands and the full flag set; the short form is:
llama-server -hf AtomicChat/Inkling-2-Mini-GGUF:AD-Q4_K --jinja -ngl 99 -c 8192 --port 8080
Replace AD-Q4_K with the file from the hardware table that fits your machine. Leave the server running. Any port works, and 8080 is only the value the app prefills.
Step 3: Open Cloud and pick llama.cpp server
Click Cloud in the left sidebar, which is where every provider the app reaches over HTTP lives, your own server included. In the Connection card, open the Select a provider dropdown: the list starts with a Self-hosted group, and the entry to take is llama.cpp server. That entry is the client one, not the bundled Llama.cpp engine that runs files for you.
Settings → Model Providers is the wrong door here. That submenu lists the local engines, and this provider's route in it redirects back to Cloud.
Step 4: Point it at your server
Base URL is already filled with http://localhost:8080/v1. Change the host or the port if your llama-server runs somewhere else, and keep the /v1 suffix: the app requests the model list at the base URL plus /models.
Leave API key empty. The app prints the rule under it: "This endpoint runs on your machine and needs no API key." Fill it only when you started llama-server with --api-key.
Step 5: Load the model list
In the Models card, click Reload models. The app requests http://localhost:8080/v1/models and lists what the server advertises. If nothing appears, click the + button next to it, type the id llama-server answers to into the Model ID field of the Add New Model dialog, and click Add Model.
Step 6: Chat locally
Open a chat, pick the model in the picker at the top, and send. Selecting it starts the app's own local API server if it is not already running, so there is nothing else to switch on.
Thinking runs at high effort by default. The level picker next to the lightbulb icon in the chat sends the reasoning effort to your server as low, medium, or high, and the two higher levels go out as high. Even the lowest setting leaves some reasoning on.
Requests do not go straight to port 8080. They travel from the chat through that local API server at http://127.0.0.1:1337/v1 and on to your llama-server, so coding agents like Claude Code and Cline pointed at that address can use the model too.
What you get on this path, and what you do not
On this path Atomic Chat is the client and llama-server runs the model, so part of what the app normally does for a local model is now yours to do:
- You start and stop llama-server yourself. The app shows no Start or Stop button for this model and no memory-fit estimate, and Reload models finds nothing until the server is already up.
- The app's llama.cpp settings do not reach it. Context size, GPU layers, and the micro-batch are flags you pass to llama-server: -c, -ngl, and -ub.
- Capabilities are not detected. A model you add by hand gets text completion only: tick Vision and Tools in the model's edit dialog, and image input additionally needs the vision projector on the server, which -hf downloads for you.
- This is the desktop app. The iOS and Android builds have no such provider.
How to run Inkling 2 Mini with llama.cpp
You might prefer to run the model directly through llama.cpp if you need:
- An OpenAI-compatible local endpoint with explicit flags
- Precise control over GPU offload
- A reproducible server configuration
Upstream llama.cpp has supported Inkling 2 Mini since release b12600 on launch day, November 20, 2026, so our GGUFs run on any build from b12600 up with no patches.
Step 1: Build current llama.cpp
For NVIDIA CUDA:
git clone https://github.com/ggml-org/llama.cpp cd llama.cpp cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON cmake --build build -j --target llama-cli llama-server llama-mtmd-cli
For Apple Silicon, Metal is enabled by default:
cmake -B build -DCMAKE_BUILD_TYPE=Release cmake --build build -j --target llama-cli llama-server llama-mtmd-cli
Step 2: Run the model
The following command downloads AD-Q4_K from Hugging Face, offloads all layers to the GPU, applies Thinking Machines' recommended temperature, and limits the context to 8K:
./build/bin/llama-cli \ -hf AtomicChat/Inkling-2-Mini-GGUF:AD-Q4_K \ --jinja \ --temp 1.0 \ -ngl 99 \ -c 8192
Replace AD-Q4_K with another quant name if your system has a different memory capacity. The one exception is AD-IQ2_XXS: its tag also matches AD-IQ2_XXS-Q2, so download that file and pass it with -m instead. On a card where the file does not fit, add --n-cpu-moe 8 to keep the expert weights of the first 8 layers in system RAM, and raise the number until the model loads.
Step 3: Expose a local OpenAI-compatible API
Replace llama-cli with llama-server:
./build/bin/llama-server \ -hf AtomicChat/Inkling-2-Mini-GGUF:AD-Q4_K \ --alias inkling-2-mini \ --jinja \ --temp 1.0 \ -ngl 99 \ -c 8192 \ --host 127.0.0.1 --port 8080
Test it with:
curl http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{ "model": "inkling-2-mini", "messages": [ { "role": "user", "content": "Explain what a looped transformer changes about inference." } ], "chat_template_kwargs": { "reasoning_effort": "low" } }'The chat_template_kwargs field sets the reasoning effort per request: none, minimal, low, medium, high, xhigh, or max. Bind the server to 127.0.0.1 unless other machines on your network need access.
Running vision through llama.cpp
Image input needs one extra file, the vision projector, which is shared by all quants. With -hf, llama.cpp downloads it alongside the quant. For a local file, download mmproj-Inkling-2-Mini-Q8_0.gguf from the repository once and pass it alongside whichever quant you picked:
./build/bin/llama-mtmd-cli \ -m Inkling-2-Mini-AD-Q4_K.gguf \ --mmproj mmproj-Inkling-2-Mini-Q8_0.gguf \ --image your-photo.jpg \ -ngl 99 -c 8192 \ -p "What is in this image?"
The projector is 0.35 GB.
Running it with Ollama or LM Studio
In Ollama, ollama run inkling-2-mini pulls a 20 GB NVFP4 build that runs on Ollama's MLX engine, so that route is for Macs. LM Studio lists the model in its catalog with its own Q4_K_M, Q6_K, and Q8_0 GGUF builds, in the same merged layout as ours, so our files load in LM Studio as well.
Troubleshooting
llama.cpp says tensor 'blk.24.attn_q.weight' not found
Loading stops with check_tensor_dims: tensor 'blk.24.attn_q.weight' not found. The file was built for a different pull request, #32695, which is still open and stores the loop as 48 layers, while the merged llama.cpp reads 24 layers plus the loop settings. Before downloading from another repo, open the file's metadata on Hugging Face: general.architecture inkling2_loop with block_count 24 loads on current llama.cpp, and block_count 48 does not. We converted our files on November 21 under the same names, so if you downloaded one on launch day, download it again. Unsloth keeps the files for the merged runtime in a separate Mainline_Support folder, and those need a llama.cpp build from after pull request #32704, merged on November 21.
You run out of memory at long context
The weights fit, then a long conversation crashes the model, or llama-server fails to start at a large -c value. On this model the usual cause is the compute buffer: llama.cpp reserves about 96 KB of it per token of context at the default -ub 512, three times the 32 KB of attention cache. Set an explicit context with -c instead of 0, and pass -ub 256 to halve the buffer, at some cost in prompt processing speed. If you need more room, step one quant down: every gigabyte you free adds about 7.5K of context.
Thinking mode won't turn off
The chat template has no on/off switch for thinking: it reads reasoning_effort, from none (0.0) to max (0.99), and even none does not disable reasoning completely. Pass reasoning_effort set to none or minimal in chat_template_kwargs for the shortest reasoning.
The model keeps re-checking its own work
In Thinking Machines' testing, the Mini sometimes got stuck in long reasoning loops, re-checking its work without making progress, most often while debugging code. The lab recommends limits on output tokens and agent turns. Set max_tokens on API requests, or -n in llama-cli, and cap the number of turns in your coding agent.
Frequently asked questions
The questions that come up most often about running Inkling 2 Mini on your own hardware.
How much VRAM does Inkling 2 Mini need?
About 16.5 GB for the 4-bit file, and our builds run from 6.88 to 27.74 GB. A 24 GB GPU runs AD-Q5_K with around 24K of context, and 16 GB cards run AD-IQ3_S. On Apple Silicon, macOS gives the GPU about two thirds of unified memory up to 32 GB, so a 32 GB Mac runs AD-Q4_K with room for context.
Can I run Inkling 2 Mini on 16GB of VRAM?
Yes, with tradeoffs. AD-IQ3_S (13.34 GB) fits in 16 GB with around 8K of context, at 94.2% top-1 agreement against the original. A 4-bit build needs some expert layers in system RAM, which costs less speed on a MoE than on a dense model. On 12 GB, AD-IQ2_M runs with around 12K of context.
Can I run Inkling 2 Mini on 8GB of VRAM?
Yes, with tradeoffs. AD-IQ1_S is 6.88 GB, so on an 8 GB card the weights fit and the buffers do not; keep a few expert layers on the CPU with --n-cpu-moe. That build picks the same token as the original 79.5% of the time.
Can you run Inkling 2 Mini on a Mac?
Yes. A 16 GB Mac runs AD-IQ2_M, a 24 GB Mac runs AD-IQ3_S, and a 32 GB Mac runs AD-Q4_K with around 32K of context, all natively through Metal; Q8_0 runs from 48 GB up. Ollama's 20 GB NVFP4 build is the other route on a Mac. For models sized for a 16 GB Mac from the start, see our best local LLM for a 16GB Mac roundup.
Has Inkling 2 Mini been released yet?
Yes. The Inkling 2 Mini release date was November 20, 2026: Thinking Machines published the weights on the Inkling-2-Mini model page on Hugging Face. Our GGUF builds are in the AtomicChat/Inkling-2-Mini-GGUF repository.
What is the difference between Inkling 2 Mini and Inkling 2 Small?
The difference is size. Inkling 2 Small is a 594B-parameter MoE model with 17B active, and its main checkpoint needs a GPU cluster with at least 500 GB of VRAM. Inkling 2 Mini has 26.4B parameters with 3.4B active and fits on a single 24 GB GPU. Both accept text and image input with a 1M-token context window.
How does Inkling 2 Mini compare to Qwen 3.8 27B?
They're the closest pair in Thinking Machines' table, and both fit a 24 GB card at 4-bit. The Mini leads on five of the seven launch benchmarks above, including SWE-bench Verified (79.2 vs 77.8), SWE-bench Pro (Hard) (77.9 vs 73.8), and LongBench v2 (66.9 vs 52.3); Qwen 3.8 27B leads on Terminal Bench 2.1 (73.0 vs 71.6) and MMMU Pro (76.3 vs 74.9). Artificial Analysis's Intelligence Index puts both at 20.1. For the dense model, see our guide to running Qwen 3.8 27B locally.
What is the context length of Inkling 2 Mini?
1,048,576 tokens, or 1M. The attention cache alone is about 1.3 GB at 32K, but llama.cpp adds a compute buffer that brings the total to about 4.5 GB on top of the file. Start at 8K and raise it as the work demands.
Does Inkling 2 Mini support vision locally?
Yes. Inkling 2 Mini is a native multimodal model that accepts image input alongside text. In Atomic Chat connected to your llama-server, you attach an image to the chat and ask about it, with nothing leaving your machine; tick Vision for the model in its edit dialog first. Through llama.cpp, image input needs the vision projector, mmproj-Inkling-2-Mini-Q8_0.gguf at 0.35 GB, which -hf downloads for you.
Does Inkling 2 Mini work with Ollama or LM Studio?
Yes, in Ollama on Apple Silicon: ollama run inkling-2-mini pulls a 20 GB NVFP4 build for its MLX engine. LM Studio lists the model in its catalog with its own GGUF builds, and our files load there as well.
Is there an uncensored or abliterated Inkling 2 Mini?
Yes. We publish AtomicChat/Inkling-2-Mini-Uncensored-GGUF: a LoRA adapter for any of our files, plus AD-Q4_K, AD-IQ3_S, and AD-IQ2_M with the edit built in. Our guide to abliterated models covers how that edit is made and what it costs.
Is Inkling 2 Mini free for commercial use?
Yes. Inkling 2 Mini ships under the Apache 2.0 license: commercial use, modification, and redistribution are all allowed. Thinking Machines also publishes an acceptable use policy for the model.
Bottom line
If your machine has a 24 GB GPU or 32 GB of unified memory, Inkling 2 Mini is the Inkling 2 model to run locally: serve AD-Q5_K on the GPU or AD-Q4_K on the Mac with llama-server from b12600 up, connect Atomic Chat to it, start at 8K context, and raise it as the work demands. The same server is a local OpenAI-compatible API for any other tool.
Key takeaways:
- Inkling 2 Mini is a 26.4B mixture-of-experts multimodal model with 3.4B active parameters.
- Thinking Machines released it on November 20, 2026 under Apache 2.0.
- Our GGUF builds run from 6.88 to 27.74 GB. For a 24 GB card, take AD-Q5_K.
- Long context costs memory on top of the file: about 1.3 GB at 8K and 4.5 GB at 32K.
- Between neighbouring quants, the larger one is worth the extra gigabytes; error grows fastest below 12 GB.

