Best Local LLMs for 16GB RAM or VRAM in 2026

We made a roundup of the best AI models to run locally if you have 16GB of RAM or VRAM.

Selection criteria

As a rule of thumb, if you have 16GB of VRAM you can run 20B–35B models, and if you have 16GB of total RAM or unified memory you can run 8B–12B models. That leaves us with the following models:

Best Local LLMs for 16GB VRAM

Qwen 3.8 27B

Qwen 3.8 27B is a 27B-parameter dense multimodal model from the Qwen team. It has 64 language-model layers, a native context length of 262,144 tokens, optional extension to 1M tokens, and support for text, images, and video. Thinking is enabled by default and can be disabled at inference time.

For a 16GB GPU, we recommend our 13.8GB AD-IQ3_S build from AtomicChat/Qwen3.8-27B-GGUF.

Qwen 3.8 has a relatively large KV cache. In llama.cpp, it uses about 256KB per token, or roughly 2GB at 8K context and 8GB at 32K. With AD-IQ3_S, a 16GB GPU therefore has room for about 8K context with full GPU offload.

AD-IQ3_S uses an Atomic Dynamic layout — our proprietary layout in which different tensors are quantized at different precisions according to their measured sensitivity.

To build that layout, we first generated an importance matrix from 4,967,044 tokens across 3,004 documents. The calibration set includes code, reasoning, tool use, multilingual text, long-context samples, structured data, graphics, and vocabulary coverage.

We then tested which parts of the model were most sensitive to lower precision. The first and last layers were kept at higher precision, as were several attention-related tensors. Layers 52–62 showed the strongest activation peaks. Increasing precision for the attention gate and state-output tensors added about 0.16GB and reduced the remaining divergence by 11% in our tests.

The finished quants were evaluated against the original BF16 model on held-out data that was not used for calibration. We report both top-1 agreement and per-token KL divergence.

We publish the calibration data, quantization rules, evaluation procedure, and raw measurements with the release. The evaluation uses 87 held-out 4K-context chunks and compares every quant directly with BF16. For reference, Q8_0 reaches 98.92% top-1 agreement with a KL divergence of 0.00064, so it is also measured rather than treated as a lossless baseline.

Qwen 3.8 27B benchmarks

Qwen 3.8 27B pros

• Strong general reasoning, coding, research, and agentic capabilities
• Native image and video understanding
• Thinking can be enabled, disabled, or adjusted by reasoning effort
• Atomic Dynamic quant has measured fidelity data
• Fits fully in 16GB VRAM at AD-IQ3_S

Qwen 3.8 27B cons

• The recommended 3-bit build picks a different top token from the original about 7.6% of the time
• Practical context is around 8K on a 16GB card
• CPU offload is costly because the model is dense

When to choose Qwen 3.8 27B: Choose it if you have a 16GB GPU and want the strongest balanced model in this guide for general work. Start with AD-IQ3_S and an 8K context. Use the 13.0GB AD-IQ3_S-IQ3_XXS build if your runtime still runs out of VRAM.

Ornith 1.5 35B-A3B

Ornith 1.5 35B-A3B is a 35B mixture-of-experts reasoning model developed for coding and agentic workflows. It activates about 3B parameters per token and supports a 262K native context window.

The Atomic Chat GGUF builds include two files near the 16GB boundary. AD-IQ3_XXS-IQ2_S is 13.7GB and leaves limited but usable room for context. AD-IQ3_S-IQ3_XXS is 15.5GB, which can fit in raw capacity but is too tight once the KV cache and buffers are allocated.

If the lower-bit file loses too much quality for your workload, Ornith's MoE architecture gives you another option: run the 22.1GB AD-Q5_K-Q4_K build with the experts in system RAM. In llama.cpp, --cpu-moe places the routed experts on the CPU while attention, routing, and shared layers remain on the GPU. This is slower than full GPU residence, but it is more practical than offloading ordinary dense layers.

Ornith 1.5 35B-A3B benchmarks

Ornith 1.5 35B-A3B pros

• Built specifically for coding, terminal use, and agentic tasks
• Only about 3B parameters are active for each token
• Thinking mode can be switched off
• MoE expert offload provides a higher-quality alternative to extreme quantization
• MIT license

Ornith 1.5 35B-A3B cons

• Fully GPU-resident 16GB use requires a low-bit quant
• The 15.5GB build leaves too little room for ordinary context use
• Expert offload requires enough system RAM for the full file and reduces speed
• The model's reasoning output increases latency and token use

When to choose Ornith 1.5 35B-A3B: Choose it if agentic coding is the main workload. Use AD-IQ3_XXS-IQ2_S for full 16GB GPU residence, or use the higher-quality AD-Q5_K-Q4_K build with expert offload if you have at least 24GB of system RAM in addition to the GPU.

gpt-oss-20b

gpt-oss-20b is OpenAI's smaller open-weight reasoning model. It has 20.9B total parameters, but only 3.6B parameters are active for each token. This makes it much cheaper to run than a dense 20B model while still keeping the capacity of a larger model.

The model uses a Mixture-of-Experts (MoE) architecture with 32 experts, four of which are selected for each token. It has 24 layers and alternates dense attention with locally banded sparse attention. Grouped multi-query attention is used to reduce KV-cache cost, and the native context window is 128K tokens.

The MoE weights are released directly in MXFP4. The weights occupy about 12.8GiB, allowing the model to fit within 16GB of memory.

gpt-oss-20b is a reasoning model and supports low, medium, and high reasoning effort.

It was also trained for agentic workloads. The model supports function calling, structured outputs, web browsing, and Python tools when these are provided by the inference environment.

OpenAI trained the model primarily on English text, with an emphasis on STEM, coding, and general knowledge. It is text-only and does not accept image or audio input.

gpt-oss-20b benchmarks

gpt-oss-20b pros

• Officially designed for 16GB devices
• Strong reasoning, coding, function calling, and structured output
• Configurable reasoning effort
• Native MXFP4 weights avoid relying on an unofficial extreme quant
• Broad runtime support, including llama.cpp, Ollama, LM Studio, and Metal

gpt-oss-20b cons

• Text-only
• Very little memory margin on a 16GB RAM system
• Requires the Harmony prompt format to behave correctly
• Long reasoning traces increase latency
• The full 128K context is not practical at the 16GB minimum

When to choose gpt-oss-20b: Choose it for local reasoning, coding, function calling, and structured-output workflows. It is a particularly clean option when you want an official low-memory checkpoint instead of a community 3-bit conversion.

Gemma 4 26B-A4B

Gemma 4 26B-A4B is a 25.2B-parameter Mixture-of-Experts model from Google DeepMind. Only 3.8B parameters are active per token: the model has 128 routed experts and selects eight of them for each token. This gives it the capacity of a much larger model without the compute cost of activating all 25B parameters on every forward pass.

The model has 30 layers and uses a hybrid attention layout with 1024-token sliding-window attention mixed with global attention. It supports a native context length of 256K tokens. The base model is multimodal and accepts text and images.

For a 16GB GPU, we recommend our 13.9GB IQ4_XS build.

These GGUFs were quantized by us directly from Google's original weights. We first converted the model to F16 GGUF, then built an importance matrix over our calibration corpus and used it when generating the full quantization ladder. The matrix is published with the repository as imatrix-coding.gguf.

The importance matrix records which weights have the largest effect on model activations during calibration. llama.cpp can use that information when producing low-bit IQ and K quants, giving more of the available precision to the parts of the model that matter most.

The resulting ladder ranges from 10.6GB Q2_K to 26.9GB Q8_0. Around the 16GB limit, the useful choices are IQ3_M at 12.4GB, Q3_K_L at 13.8GB, and IQ4_XS at 13.9GB. We recommend IQ4_XS to run on 16GB hardware because this build has very little quality loss, similar to that of 4-bit builds, at almost the same file size as Q3_K_L.

The original Gemma 4 model supports image input, but this Atomic Chat repository currently contains text-only GGUFs and does not include the vision projector. Use the original or another multimodal build if image input is required. The Gemma 4 chat template should also be enabled when running these files in llama.cpp.

Gemma 4 26B-A4B benchmarks

Gemma 4 26B-A4B pros

• MoE compute cost is lower than its total parameter count suggests
• Importance-matrix quants are built directly from the original weights
• IQ4_XS leaves about 2GB for runtime and context
• 256K architectural context window
• Apache 2.0 license

Gemma 4 26B-A4B cons

• Q4_K_M does not fit in 16GB VRAM
• The recommended build is a lower-bit IQ4 quant
• Atomic Chat's current GGUF is text-only
• The full 256K context is not realistic on 16GB

When to choose Gemma 4 26B-A4B: Choose it if you want a recent MoE generalist.

Best Local LLMs for 16GB RAM or Unified Memory

A computer with 16GB of RAM or unified memory has a smaller model budget than a 16GB graphics card, so we've selected models whose files weigh 7–9GB.

Qwen 3.5 9B

Qwen 3.5 9B is a 9.7B-parameter dense model from the Qwen team. It has 32 layers and uses a hybrid architecture built around Gated DeltaNet and full attention: every group of four layers contains three DeltaNet layers followed by one attention layer. The native context length is 262,144 tokens.

For a 16GB unified-memory system, we recommend our 7.4GB Q6_K build. These GGUFs were quantized by us from the original Qwen weights using an importance matrix built over our calibration corpus. The matrix is used during quantization to preserve more precision in weights that have a larger effect on model activations.

We also provide a 6.4GB UD-Q4_K_XL build. It keeps the token embeddings and output tensors at Q8_0 while quantizing the rest of the model more aggressively. It sits close to Q5_K_M in size and is useful when memory is tighter than the 7.4GB required by Q6_K.

Note: This Atomic Chat repository contains the text-only model and does not ship a vision projector, so these GGUFs are text-only builds.

Qwen 3.5 9B benchmarks

Qwen 3.5 9B pros

• Strong general reasoning and coding for its size
• Q6_K leaves useful system-memory headroom
• Thinking and non-thinking modes
• Hybrid attention reduces KV-cache growth compared with a conventional full-attention model
• Broad multilingual support

Qwen 3.5 9B cons

• Atomic Chat's current GGUF does not include image support
• Q8_0 may push a busy 16GB computer into swap
• The architectural 262K context is far above what a 16GB system can use
• Lower ceiling than the 20B–35B models that fit dedicated 16GB GPUs

When to choose Qwen 3.5 9B: Start here if you have 16GB of total RAM or unified memory and want one model for writing, analysis, coding, and multilingual chat. Use Q6_K; drop to Q5_K_M if your system swaps under longer prompts.

Ornith 1.5 9B

Ornith 1.5 9B is the smallest model in the Ornith 1.5 family. It has an 8.95B language model and a 0.46B vision encoder, with a 262,144-token native context window. Ornith targets reasoning and agentic workloads, including coding, terminal tasks, and tool use.

Its 32 layers mix two attention mechanisms: 24 use linear attention and eight use full attention. Only the full-attention layers maintain a KV cache; the other layers use a fixed recurrent state. As a result, long context is relatively cheap for a model of this size. In llama.cpp, the cache uses roughly 32KB per token, or about 2GB at 64K context.

For a 16GB unified-memory system, we recommend our 8.55GB AD-Q8_0-Q6_K build. It reaches 97.46% top-1 agreement with the original BF16 model while saving almost 1GB compared with the 9.53GB Q8_0 build. That extra memory is more useful for context, the runtime, and other applications on a machine where all of them share the same 16GB pool.

AD-Q8_0-Q6_K comes from our Atomic Dynamic quantization work on Ornith. We tested eight tensor layouts at the same size class and measured each one against the BF16 reference. The experiments showed that the attention gate and state-output tensors matter disproportionately: together they account for about 9% of the weights, but increasing their precision produced the largest improvement among the layouts we tested.

We generated an importance matrix from the BF16 weights using 4.96 million calibration tokens across 9,686 chunks. The corpus covers agentic tool traces, code, reasoning, multilingual text, long context, structured data, graphics, and a tokenizer-specific vocabulary sweep. It covers 99.5% of Ornith's vocabulary.

As a result, our 5.93GB AD-Q5_K-Q4_K is both smaller and more accurate than the 6.47GB stock Q5_K_M. At roughly the same size as stock Q4_K_M, AD-Q4_K-IQ4_XS reduces KL divergence by about 31%. At higher precision the gains become smaller, which is why the ladder also includes stock Q6_K and Q8_0 rather than forcing an AD layout onto every size class.

The repository also includes a 0.92GB vision projector, so the GGUF can retain the model's image support. For text-only use, the projector can be omitted to save memory.

Ornith 1.5 9B benchmarks

Ornith 1.5 9B pros

• Strong agentic coding performance for a 9B model
• Recent August 2026 release
• Atomic Dynamic build is close to the original while fitting shared memory
• Hybrid attention keeps KV-cache growth relatively low
• Image input and MIT license

Ornith 1.5 9B cons

• More specialized toward coding and agentic work than a general assistant
• Thinking is enabled by default and increases response latency
• Vision requires the separate projector file
• Recent runtimes are required for correct model and chat-template support

When to choose Ornith 1.5 9B: Choose it for a 16GB coding laptop or Mac. Use AD-Q8_0-Q6_K when the machine is otherwise lightly loaded, or AD-Q5_K-Q4_K when you need more memory for an IDE and browser.

Gemma 4 12B

Gemma 4 12B is Google's 11.95B-parameter dense model with 48 layers and a mix of 1024-token sliding-window and global attention. It supports up to 256K context and accepts text, images, and audio in the base model.

Our 7.4GB Q4_K_M build leaves enough of the shared memory pool for the OS, KV cache, and runtime without dropping into the lower-bit quants. Q5_K_M at 8.5GB is a good alternative when the system has more memory to spare.

Gemma 4 12B benchmarks

Gemma 4 12B pros

• Larger dense model than the other shared-memory recommendations
• Text, image, and audio support
• Atomic Chat ships the required vision projector
• Importance-matrix quant built from the original weights
• Q4_K_M leaves useful system-memory headroom

Gemma 4 12B cons

• Multimodal inputs consume additional memory
• Q6_K at 9.8GB can be tight once the OS and context are included
• Requires the correct Jinja chat template
• Full 256K context is not practical with 16GB of total memory

When to choose Gemma 4 12B: Choose it if screenshots, images, documents, or audio are central to your workflow. Start with Q4_K_M and a modest context; move to Q5_K_M only after checking live memory use.

LFM2.5 8B-A1B

LFM2.5 8B-A1B is Liquid AI's 8.3B-parameter MoE model built for on-device assistants. It activates about 1.5B parameters per token and combines 18 double-gated convolution layers with six GQA layers. Liquid AI trained LFM2.5 on 38 trillion tokens and expanded the context window from 32K in LFM2 to 128K.

The model uses 32 routed experts with four active per token. Its low active parameter count keeps inference compute relatively small, while the hybrid convolution/attention architecture targets fast local generation. Liquid AI specifically tuned this release for instruction following, reasoning, and multi-step tool use.

LFM2.5 also expands the vocabulary from 65K to 128K tokens, improving tokenization for several non-Latin languages. Liquid AI reports particularly large reductions in token count for languages such as Hindi, Thai, Vietnamese, Indonesian, and Arabic.

The model is text-only and supports function calling through its ChatML-like template. This makes it especially relevant for local assistants that alternate between generation and tool execution rather than spending all of their time on long free-form responses.

LFM2.5 8B-A1B benchmarks

LFM2.5 8B-A1B pros

• Low active parameter count for fast generation
• Designed for on-device assistants and chained tool calls
• Q6_K leaves substantial memory headroom
• Supports llama.cpp, MLX, vLLM, and SGLang
• 128K architectural context window

LFM2.5 8B-A1B cons

• Text-only
• Lower hard-task ceiling than Qwen 3.8 27B or Ornith 1.5 35B
• Q8_0 provides less headroom for long prompts on a 16GB system
• License is less permissive than Apache 2.0 or MIT

When to choose LFM2.5 8B-A1B: Choose it for a responsive local assistant, tool calling, or CPU-heavy use. Use Q6_K for the best balance on a 16GB computer.

FAQ

What is the best local LLM for 16GB VRAM in 2026?

Qwen 3.8 27B with Atomic Chat's 13.8GB AD-IQ3_S quant is the best balanced choice. It fits fully in 16GB VRAM, supports text, images, and video in the base model, and covers reasoning, coding, research, and agentic work. Ornith 1.5 35B-A3B is the stronger specialist when agentic coding is the priority.

What is the best local LLM for 16GB RAM?

Qwen 3.5 9B at Q6_K is the best general starting point. The 7.4GB file leaves enough memory for the operating system, runtime, and a useful context window. Choose Ornith 1.5 9B for coding or Gemma 4 12B for multimodal input.

Can I run a 30B model on 16GB VRAM?

Yes, with tradeoffs. A dense 27B model needs a 3-bit build such as Qwen 3.8 27B AD-IQ3_S. A 35B MoE can also fit at a low quant because only a small subset of its experts is active for compute, although all weights still occupy memory. Mainstream Q4 files in this range are generally too large to leave context headroom.

Can I run a 20B model on 16GB RAM?

gpt-oss-20b is specifically designed to run within 16GB and its native checkpoint is 12.8GiB. However, a computer with only 16GB of total system RAM has little space left for the OS, runtime, and KV cache. It is possible with an efficient backend and modest context, but a 7–9GB model file will usually provide a smoother everyday experience.

What context length can I use with 16GB VRAM?

It depends mainly on model size, architecture, and KV-cache precision. Qwen 3.8 27B AD-IQ3_S leaves room for roughly 8K context on a 16GB GPU. Smaller hybrid models can go much further because their weights leave more free memory and only some layers maintain a conventional KV cache. Quantized KV storage can extend the usable context substantially.

What quantization should I use for 16GB VRAM?

For a dense 27B model, use an IQ3 or measured mixed 3-bit quant around 13–14GB. For a 20B MoE model, native MXFP4 or an ordinary Q4 build may fit. For 9B–12B models, Q6 or Q8 usually fits comfortably and preserves more of the original model.

How much VRAM does a 27B model use with llama.cpp?

The weights alone range from about 13.8GB at Atomic Chat's AD-IQ3_S to 17.1GB at AD-Q4_K for Qwen 3.8 27B. llama.cpp also allocates the KV cache, compute buffers, and other runtime memory. A file that fits on disk or even loads into VRAM may still fail when the context grows, so judge the setup by live memory use rather than GGUF size alone.

Bottom line

• If you've got a graphics card with 16GB of VRAM, Qwen 3.8 27B at AD-IQ3_S offers a good balance of capability and memory use.
• If you've got 16GB of RAM or unified memory, aim for a model file around 7–9GB, such as Qwen 3.5 9B Q6_K, Ornith 1.5 9B, or Gemma 4 12B.
• You'll usually get better performance from a model that fits entirely in fast memory than from a larger model that depends heavily on CPU offload or swap.
• Don't treat the GGUF file size as the total memory requirement. You also need memory for the KV cache, compute buffers, the runtime, and the operating system.
• Atomic Dynamic quants reduce the memory required by Qwen 3.8 27B and Ornith 1.5 35B by using different precision levels for different tensor groups. Individual builds also include measured fidelity data.