How to Run Ornith 1.5 35B Locally: GGUF, Hardware and Benchmarks

In this guide, you'll learn:

• What Ornith 1.5 35B is and how it differs from the 9B and the 397B
• What hardware you need to run it
• How to run Ornith 1.5 35B locally with Atomic Chat or llama.cpp

If your machine is smaller than that, see our guide to running Ornith 1.5 9B locally. This article is only about the 35B.

What is Ornith 1.5 35B?

Ornith 1.5 35B is an open-weight mixture-of-experts language model developed by DeepReinforce, the mid-size member of the new Ornith 1.5 family. The family is trained for agentic coding and ships in three sizes, a 397B MoE flagship, this 35B MoE, and a 9B dense, all under the MIT license. The 35B weights went public on Hugging Face on August 19, 2026.

Ornith 1.5 35B main specs:

A3B in the name means 3B parameters are activated per token out of 35B total. Speed follows the 3B, memory follows the 35B: every expert has to be resident, because the router picks a different eight of the 256 on every token. The way around that number is to keep the experts in system RAM instead of VRAM, which is what makes a 12 GB card viable for this model. We'll put numbers on this in the hardware section.

The 35B is also multimodal, meaning it can process images alongside text. And it uses the hybrid attention design of Qwen 3.5, the architecture it builds on: 30 of the 40 layers use linear attention, a form of attention whose memory use stays the same no matter how long the conversation gets. Only the other 10 keep a regular KV cache, the part that normally grows with every token. In practice, that means long chats and big documents cost far less memory here than on a typical 35B model.

Ornith is DeepReinforce's self-improvement project: the model proposes its own training tasks and learns from its own attempts. The 1.5 generation extends that loop beyond coding into reasoning and general agentic work. The 35B is a reasoning model: it opens each answer with a thinking block before the final response, and you can switch thinking off entirely.

The 35B also carries the Multi-Token Prediction head the 9B lacks: a 1.9B block that drafts several tokens per forward pass, the same mechanism behind speculative decoding. We publish the head as a separate draft file; the setup is in the llama.cpp section.

The Ornith 1.5 release includes three different models, and they're easy to confuse:

• Ornith 1.5 397B is the MoE flagship that DeepReinforce benchmarks against Claude Opus 4.8. At 397B parameters it takes a multi-GPU datacenter cluster to serve, the same class of hardware as Kimi K3.
• Ornith 1.5 9B is the dense vision-language model at the bottom of the lineup: it fits an 8 GB GPU or a 16 GB MacBook, and it has its own Ornith 1.5 9B guide.
• Ornith 1.5 35B is the MoE this guide is about, and the strongest of the three that runs on an everyday machine: a desktop with a 24 GB GPU, a 32 GB Mac, or a 12 GB card with the experts in system RAM.

Ornith 1.5 35B benchmarks

DeepReinforce's launch numbers, from the model card, compare the 35B with Ornith 1.0 35B, Qwen3.6 35B A3B, Gemma 4 31B, and Qwen3.5-397B:

The 35B comes out ahead of Qwen3.5-397B, a model eleven times its size, on five of the six benchmarks. A 397B-level model now runs on a single 24 GB card.

Here's the same class of local models, each scored by its own vendor:

All scores in this second table come from each vendor's own model card or release blog, and every lab runs its own eval setup, so small gaps between columns don't mean much. A hyphen means the vendor didn't publish that number.

The direct rival here is Qwen3.6 35B A3B: a 35B MoE with 3B active parameters like this one, and the model DeepReinforce's own card compares against first. By the launch numbers the 35B beats it on every coding and agentic benchmark the card reports: 67.8 against 52.5 on Terminal-Bench 2.1, and 79 against 73.4 on SWE-bench Verified.

The more useful reference for the 35B is its predecessor. Ornith 1.0 35B shipped in June 2026, and with a 75.6 on SWE-bench Verified it has been one of the strongest MoE models for local coding in this size class. The 1.5 comes out ahead on every row of the launch table, and the two share the same architecture and dimensions, so the hardware math transfers one to one: if you run Ornith 1.0 today, every 1.5 build fits the same machine.

Ornith 1.5 35B GGUF: we quantized it ourselves

We built the AtomicChat Ornith 1.5 35B GGUF repository from DeepReinforce's original BF16 weights, then measured every build against the unquantized reference. The repository holds thirteen builds from 13.7 to 36.9 GB, plus the vision projector, a separate download shared by all of them. The Multi-Token Prediction head is not inside these files: a plain forward pass never runs it, so we ship it as a separate draft file instead. The AD prefix stands for Atomic Dynamic, and a double name like AD-Q5_K-Q4_K says what the two largest tensor groups got; when both match, the name collapses to one.

How to read the table: KL divergence measures the distance between the quant's output distribution and the original weights. Lower values are better, and zero means identical. Top-1 match is the percentage of positions where the quant selects the same next token as the reference. For a single summary measure, use top-1 match. In the next section we'll explain which file fits your hardware.

AD-Q5_K-Q4_K is smaller than stock Q5_K_M and more accurate at the same time: 22.1 GB against 24.7 GB, and 0.0251 against 0.0269 on KL divergence. One step up, AD-Q6_K-Q5_K is 2.3 GB smaller than stock Q6_K and again more accurate.

We measured every number in the table ourselves. The reference is the original BF16 weights, the calibration corpus is public, and the raw logs are in the metrics repo, so you can check any of it yourself. The full recipe is on the model card.

Three publishers ship a file called Q4_K_M for this model, and they are not the same file:

A quant name only says which recipe the publisher requested. Each publisher makes its own layout choices under the same label, which is why one name comes out at three different sizes. When you pick a build, go by file size and measured quality.

Ornith 1.5 35B hardware requirements

For Ornith 1.5 35B, the system requirement to check is memory. The table below shows the VRAM you need to run the whole file on the GPU; the section after it covers running the experts from system RAM:

On Apple Silicon, count about 75 percent of unified memory as available: a 32 GB Mac reads the 24 GB row.

Note: if you are choosing between two neighbouring files, take the larger one. It costs a gigabyte or two more and picks the wrong token noticeably less often, especially below 16 GB, where quality falls fastest.

Running it on a smaller GPU

With a smaller card, keep the experts in system RAM instead of dropping to a smaller file. The experts are 93 percent of the weights, but only 3B of them run per token, so the arithmetic stays cheap while the bandwidth cost moves to RAM; the same trade is what makes DeepSeek V4 Flash practical on desktop hardware. In llama.cpp, --cpu-moe puts every expert on the CPU, and --n-cpu-moe N offloads the experts of the first N layers and keeps the rest on the GPU. That lets a 12 GB card serve the 22.1 GB AD-Q5_K-Q4_K at usable speed, with attention, the router, and the shared expert still in VRAM. The exact command is in the llama.cpp section; count the full file size against RAM plus VRAM when you plan the download.

What hardware can run Ornith 1.5 35B?

• 12 GB GPUs (RTX 3060, RTX 4070): can serve the 22.1 GB AD-Q5_K-Q4_K with the experts in system RAM; the command is in the llama.cpp section.
• 16 GB GPUs (RTX 4060 Ti 16 GB, RTX 5060 Ti): can run AD-IQ3_S-IQ3_XXS entirely in VRAM, or a 4-bit build with the experts offloaded.
• 24 GB GPUs (RTX 3090, RTX 4090): can run AD-Q4_K-IQ4_XS entirely in VRAM with room for context, or AD-Q5_K-Q4_K when the conversation stays short.
• RTX 5090 (32 GB VRAM): can run AD-Q6_K entirely in VRAM, and at that quant the model behaves close to the original.
• 48 GB and up (RTX 6000-class, or two 24 GB cards): can run Q8_0 with the full file in VRAM.
• MacBook Pro (32 GB unified memory): can run AD-Q5_K-Q4_K through Metal. macOS caps GPU-addressable memory at roughly 75 percent of unified memory by default, so a 32 GB Mac gives the model about 24 GB to work with.
• Mac Studio, MacBook Pro M4/M5 Max (48-64 GB): can run AD-Q6_K from 48 GB of unified memory, and Q8_0 from 64 GB. On Apple Silicon the GGUF builds run natively through Metal; see our GGUF vs MLX guide if you're choosing between formats. DeepReinforce also ships official MLX builds at 4, 6, and 8 bits.

How to run Ornith 1.5 35B locally with Atomic Chat

Atomic Chat is a free, open-source local AI app we built. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.

Here's how to run Ornith 1.5 35B with Atomic Chat:

Step 1: Install Atomic Chat

Download Atomic Chat from atomic.chat and install the build for your platform:

• macOS: a universal .dmg (Intel and Apple Silicon), macOS 13.6 or later
• Windows: an .exe installer for x64
• Linux: a self-contained .AppImage for x86_64, which needs no root
• iOS: through the App Store
• Android: through Google Play

On Linux, mark the AppImage executable with chmod +x and run it directly. If the app asks about FUSE on first launch, install it with sudo apt install fuse libfuse2 on Debian or Ubuntu, or sudo dnf install fuse fuse-libs on Fedora.

Step 2: Find our Ornith 1.5 35B GGUF

Open the Models tab and search for:

AtomicChat/Ornith-1.5-35B-A3B-GGUF

The catalog is backed by Hugging Face, so you'll also see builds from other publishers as they appear. Choose the result published by AtomicChat, then expand the Download Options picker to list the available quantizations.

Note: modified Ornith 1.5 repositories, uncensored and abliterated forks among them, appeared on Hugging Face within a day of the release. If you're downloading from anywhere other than the app catalog, see the Troubleshooting section for how to check what you're pulling.

Step 3: Pick a quant for your memory

Use the hardware table above. For the most common configurations:

• 24 GB GPU: download AD-Q4_K-IQ4_XS.
• 32 GB Mac: download AD-Q5_K-Q4_K.
• 32 GB GPU: download AD-Q6_K.

The picker lists each build under its short tag, without the AD prefix, and a double name like AD-Q5_K-Q4_K shows up under the tag of its smaller half. The sizes shown in the picker differ a little from the file sizes in our tables; when two rows share a name, go by size. Select the largest quant that fits with context headroom. If you're not sure what the quant names mean, see our guide to what GGUF is and how quantization works.

Step 4: Set the context, thinking, and sampling

The model declares a 262K context window, but configuring the maximum allocates the KV cache up front. Start with 8,192 tokens for chat or 32,768 for code and document work, and increase it only when the workload requires it.

The context control sits next to the chat input: the percentage shows how much of the window the conversation has used, and clicking it opens the Context Size slider. The maximum comes from the model, 256K for this build (the same 262,144 tokens the specs table lists), and 0 loads the value from the GGUF.

Two more settings live on the model itself: open Settings → Model Providers → Llama.cpp, find the Ornith 1.5 build in the model list, and click the gear icon on its row.

• Auto Increase Context Size: on by default. It grows the context when a conversation runs out of room, which can push a build that only just fit past your memory limit. Turn it off when you're working close to that limit.
• GPU Layers: -1 offloads every layer, which is what you want when the file fits in VRAM.

Changing Context Size or GPU Layers restarts the model, so set them before a long conversation.

Thinking is on by default. For models that support reasoning, Atomic Chat shows a lightbulb icon next to the chat: click it to switch thinking on and off.

Sampling is a separate panel: click the sliders icon next to the model name at the top of the chat. DeepReinforce recommends temperature 0.6, top_p 0.95, and top_k 20; the engine doesn't pick up the model's stock values, so set them yourself. The published benchmark runs raise the temperature to 1.0. These settings are saved on the assistant, your chat profile, not on the model, so they stay the same when you switch models.

Step 5: Chat locally

After the download completes, Atomic Chat loads the model and opens it in the built-in chat.

Since the 35B is a native vision-language model, you can attach an image to the chat and ask about it. The image, the weights, and your prompts stay on your machine.

Atomic Chat also exposes an OpenAI-compatible API server at http://localhost:1337/v1. Any tool that speaks the OpenAI API, including coding agents like Claude Code and Cline, can use the local model as a drop-in replacement for a cloud one.

How to run Ornith 1.5 35B with llama.cpp

You might prefer to run the model directly through llama.cpp if you need:

• An OpenAI-compatible local endpoint with explicit flags
• Precise control over GPU and expert offload
• A reproducible server configuration

The 35B uses the same MoE architecture family as Qwen 3.5, which upstream llama.cpp has supported since February 2026, so the release-day GGUFs run on a current build with no patches.

Step 1: Build current llama.cpp

For NVIDIA CUDA:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-cli llama-server

For Apple Silicon, Metal is enabled by default:

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server

Step 2: Run the model

The following command downloads AD-Q4_K-IQ4_XS from Hugging Face, offloads all layers to the GPU, applies DeepReinforce's recommended sampling, and limits the context to 8K:

./build/bin/llama-cli \
  -hf AtomicChat/Ornith-1.5-35B-A3B-GGUF:AD-Q4_K-IQ4_XS \
  --jinja -fa on \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192

Replace AD-Q4_K-IQ4_XS with another quant name if your system has a different memory capacity. Keep the --jinja flag on every run: it applies the model's own chat template. To reproduce DeepReinforce's benchmark setup, raise the temperature to 1.0.

Step 3: Expose a local OpenAI-compatible API

Replace llama-cli with llama-server:

./build/bin/llama-server \
  -hf AtomicChat/Ornith-1.5-35B-A3B-GGUF:AD-Q4_K-IQ4_XS \
  --alias ornith-1.5-35b \
  --jinja -fa on \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

Test it with:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "ornith-1.5-35b", "messages": [ { "role": "user", "content": "Summarize what expert routing changes about memory use." } ] }'

Bind the server to 127.0.0.1 unless other machines on your network need access.

Running it with the experts in system RAM

If the file does not fit your card, add --cpu-moe instead of taking a smaller quant:

./build/bin/llama-server \
  -hf AtomicChat/Ornith-1.5-35B-A3B-GGUF:AD-Q5_K-Q4_K \
  --cpu-moe \
  --jinja -fa on \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -ngl 99 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

--cpu-moe puts every expert on the CPU. The finer control is --n-cpu-moe N, which offloads the experts of the first N layers and keeps the rest on the GPU: raise N until the model stops running out of memory, and keep -ngl 99 either way, since attention, the router, and the shared expert still belong in VRAM.

Running vision through llama.cpp

Image input needs one extra file, the vision projector, which is shared by all quants. Download it once and pass it alongside whichever quant you picked:

llama-mtmd-cli \
  -m Ornith-1.5-35B-A3B-AD-Q4_K-IQ4_XS.gguf \
  --mmproj mmproj-Ornith-1.5-35B-BF16.gguf \
  --image your-photo.jpg --image-min-tokens 1024 \
  -ngl 99 -c 8192 \
  -p "What is in this image?"

Keep --image-min-tokens 1024: without it, dense images such as charts and screenshots get too few visual tokens, and the model answers from what it already knows rather than from what it was shown. Lines reading find_slot: non-consecutive token position are normal; that is how this family numbers image patches, and they appear in successful runs too.

Running it with Ollama or LM Studio

Both apps run llama.cpp underneath. Ollama added ornith-1.5 to its library on release day, so the official tag works: ollama run ornith-1.5:35b pulls a 23 GB build with the vision projector included. The library tag ships one build per size; for a specific quant from our table, pull it from Hugging Face directly:

ollama run hf.co/AtomicChat/Ornith-1.5-35B-A3B-GGUF:Q5_K_M

In LM Studio, search for the repository name in the model catalog; our files load there as well.

Troubleshooting

The GGUF you downloaded is not the real model

Repositories from other publishers claimed the Ornith 1.5 name on Hugging Face within a day of the release, including uncensored and abliterated forks. Before downloading from an unfamiliar repo, check that the publisher is either the official ornith-ai organization or a quantizer you recognize, that the model card exists, and that the file list actually contains GGUF files of a plausible size. A 35B model at 4-bit cannot weigh 400 MB.

The model rambles or never quite stops

Upstream sampling defaults are not what llama.cpp falls back to, so the engine runs its own, and the output rambles. Set DeepReinforce's recommended values yourself: temperature 0.6, top_p 0.95, top_k 20. In Atomic Chat, sampling lives behind the sliders icon next to the model name; for chat tasks you can also switch thinking off with the lightbulb icon.

Generation comes back empty on a Mac at long context

On Metal, prompts past roughly 16K tokens can return an empty response on this architecture: llama.cpp issue #27442 tracks it, reproduced across current builds. Until the fix lands, keep Context Size at 16,384 or below on Apple Silicon, or run the long-context work on a CUDA machine.

The model is slow on a card that should fit it

The file loads, then generation crawls. When the weights only just fit, part of them spills into shared system memory and the GPU reads it over the bus on every token. Offload deliberately instead: pass --n-cpu-moe N and raise N until the rest sits in VRAM. Attention and the router stay on the GPU, and only the experts read from system RAM.

You run out of memory at long context

The weights fit, then a long conversation crashes the model. That's the KV cache growing past your headroom. Set an explicit Context Size instead of 0, turn off Auto Increase Context Size when you're near the limit, and if you need more room, move the experts to system RAM with --n-cpu-moe instead of stepping down a quant.

Frequently asked questions

The questions that come up most often about running Ornith 1.5 35B on your own hardware.

How much VRAM does Ornith 1.5 35B need?

About 20 GB for AD-Q4_K-IQ4_XS, the build for 24 GB cards, at 92.71% top-1 agreement against the original. From 32 GB, AD-Q6_K behaves close to the original, and 48 GB runs Q8_0. With less VRAM, the experts move to system RAM: see the next question.

Can I run Ornith 1.5 35B on a 12 GB GPU?

Yes, if your system RAM covers the rest of the file. Pass --cpu-moe in llama.cpp and the experts, 93 percent of the weights, live in RAM while attention stays on the GPU: a 12 GB card serves the 22.1 GB AD-Q5_K-Q4_K at usable speed that way. Count the full file size against RAM plus VRAM when you pick the quant.

Can you run Ornith 1.5 35B on a Mac?

Yes. macOS caps GPU-addressable memory at roughly 75 percent of unified memory, so a 32 GB Mac gives the model about 24 GB: enough for AD-Q5_K-Q4_K. From 48 GB, run AD-Q6_K; Q8_0 fits from 64 GB. DeepReinforce also ships official MLX builds at 4, 6, and 8 bits. An open llama.cpp bug returns empty responses past 16K context on Metal; see Troubleshooting.

Is Ornith 1.5 35B better than Qwen3.6 35B A3B?

Yes, by DeepReinforce's launch numbers: it comes out ahead on every coding and agentic benchmark the card reports, including 67.8 against 52.5 on Terminal-Bench 2.1. The two models share the same shape, a 35B MoE with 3B active parameters, so they fit the same hardware.

How does Ornith 1.5 35B compare to Qwen 3.8 27B?

They tie on GPQA Diamond at 89.2, and on Terminal-Bench Qwen 3.8 27B holds the edge: 73.0 against 67.8, on different harness versions. The trade is hardware: the 27B is dense and needs a 24 GB card, while the 35B activates 3B parameters per token, so it generates faster. Expert offload also runs it on a 12 GB card. If your machine fits either, see our guide to running Qwen 3.8 locally and pick by workload.

What is the difference between Ornith 1.5 9B, 35B, and 397B?

The difference is size and hardware class. The 397B is the MoE flagship that DeepReinforce benchmarks against Claude Opus 4.8, and it needs a datacenter cluster. The 9B is a dense vision-language model for an 8 GB GPU or a 16 GB Mac, covered in the Ornith 1.5 9B guide. The 35B in between is a 3B-active MoE: the strongest of the three on a high-end desktop, and the one this guide covers.

Is Ornith 1.5 35B better than Ornith 1.0 35B?

Yes, by DeepReinforce's launch numbers: Terminal-Bench 2.1 goes from 64.2 to 67.8, SWE-bench Verified from 75.6 to 79, GPQA Diamond from 86.2 to 89.2, and MCP-Atlas from 64.4 to 70.2. The architecture and dimensions carry over, so if you run Ornith 1.0 today, every 1.5 build fits the same machine.

What is the MTP head and does it make the model faster?

MTP stands for Multi-Token Prediction: a 1.9B head that drafts several tokens per forward pass, which llama.cpp turns into speculative decoding with --spec-type draft-mtp. We publish it as a separate draft file rather than inside the quants, so the model runs fine if you never download it. The setup is in the llama.cpp section.

Does Ornith 1.5 35B support vision locally?

Yes. The 35B ships a vision encoder, and the projector file it needs is a separate download shared by all quants. In Atomic Chat you attach an image to the chat and ask about it, with nothing leaving your machine; through llama.cpp you pass the mmproj file alongside whichever quant you picked.

Does Ornith 1.5 35B work with Ollama or LM Studio?

Yes. Ollama has an official entry: ollama run ornith-1.5:35b pulls a 23 GB build. Our GGUF builds also load directly: point Ollama at hf.co/AtomicChat/Ornith-1.5-35B-A3B-GGUF, or search the repository name in LM Studio.

Is Ornith 1.5 35B free for commercial use?

Yes. The whole Ornith 1.5 family ships under the MIT license: commercial use, modification, and redistribution are all allowed.

Bottom line

If your machine has a 24 GB GPU or a 32 GB Mac, Ornith 1.5 35B is the strongest Ornith you can run locally: download AD-Q4_K-IQ4_XS on the GPU or AD-Q5_K-Q4_K on the Mac, start at 8K context, and raise it as the work demands. On a 12 GB card, keep the experts in system RAM with --cpu-moe instead of stepping down to a smaller model. The llama.cpp route serves the same files behind a local OpenAI-compatible API.

Key takeaways:

• Ornith 1.5 35B is a 35B MoE for agentic coding that activates about 3B parameters per token, with a 262K native context window.
• The weights went public on August 19, 2026, under the MIT license, with the 9B and the 397B alongside.
• Our GGUF builds run from 13.7 to 36.9 GB. For a 24 GB card, take AD-Q4_K-IQ4_XS; for a 32 GB Mac, AD-Q5_K-Q4_K.
• A 12 GB GPU runs the 22.1 GB build with the experts in system RAM: pass --cpu-moe.
• Upstream sampling defaults are not what llama.cpp falls back to: set temperature 0.6, top_p 0.95, and top_k 20 yourself.