Qwen2.5-7B-Instruct

Updated
05.10.2026
Reasoning
Code
Multilingual
Tools

A 7.61B instruction-tuned LLM from Alibaba’s Qwen2.5 series with strong coding, math, and multilingual ability across 29+ languages.

At a glance

  • License: Apache 2.0
  • Parameters: 7.61B total, 6.53B non-embedding
  • Context length: 131,072 tokens (32,768 in the shipped config)
  • Modalities: Text input, text output
  • Minimum hardware: 6 GB of memory for the Q3_K_M GGUF (3.81 GB)

What is Qwen2.5-7B-Instruct?

Qwen2.5-7B-Instruct is the instruction-tuned 7B model in Alibaba Cloud's Qwen2.5 series, a release that spans base and instruct models from 0.5 to 72 billion parameters. The Qwen team published it in September 2024 under Apache 2.0, and the repo has passed 11.5 million downloads on Hugging Face. Compared with Qwen2 it carries more knowledge and better coding and math, and at 7.61B parameters the Q4_K_M GGUF build is 4.68 GB across its two split files.

SpecificationQwen2.5-7B-Instruct
Total parameters7.61B
Non-embedding parameters6.53B
ArchitectureDense transformer with RoPE, SwiGLU, RMSNorm and attention QKV bias
Layers28
Attention heads (GQA)28 for queries, 4 for keys and values
Context window131,072 tokens (32,768 in the shipped config)
Max generation8,192 tokens
Languages29+ including English, Chinese, French, Spanish, Russian, Japanese, Arabic
Release dateSeptember 2024
LicenseApache 2.0

The shipped config.json is set to 32,768 tokens of context. To go past that, up to the full 131,072, Qwen documents a YaRN rope_scaling block with factor 4.0 that you add to the config, and recommends vLLM for deployment. vLLM currently supports only static YaRN, where the scaling factor stays constant no matter the input length, which can cost some quality on short prompts. Qwen's advice is to enable it only when you actually process long texts.

What Qwen2.5-7B-Instruct is good at

Qwen does not print benchmark numbers on the model card itself; the detailed evaluation results live in the Qwen2.5 blog post. The card's own claims are still concrete. Compared with Qwen2, the model has significantly more knowledge and greatly improved coding and mathematics, gains Qwen credits to the specialized expert models it trained for those domains.

The instruct tuning targets practical chatbot work: better instruction following, generating long texts past 8K tokens, understanding structured data such as tables, and generating structured output, especially JSON. Qwen also calls the model more resilient to varied system prompts, which helps role-play setups and condition-setting for chatbots. Multilingual coverage spans over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic.

Qwen2.5-7B-Instruct hardware requirements

The system requirement to check is memory: the file has to fit in your RAM or VRAM with room left for context. Qwen publishes official GGUF builds in Qwen/Qwen2.5-7B-Instruct-GGUF; several ship as split files, and the sizes below are the combined download.

MemoryBuild to pickFile size
6 GBQ3_K_M3.81 GB
8 GBQ4_K_M4.68 GB
12 GBQ6_K6.25 GB
16 GBQ8_08.10 GB
24 GB and upFP1615.23 GB

When two builds both fit, take the larger one. If the format is new to you, start with what GGUF is, and see the best local LLMs for a 16 GB Mac for how a 7B compares with everything else in that class.

How to run Qwen2.5-7B-Instruct in Atomic Chat

Atomic Chat is a free local app for macOS, Windows and Linux. It includes a Hugging Face model browser and a built-in chat, with no manual llama.cpp build required.

  1. Download Atomic Chat for your platform and open it.
  2. Search for Qwen2.5-7B-Instruct in the model browser and open Download Options.
  3. Pick the build that fits the memory you have, then start a chat.

The series scales up from here: the next step is Qwen2.5-14B-Instruct, and every Qwen model you can run locally is on the family page.

Qwen2.5-7B-Instruct license

Qwen2.5-7B-Instruct is released under Apache 2.0, with the license file included in the repo. That permits commercial use, modification and redistribution with no royalties, so you can build products on top of the model and run it on your own hardware without a usage fee.

Get the weights from Hugging Face

pip install -U transformers
huggingface-cli download Qwen/Qwen2.5-7B-Instruct
# or run quantized via Ollama:
ollama run qwen2.5:7b-instruct
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)
messages = [{"role": "user", "content": "Give me a short intro to LLMs."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://localhost:8000/v1", apiKey: "sk-local" });
const res = await client.chat.completions.create({
  model: "Qwen/Qwen2.5-7B-Instruct",
  messages: [{ role: "user", content: "Hello!" }]
});
console.log(res.choices[0].message.content);
Desktop
macOS
(Intel and Apple Silicon)
Download
Windows
(x64)
Download
Windows
(ARM64)
Download
Linux
(x86_64)
Download

Frequently asked questions

Qwen2.5-7B-Instruct is an instruction-tuned large language model with 7.61 billion parameters, built by Alibaba Cloud's Qwen team and released in September 2024. It is post-trained for chat, instruction following, coding, and mathematics, and supports a context window of up to 128K tokens.

At full precision the 7.61B model needs roughly 16 GB of VRAM, so a single 16 GB GPU works well. With 4-bit quantization (GGUF or AWQ) it runs comfortably on an 8 GB GPU, and quantized GGUF builds can also run on CPU or Apple Silicon through llama.cpp and Ollama.

Yes. Qwen2.5-7B-Instruct is released under the Apache 2.0 license, which permits free use, modification, redistribution, and commercial deployment with no royalty. The weights are openly downloadable from Hugging Face, so you can self-host it without an API.

Qwen2.5-7B-Instruct supports a maximum context length of 131,072 tokens (128K) and can generate up to 8,192 tokens in a single response. The shipped config defaults to 32,768 tokens; handling the full 128K window requires enabling YaRN rope scaling, which frameworks like vLLM support.

The model supports more than 29 languages, including English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic. Qwen2.5 also improved structured-data understanding and JSON output, which carries across these languages.