What is VRAM? It is the memory on your graphics card; the model must fit in it to run at full speed (full explainer: what is VRAM).
Short answer: at 4-bit quantization, plan on roughly 0.61 GB per billion parameters (Q4_K_M ≈ 4.89 bits/weight) plus a few GB for context. 8B models fit 8 GB cards tightly; 30B-class models want 24 GB; 70B models need 48 GB or a 128 GB unified-memory system.
| Model | Params (total / active) | Est. memory, Q4 @ 8k ctx | Est. memory, Q8 @ 8k ctx | Cheapest fit in our data (Q4) |
|---|---|---|---|---|
| Qwen3 4B | 4.0B / 4.0B | 4.9 GB | 6.8 GB | GeForce RTX 5060 ($299, GPU launch MSRP) |
| Llama 3.1 8B | 8.03B / 8.03B | 7.0 GB | 10.6 GB | GeForce RTX 5060 ($299, GPU launch MSRP) |
| Qwen3 14B | 14.8B / 14.8B | 11.5 GB | 18.2 GB | GeForce RTX 5060 Ti 16GB ($429, GPU launch MSRP) |
| Gemma 3 27B | 27.4B / 27.4B | 19.2 GB | 31.6 GB | GeForce RTX 3090 ($1,499, GPU launch MSRP) |
| Qwen3 32B | 32.8B / 32.8B | 22.5 GB | 37.3 GB | GeForce RTX 3090 ($1,499, GPU launch MSRP) |
| Qwen3 30B-A3B (MoE) | 30.5B / 3.3B | 21.1 GB | 34.9 GB | GeForce RTX 3090 ($1,499, GPU launch MSRP) |
| gpt-oss-20b (MoE) | 21B / 3.6B | 13.7 GB | 24.8 GB | GeForce RTX 5060 Ti 16GB ($429, GPU launch MSRP) |
| Llama 3.3 70B | 70.6B / 70.6B | 46.8 GB | 78.7 GB | GMKtec EVO-X2 (128GB / 1TB) ($3,499.99, system, listed price) |
| gpt-oss-120b (MoE) | 117B / 5.1B | 64.7 GB | 126.8 GB | GMKtec EVO-X2 (128GB / 1TB) ($3,499.99, system, listed price) |
| Qwen3 235B-A22B (MoE) | 235B / 22B | 146.1 GB | 252.2 GB | No single device in our dataset |
| Llama 3.1 405B | 405.9B / 405.9B | 253.3 GB | 436.5 GB | No single device in our dataset |
| DeepSeek-R1 671B (MoE) | 671B / 37B | 412.6 GB | 715.4 GB | No single device in our dataset |
Parameter counts: Hugging Face: deepseek-ai/DeepSeek-R1 model card; Google: Gemma 3 model card; Hugging Face: openai/gpt-oss-120b model card; Hugging Face: meta-llama/Llama-3.1-8B; Hugging Face: meta-llama/Llama-3.3-70B-Instruct; Hugging Face: meta-llama/Llama-3.1-405B; Hugging Face: Qwen3 model cards (e.g. Qwen/Qwen3-30B-A3B). Memory figures are our estimates (see methodology), not measurements.
What changes the number
- Quantization: 8-bit roughly doubles 4-bit memory; FP16 roughly quadruples it. See quantization explained.
- Context length: the KV cache grows linearly with context.
- Runtime: different engines add different overhead.