Gemma 2 27B VRAM: what it needs at Q4_K_M, Q8_0 and FP16
Runs in your browser — nothing you paste leaves this page. How we prove that
LLM VRAM calculator playground
KV cache is counted at FP16. Overhead is a fixed 1.5 GiB estimate for the runtime and compute buffers. A custom size borrows the architecture of the nearest preset.
Results update as you type — press Enter to run now.
GPU tiers
- 8 GiBtoo smallRTX 4060, RTX 3070, RX 7600
- 12 GiBtoo smallRTX 4070, RTX 3060 12GB, RTX 5070
- 16 GiBtoo smallRTX 4060 Ti 16GB, RTX 4080, RTX 5080
- 24 GiBfitsRTX 4090, RTX 3090, RX 7900 XTX
- 32 GiBfitsRTX 5090, V100 32GB
- 48 GiBfitsRTX A6000, RTX 6000 Ada, L40S
- 80 GiBfitsA100 80GB, H100 80GB
At Q4_K_M with an 8,192-token context, Gemma 2 27B needs about 19.7 GiB of VRAM: 15.4 GiB of weights, 2.88 GiB of KV cache and 1.50 GiB of runtime overhead. The smallest common GPU tier that holds it is 24 GiB (RTX 4090, RTX 3090, RX 7900 XTX).
How much VRAM Gemma 2 27B needs
Gemma 2 27B, Google's largest Gemma 2 model from June 2024, was built to run inference at full precision on a single 80 GB datacentre GPU such as an A100 or H100; quantized, it fits a single 24 GiB consumer card.
With 8,192 tokens of context the estimate is 54.7 GiB at FP16, 31.1 GiB at Q8_0 and 19.7 GiB at Q4_K_M. FP16 needs an 80 GiB card, Q8_0 needs a 32 GiB card, and Q4_K_M needs a 24 GiB card. The weights are 27 billion parameters times the bits per weight of each format (16, 8.5 and 4.89), divided by eight; everything is in GiB, the unit GPU memory is sold in.
Context length and the KV cache
The KV cache stores a key and a value vector for every token in every layer. Gemma 2 27B has 46 layers, 16 KV heads and a head dimension of 128, read from its Hugging Face config.json, so at FP16 each token costs 368 KiB. That is 2.88 GiB at 8,192 tokens, which is also the model's maximum context. The cache does not shrink with weight quantization; only a runtime option that quantizes the cache itself does.
Which quantization to pick
Q4_K_M at 19.7 GiB fits a 24 GiB card; Q8_0 at 31.1 GiB needs a 32 GiB card, and FP16 at 54.7 GiB needs an 80 GiB card.
Pitfalls
Like the 9B, it stops at 8,192 tokens, and 16 KV heads make the cache 368 KiB per token, 2.88 GiB at the cap. Half the layers use a 4,096-token sliding window, so runtimes that exploit it allocate less than this estimate.
Qwen2.5 32B and DeepSeek-R1-Distill-Qwen 32B compete for the same 24 GiB card; Gemma 2 9B is the small sibling.
FAQ
Questions, answered.
Tap a question to expand the answer.
Can Gemma 2 27B run on an RTX 4090?
Yes at Q4_K_M: 19.7 GiB at its full 8,192-token context. Q5_K_M at 22.3 GiB also fits 24 GiB; Q8_0 at 31.1 GiB does not.
Can Gemma 2 27B use more than 8k context?
No. Its configuration sets 8,192 as the maximum position, and pushing past it without a long-context fine-tune degrades output. Gemma 3 is the Google option for longer contexts.
More free, private DevOps tools.
The LLM VRAM Calculator is one tool in OpsCanopy — a growing canopy of browser-based validators, converters and testers that never touch a server.
Related tools
New to AI & local LLMs? Read the AI & local LLMs guide →
42 free tools, every one offline-capable — opscanopy.com works with no signup and nothing uploaded.
Related: the full LLM VRAM Calculator, the LLM Token Counter, or every AI tool.
Estimates only; the KV cache is sized at FP16 and the overhead is a fixed allowance, so always confirm against your own runtime’s report before buying hardware. OpsCanopy is free and open.