LLM VRAM Calculator: Size the GPU before you download the model.

Add three parts. Weights are the parameter count multiplied by the bits per weight of your quantization, divided by 8 to get bytes and by 2^30 to get GiB — an 8B model is about 15 GiB at FP16, 8 GiB at Q8_0 and 4.6 GiB at Q4_K_M. The KV cache grows with context length (next question), and the runtime adds a fixed buffer, which this calculator sets at 1.5 GiB. The total has to fit the card's memory with a little to spare; if it does not, pick a smaller quantization or a shorter context before reaching for a bigger GPU.

Quantization stores each weight in fewer bits than the 16 it was trained in. FP16 keeps 16 bits per weight (bpw); Q8_0 uses about 8.5 bpw and is practically lossless; Q4_K_M uses about 4.89 bpw — roughly a third of the FP16 size — with a small quality cost most people cannot notice in chat. The fractional values come from llama.cpp's mixed layouts, which keep a few sensitive tensors at higher precision and store a scale factor per block, which is why a "4-bit" file is not exactly half of an 8-bit one.

Every token in the context keeps a key and a value vector in every layer, so the KV cache is 2 × layers × KV heads × head dimension × tokens × 2 bytes at FP16. For Llama 3.1 8B (32 layers, 8 KV heads, head dimension 128) that is 128 KiB per token: 1 GiB at 8,192 tokens and 16 GiB at the full 131,072 — more than the Q4_K_M weights themselves. Grouped-query attention (8 KV heads instead of 32) cuts the cache fourfold, which is why Llama 3 fits long contexts where Llama 2 did not. The context slider draws the cache as its own bar segment so you can see where the memory goes.

GPU memory is sold in binary gigabytes: a "24 GB" RTX 4090 holds 24 GiB, which is 25.77 GB. A GB is 10^9 bytes and a GiB is 2^30 = 1,073,741,824 bytes, 7.37% larger, so dividing by 10^9 would make every model look 7.4% bigger than it is and mis-call fits right at the edge. The calculator divides by 2^30 everywhere, so a verdict compares like with like.

Weights and KV cache are not the whole picture: the runtime (llama.cpp, Ollama, vLLM) allocates compute buffers for activations, the CUDA or Metal context takes a few hundred MiB, and the driver reserves some memory of its own. 1.5 GiB is a round estimate that covers these for most models on a single card; the real value varies by runtime, batch size and GPU. Treat it as a margin, not a measurement — if a verdict lands within half a GiB of a tier, expect trouble.

Not fully on the GPU. Llama 3.1 70B at Q4_K_M needs about 40 GiB for the weights alone, before the KV cache and runtime overhead, so a single 24 GiB card cannot hold it. It fits on a 48 GiB card (RTX 6000 Ada, A6000) or across two 24 GiB cards with layer splitting; Q2_K at about 3.16 bpw brings the weights down to roughly 26 GiB, still too much for one card and with a visible quality cost. Runtimes can offload the remaining layers to system RAM, but token speed then drops to what the CPU's memory bandwidth allows, typically a few tokens per second.

Apple Silicon shares one pool of memory between CPU and GPU, so a Mac with 64 GB can load models no 24 GiB card can. By default macOS lets the GPU use roughly 75% of unified memory (about 48 GiB on a 64 GB machine) and keeps the rest for the system; the iogpu.wired_limit_mb sysctl can raise the limit at your own risk. Enter about three quarters of the Mac's memory as the budget, and remember that bandwidth, not capacity, sets token speed: an M-series Max moves roughly 400 GB/s, an RTX 4090 about 1 TB/s.

No. The calculator runs 100% client-side: the architecture presets and the arithmetic ship with the page, and every number is computed in your browser tab. Nothing is uploaded to a server, there is no account or signup, and the share link encodes your selection in the URL fragment, which browsers never send to a server.

For the weights alone: an 8B model needs about 4.6 GiB in Q4_K_M, 7.9 GiB in Q8_0 and 14.9 GiB in FP16 (a 7B about 4.0, 6.9 and 13.0 GiB); a 14B needs about 8.0, 13.9 and 26.1 GiB; a 70B about 39.9, 69.3 and 130.4 GiB. Add the KV cache for your context — at 8,192 tokens that is 1 GiB for Llama 3.1 8B and 2.5 GiB for Llama 3.1 70B — plus the 1.5 GiB runtime overhead, so an 8B in Q4_K_M at 8k context lands near 7.1 GiB in total. Each popular model has its own page with its real layer count and KV heads filled in, so you can read the verdict for that model directly.

The weights are a fixed size, but the KV cache grows with every token: it costs 2 × layers × KV heads × head dimension × 2 bytes per token in FP16. For Llama 3.1 8B that is 2 × 32 × 8 × 128 × 2 = 128 KiB per token, so the full 131,072-token context needs 16 GiB of cache against about 4.6 GiB of Q4_K_M weights. Grouped-query attention is what keeps it that small — 8 KV heads instead of 32 cuts the cache to a quarter — while older multi-head-attention models with one KV head per query head run out of memory at far shorter contexts.