GPU VRAM Calculator
Estimate the GPU VRAM needed to run or train a transformer / large language model. Enter parameter count, precision (fp16, int8, int4…), context length and batch size to break down weights, KV cache, activations, gradients and optimizer state, and check which GPUs it fits on.
Input
Pick a known model to fill parameters, hidden dim and layers, or choose Custom.
Total model parameters in billions (7 for a 7B model).
Model hidden dimension (drives KV cache and activation size).
Number of transformer layers.
Tokens per sequence (context length).
Output
| Component | Memory | Share |
|---|---|---|
| No data yet | ||
| GPU | VRAM | Fit | GPUs Needed |
|---|---|---|---|
| No data yet | |||
Guides
The GPU VRAM Calculator estimates how much video memory a transformer or large language model needs to run — for both inference and training — so you can tell in seconds whether a model fits on your GPU before you rent, buy or download anything.
Enter a parameter count (or pick a preset like Mistral 7B or Llama 3 70B), choose a precision, and set your context length and batch size. The tool breaks the total down into every component that consumes VRAM and checks the result against a range of common GPUs.
How VRAM is calculated
Memory sizes are reported in GiB (base‑1024) — the same unit nvidia-smi shows.
Inference needs three things in memory:
- Model weights —
parameters × bytes per parameter. Precision sets the bytes: fp32 = 4, fp16/bf16 = 2, int8 = 1, int4 = 0.5. A 7‑billion‑parameter model in fp16 is7B × 2 = 14 GBof weights. - KV cache — the attention keys and values cached for every token:
2 × batch × sequence length × hidden dim × layers × 2 bytes. This grows with context length and is why long‑context inference can need far more memory than the weights alone. Quantized (int8/int4) models still keep the KV cache in fp16. - Activations and ~10% framework overhead — scratch buffers plus the CUDA context and allocator fragmentation.
Training is much heavier because it also stores:
- Gradients — one value per parameter.
- Optimizer state — Adam/AdamW keeps fp32 momentum and variance, 8 bytes per parameter; SGD with momentum uses 4; plain SGD uses 0.
- fp32 master weights — kept under mixed‑precision training (+4 bytes per parameter).
- Activations — the dominant training cost, reduced by gradient checkpointing (recomputing activations in the backward pass) at the price of extra compute.
This is why full fine‑tuning of a 7B model with Adam needs roughly 16–20 bytes per parameter — over 100 GB — while running the same model for inference in fp16 needs only about 15 GB.
Tips for fitting a model
- Quantize. Dropping from fp16 to int4 cuts weight memory by 4×, turning a 70B model from ~140 GB down to ~35 GB.
- Shorten the context. The KV cache scales linearly with sequence length and batch size.
- Use LoRA / QLoRA for fine‑tuning to avoid storing full gradients and optimizer state for every weight.
Are these numbers exact?
No — treat them as a planning baseline. Real usage depends on the framework, the attention implementation (FlashAttention lowers activation memory), tensor parallelism and allocator behavior. The estimates run entirely in your browser; nothing is uploaded.