Enter a model's parameter count to estimate the GPU memory needed for inference or fine-tuning.
Running or training a language model consumes GPU memory across several distinct categories, not just the model's raw weights. Model weights are the parameters themselves, the baseline cost that scales directly with parameter count and precision. Activations are the intermediate values computed during a forward pass, these scale with batch size and sequence length, and grow substantially during training since they must be retained for the backward pass. Gradients and optimizer states only apply during training, not inference, and are frequently the largest consumers of memory, an Adam optimizer alone typically needs about 4x the model weight size just for its internal momentum and variance tracking states.
This is the single most common source of "out of memory" errors for people getting started with LLM fine-tuning. Running a 7B parameter model for inference in FP16 needs roughly 14GB, well within a single consumer GPU. Full fine-tuning that same model needs weights plus gradients plus optimizer states plus activations, often 5-6x more memory than inference alone, which is why full fine-tuning of even moderately sized models typically requires professional-grade GPUs or multiple GPUs working together.
LoRA (Low-Rank Adaptation) dramatically reduces fine-tuning memory requirements by freezing the original model weights entirely and only training small additional "adapter" matrices injected into each layer. Since the vast majority of parameters never receive gradients or optimizer state, the memory overhead for gradients and optimizer states drops to a small fraction of the full fine-tuning cost, this is exactly why LoRA made fine-tuning 7B+ models feasible on a single consumer GPU, something that would otherwise require enterprise hardware with full fine-tuning.
Actual memory usage depends on many factors this calculator simplifies: sequence length, specific model architecture, framework overhead, gradient checkpointing (which trades compute for memory), and whether techniques like quantization or memory-efficient attention are used. Treat this as a planning estimate to rule out obviously insufficient hardware, not an exact prediction, always leave meaningful headroom (at least 10-20%) beyond the estimate when selecting actual hardware.