Neural Network Parameter Counter

Build your network layer by layer and instantly see total trainable parameters and estimated model size.

Advertisement
Total Parameters
Size (FP32)
Size (FP16)
Size (INT8)
Per-Layer Breakdown
Advertisement

How Parameters Are Counted in a Dense Layer

In a fully connected (dense) layer, every neuron in one layer connects to every neuron in the next layer, each connection has a learnable weight, plus every neuron in the receiving layer has one learnable bias term. The formula is straightforward: parameters = (inputs × outputs) + outputs. A layer going from 784 inputs (like a flattened 28×28 image) to 128 neurons has 784 × 128 + 128 = 100,480 parameters, just for that one layer transition.

How Parameters Are Counted in a Convolutional Layer

Convolutional layers use weight sharing, the same small filter (kernel) slides across the entire input, dramatically reducing parameters compared to a dense layer covering the same input size. The formula is: parameters = (kernel_height × kernel_width × input_channels + 1) × output_channels, the +1 accounts for one bias term per output filter. A 3×3 convolution from 32 to 64 channels has (3×3×32 + 1) × 64 = 18,496 parameters, versus what would be millions of parameters if that same transformation were done with a dense layer.

Why Model Size in Memory Matters

Each parameter needs to be stored somewhere, and the storage format directly affects model size. FP32 (32-bit floating point) is the default training precision, using 4 bytes per parameter, this is the most precise but also the largest. FP16 (16-bit, "half precision") uses 2 bytes per parameter, commonly used for inference and increasingly for training, roughly halving memory with usually minimal accuracy loss. INT8 (8-bit integer quantization) uses just 1 byte per parameter, a common technique for deploying models on memory-constrained devices like phones, at the cost of some accuracy.

Why This Matters for Real Deployment

A model with 100 million parameters at FP32 needs roughly 400MB just to store the weights, before accounting for activation memory during inference, which can be substantially larger depending on batch size and input dimensions. This is exactly why understanding parameter count matters early in model design, it directly predicts whether a model will fit on your target hardware, whether that's a cloud GPU, a laptop, or a mobile device.