Calculator 15 • AI Infrastructure & Compute Sizing
LLM GPU VRAM & Fine-Tuning Calculator
Size model weights, KV-cache concurrency, QLoRA/LoRA/Full AdamW fine-tuning memory, and compare dedicated H100 SXM GPUs against managed token APIs.
1. Model & Context Workload
Inference VRAM
186.4 GB
Weights: 140.0 GB
Serving GPUs Needed
3x GPUs
$5,365/mo cluster
LoRA Fine-Tune VRAM
198.0 GB
BF16 LoRA r=16
Full Fine-Tune VRAM
1,120 GB
14x GPUs (ZeRO-3)
Workload Memory & Self-Hosted vLLM vs. API Breakeven
| Workload Mode | Total VRAM Needed | Min GPUs | Monthly Compute Cost |
|---|---|---|---|
| vLLM Production Inference (Weights + KV Cache + 12% Overhead) | 186.4 GB | 3x | $5,365/mo (24/7) |
| QLoRA 4-Bit Fine-Tuning (NF4 + Paged AdamW) | 58.5 GB | 1x | On-demand job |
| BF16 LoRA Fine-Tuning (16-Bit Base + Adapters) | 198.0 GB | 3x | On-demand job |
| Full BF16 Parameter Fine-Tuning (AdamW + Gradients) | 1,120.0 GB | 14x | Multi-node FSDP/ZeRO-3 |
| Managed Token API Equivalent ($1.20 / 1M blended tokens) | Serverless | 0 GPUs | $4,320/mo |