LLM Inference Calculator
Estimate RAM, disk space, speed and cost for running large language models.
LLM Inference Calculator — estimate RAM, disk, tokens per second and cost for large language models before you run them.
How to use LLM Inference Calculator
- Pick a model preset or enter total parameters in billions.
- For MoE models, tick the MoE box and enter active parameters per token.
- Choose quantization bits and your context length in tokens.
- Pick your hardware preset or enter memory, bandwidth and TFLOPS.
- Enter your own input and output rates per million tokens, or keep the SAMPLE rates.
- Press Calculate Estimate and read the ESTIMATED results, then copy or download them.
Features
- Dense and MoE model support with separate total and active parameters
- Disk size, KV cache and total RAM estimates for any context length
- Tokens per second estimate with memory-bandwidth or compute bottleneck shown
- Time to first token estimate from prompt length and hardware speed
- Cost per million tokens from your own editable SAMPLE rates
- Hardware presets from laptop CPU to H100, plus fully custom hardware
- Published project figures table with a verify note
- Copy and download your estimate as plain text
Frequently asked questions
Is it free?
Yes, unlimited.
Login to use this free online tool. Create a free account to get started.