How much does training a model cost?
Each training token passes through the model’s active parameters. The simplest useful estimate is training cost ≈ training tokens × active parameters.
Hardware, memory and efficiency adjust the final price, but tokens and active parameters explain most of the core computation.
How does the number of training tokens affect cost?
Almost linearly. Twice as many tokens means roughly twice the computation and therefore roughly twice the cost.
What do active parameters change?
They determine how much work happens for each token. Twice the active parameters means roughly twice the computation per token.
What do total parameters change?
They increase capacity. They don’t all run for every token, but a larger model may need more tokens so every expert learns enough.
Can differently sized models cost the same per token?
Yes. A 100B/10B-active model and a 200B/10B-active model activate about the same parameters per token. The larger model still needs more memory and may need more training data.
Our DeepSeek V4 Flash estimate
probably cost
$5–10M to train
$5–10M estimates the successful run. It excludes research, failed experiments, data work, salaries and infrastructure ownership.
V4 report · V3 report