Advanced Concepts
Scaling law
An empirical relationship showing how model performance changes as data, parameters, or compute increase.
Example
A scaling law estimates how performance changes with more data, parameters, or compute.
Why it matters
It helps researchers estimate how to allocate training resources before committing to a larger run.
You'll hear it
“Does that trend still hold at the scale we're planning?”
Can you control it?
Not directly
No, not directly. A scaling law describes a learned property, observed result, or broader concept rather than a normal user setting. You may influence it indirectly by changing the tool, data, prompt, or workflow.
Common questions
- Does a scaling law guarantee success on my task?
- No. Broad performance trends may not predict how a model handles one particular real-world task.
- Can a trend stop holding?
- Yes. Different data, designs or resource limits can change the relationship.
- Does it predict when AGI will arrive?
- Not on its own. That requires additional assumptions about future progress and what counts as AGI.