Infrastructure & Compute
Distributed training
Training a model using multiple processors or machines working together.
Example
Hundreds of machines coordinate one training run and exchange updates.
Why it matters
It makes larger or faster learning runs possible by combining resources.
You'll hear it
“The machines are spending too much time waiting for one another.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does doubling machines halve training time?
- Not necessarily. Shared work and communication add overhead.
- Can one failed machine interrupt the run?
- Yes. Recovery depends on the training setup and saved state.
- Can distributed training produce slightly different results?
- Yes. Calculation order and system behavior can affect the learning process.