Infrastructure & Compute
Sharding
Splitting model parameters or data across multiple devices or machines.
Example
A model too large for one GPU is divided across several devices.
Why it matters
It lets work or information exceed the capacity of a single device.
You'll hear it
“That portion of the model is stored on another machine.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does sharding remove the need for communication?
- No. Parts may need to exchange information to complete the work.
- Can uneven shards create slowdowns?
- Yes. A heavily loaded part can delay the rest.
- Is a shard necessarily a complete usable copy?
- No. It may contain only part of the required information.