AI Foundations
Multi-head attention
An AI method that examines several different kinds of relationships between pieces of information at the same time.
Example
A transformer combines information from multiple attention heads.
Why people use it
It lets a transformer combine several learned ways of attending to information.
What you'll hear
“Different heads can focus on different relationships.”
What this means for you
Avoid assigning a simple meaning to a head without supporting analysis.
Can you control it?
Developer-only
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does each part have a neat human-readable job?
- Not necessarily. The learned roles can overlap and may not correspond to simple ideas people can name.
- Does adding more heads always help?
- No. More heads add complexity and do not guarantee better performance.
- Can heads learn overlapping behavior?
- Yes. They do not necessarily divide the work into distinct, easily named roles.