Generative AI & LLMs
Self-attention
Attention where the model compares elements within the same sequence to understand their relationships.
Example
A sentence compares each word with the other words in that same sentence.
Why it matters
It helps a system interpret pieces of information using their surroundings.
You'll hear it
“The meaning of that word depends on the rest of this sentence.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can a word be linked to several other words?
- Yes. Different relationships can contribute to how a passage is processed.
- Does self-attention give a system a sense of self?
- No. Self refers to comparing material within the same sequence, not personal awareness.
- Is word order irrelevant when this is used?
- No. Systems usually also need information about position or order to interpret the material correctly.