Generative AI & LLMs
Lemmatization
Turning a word into its dictionary form while considering how it is used.
Example
A writing tool connects 'running' with its dictionary form, 'run.'
Why people use it
It helps treat different forms of a word as related during text analysis.
What you'll hear
“Treat 'running' as a form of 'run' here.”
What this means for you
Check that simplifying words does not erase differences your task needs.
Can you control it?
Developer-only
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can it handle irregular words?
- Yes. With suitable language knowledge, it can connect 'went' with 'go,' rather than simply trimming letters.
- Is it just chopping off word endings?
- No. It uses language information to find an appropriate base form.
- Can background information change the base form chosen?
- Yes. The word's role and meaning in the sentence can affect the choice.