Generative AI & LLMs
Pretraining
The large-scale initial training stage that teaches a foundation model broad patterns.
Example
A language model first learns broad language patterns from a very large text collection.
Why it matters
It creates broadly reusable abilities before more specialized adaptation.
You'll hear it
“The initial learning covered lots of writing, but little from our field.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does pretraining teach an assistant to follow every request?
- Not necessarily. Later work may be needed to make the system useful in a conversational product.
- Can old teaching material leave gaps about recent events?
- Yes. Events absent from its learning require newer information from another source.
- Is pretraining repeated for every conversation?
- No. It is an earlier learning stage, not something restarted whenever someone asks a question.