Model Training & Adaptation
Exploration-exploitation tradeoff
Balancing trying new options with choosing options that have worked well before.
Example
A recommendation system sometimes tests a new option instead of repeating a successful one.
Why people use it
It helps balance learning about new choices with using choices already known to work.
What you'll hear
“Should we try something new or use the reliable option?”
What this means for you
Set limits on experimentation when actions affect people or resources.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Can the best option change over time?
- Yes. Changing conditions can make a previously successful choice less useful, giving fresh exploration a purpose.
- Can always choosing the current favorite miss something better?
- Yes. An untried option may be better but never get a chance to show it.
- Can experimentation have a real cost?
- Yes. A less successful choice can affect customers, time or money while the system is learning.