I have a friend, call him Consider these dilemmas: * At a restaurant, do you pick something new from the menu, or the burger you know and love? * For travel, do you go someplace new, or the same hotel you went to five times already? * Do you listen to new music, or go with music from 5+ years ago that you know? Most people go with the former as they're younger, and the latter as they get older. ### Multi-armed bandit It turns out that machines do the same thing! ![[multi-armed bandit.png|400]] [Multi-armed bandit](https://en.wikipedia.org/wiki/Multi-armed_bandit ) is a classic decision theory problem. You have several slot machines ("one-armed bandit"). Each one is free to play, but pays out differently. In each turn, you could either: * Go with the option that has the best rewards out of everything you tried so far (exploit), or * Try a new option (explore) When you explore, you're taking a smaller reward this round, to get [^1]higher rewards for the **rest of the game**. This is how I encourage my daughter to try new food: if you don't like it, then it's just a bad taste now, but if you do like it - you get to eat it for the rest of your life! (Spoiler: it doesn't work). Because of this, it pays off to **explore earlier in the game**, because the "rest of the game" is longer. It's no surprise then that optimal algorithms explore more frequently earlier in the game. ### Neuroplasticity Humans do the same as machines. Hence aphorisms like "you can't teach an old dog new tricks" or "as the twig is bent, so grows the tree". This *both* a cultural and [neurological](https://pmc.ncbi.nlm.nih.gov/articles/PMC8844884/) trait. As [Alison Gopnik writes](https://aeon.co/essays/why-childhood-and-old-age-are-key-to-our-human-capacities), this is coded in our very way of life: the extended, protected childhood allows more time for exploration; and [[Software vs hardware (Baldwin Effect)|Baldwin effect]] allows for more "programmable" traits. As we age, it's natural we shift towards exploit, and as Gopnik suggests there are activities we can take on (like teaching) that are appropriate for that phase of life. But while this shift was [[Adaptiveness|adaptive]] in our [[Environment of evolutionary adaptedness (EEA)|slow-moving ancestral environment]], it isn't in our age and our fast moving industries. It makes sense then to also push against the [[Gravitational pull]] of favoring exploit. #published 2025-02-22