%% ### #todo - [ ] Add from [[A Brief History of Intelligence]] - [ ] [[Human Compatible]] - [ ] Chapter 2: - [ ] Defines several breakthroughs in the evolution of biological organisms - [ ] Suggests that learning speeds up the rate of evolution itself. - [ ] Baldwin effect: instinctive organism with fixed responses vs. adaptive organism that learns - [ ] Culture protects learners and speeds up evolution - [ ] Compare against my original notes at the bottom from [[Evolution of the Learning Brain]] and add what's useful. - [ ] Consider where to draw the boundary for learning and memory. What can be seen as the first versions of those? - [ ] Connect the content with related notes in my learning project. - [ ] [[Classical (or Pavlovian) conditioning involves learning the relationship between two stimuli]] is related to associative learning and bilaterian brains. - [ ] [[Instrumental (or operant) conditioning involves learning the relationship between a response and the consequence that follows it]] is related to reinforcement learning and vertebrates. - [ ] At the end, reconsider what the main "innovations" were when it comes to learning, and what the associated changes in biology were that enabled these innovations. Write section headings including the innovation and the biological structure, and then start or end the section with a 1-2 sentence summary of the main learning innovation and biological change. - [ ] Run through ChatGPT for grammatical and stylistic improvement. - [ ] Publish %% Or: What is the historical sequence that led to learning in its current form in humans? In short, it's story of **biological and cultural development** spanning billions of years, from the simplest organisms to modern humans, all adapting to their environments in a quest for survival and reproduction. The content in this note is based on the book [[A Brief History of Intelligence]] by Max Bennett unless otherwise indicated. #### The evolutionary process Learning emerged as a function of life. **Life is a physical system that can replicate itself**.^[Lee, 2020] This replication is imperfect, meaning that copies are at least somewhat different from the original. These differences, although small and infrequent, can accumulate over time and lead to meaningful differences in the ability of descendants to replicate. Meanwhile, the environment selects systems that replicate efficiently. This leads to a process of **evolution by natural selection**. As it unfolds over many generations, different life forms adapt to their environments. They hone their capacity to overcome adaptive challenges in order to survive and replicate (i.e., reproduce). This evolutionary process connects the beginning of life to where we are today, over a span of about four billion years.^[See [this](https://en.wikipedia.org/wiki/Timeline_of_the_evolutionary_history_of_life) Wikipedia article for an evolutionary timeline.] The number of adjustments is simply unfathomable, but a few **evolutionary milestones** stand out that help us understand how learning emerged. Each milestone was the result of a set of modifications, driven by environmental pressures and opportunities, built on the foundation of previous modifications, and providing the foundation of milestones to come. By means of this **ordered sequence of milestones**, ever more complex systems evolved with ever more impressive capabilities that allowed them to make it into the next round of the "game" of evolution and resist the pull of the second law of thermodynamics, i.e., the trend towards disorder and decay. #### The beginnings of life The **origin of life on Earth** is still a mystery and topic of ongoing scientific debate.^[Howard-Jones, 2018] The Earth itself formed from the solar nebula about 4.5 billion years ago. Some scientists propose that life may have begun at deep-sea hydrothermal vents around 4 billion years ago, where mineral-laden water provides a rich source of chemicals.^[Gee, 2021] Initially, non-living chemicals had to come together in the right combination to form more complex ones, such as amino acids and nucleotides.^[Howard-Jones, 2018] One combination of nucleotides, a **DNA-like molecule**, managed to **duplicate itself** — a critical evolutionary threshold. Things started to ratchet up once a system managed to exist for long enough to replicate in this way, and have its descendants do the same. Two subsequent steps led to **life**: First, DNA molecules got entrapped in protective lipid bubbles to form the first versions of **cells**. Second, ribosomes began translating sequences of DNA into particular sequences of amino acids, synthesizing **proteins** that performed different functions inside the cell. Proteins enabled primitive forms of **sensation and movement**, allowing early life forms to monitor and respond to the outside world. Furthermore, protein synthesis elevated DNA to a medium for **storing information** from which life is constructed. These steps culminated in a cell that became the "**last universal common ancestor**" of all life forms on Earth around 3.5 billion years ago. %%**Learning** my have been present in the **earliest types of lifeforms** already.^[Howard-Jones, 2018] Fundamentally, learning is a strategy that allows organisms to adapt to changing environments on short timescales. To do so, organisms need to be able to detect changes in their environment and produce appropriate behavior given these changes. **Prokaryotes** might already have had some primitive form of learning and memory.%% Cells required a continuous input of **energy** to stay alive and replicate. Around 2.4 billion years ago, cyanobacteria developed **photosynthesis**, the ability to convert sunlight and carbon dioxide into sugar, providing a much more profitable mechanism for extracting and storing energy. The subsequent multiplication of cyanobacteria drastically increased oxygen levels, which allowed another form of bacteria to emerge that converted oxygen and sugar — the products of photosynthesis — into energy and carbon dioxide, a process called **cellular respiration**. This led to a **symbiosis between photosynthetic and respiratory life** that persists to this day. Respiratory life forms needed to steal the required inputs for cellular respiration from photosynthetic life. **Hunting** became their strategy for survival. At the same time, the life forms that were being hunted developed defensive mechanisms. Thus was kicked off an **arms race between predator and prey** that drove rapid innovations on both sides and an explosive **diversification of life forms** on Earth. Larger cells called **eukaryotes** evolved, which further diversified into the first plants (who were photosynthetic), fungi and animals (who were both respirators and thus needed to hunt). The main difference between fungi and animals is that the former eat other life forms after they die through external digestion, while animals eat living life and digest it inside the body. **Multicellularity** emerged independently in plants, fungi, and animals. In multicellular organisms, cells specialize for different functions while all serving the overall purpose of survival and replication of the organism.^[Howard-Jones, 2018] The **neuron** emerged as a specialized cell for information processing in animals. Neurons differ in shape and size between species, but they all work the same way. ^[Key aspects of neurons include *all-or-nothing spikes* (i.e., they're either firing or not firing), *rate coding* (i.e., information is encoded in the firing rate), *adaptation* (i.e., neurons have a flexible relationship between environmental variables and firing rates, allowing them to encode a wide range of natural variables within a firing range of only 0-500 spikes per second), and *chemical synapses with excitatory and inhibitory neurotransmitters* (i.e., increasing or decreasing the likelihood of a postsynaptic neuron firing).] Neurons allowed animals to respond to their environment with **speed and specificity**, which was important for hunting small multicellular life forms.^[Howard-Jones, 2018] Multiple interconnected neurons formed simple **nervous systems**. In a nervous system, an input signal from sensory areas gets passed through the network to output neurons which convert the signal into a motor response.^[Howard-Jones, 2018] %%At this level, learning involves the ability to change the connections between neurons in the network in response to experience, enabling the network to produce a new output for the same input. The advent of **chemical synapses**, alongside electrical ones, enabled nervous systems to take on a larger number of different states.^[Howard-Jones, 2018] Chemical synapses allowed for the modulation of signals through neurotransmitters, which could either amplify or dampen the signals. This synaptic plasticity became the cellular basis for learning and memory, exemplified by phenomena like long-term potentiation (LTP).%% #### Bilaterians and early brains Initially, neurons formed **nerve nets** — distributed networks of interconnected cells. As the evolutionary arms race between predators and prey intensified, these nets evolved into more complex structures known as **brains**. This transition aligned with the rise of **bilaterians**, organisms with bilateral symmetry, characterized by distinct front and back sides. Bilateral symmetry enhanced **movement efficiency**: while earlier radially symmetrical animals required comprehensive motor systems for omnidirectional movement, bilaterians only needed mechanisms for forward motion and turning. The advent of brains marked a significant leap, enabling larger, multicellular organisms to **navigate by steering**. This required animals to classify sensory inputs into categories: those to pursue and those to avoid. This classification, known as **valence**, depends on both environmental signals and the animal's internal state. For example, a stimulus signaling nearby food would be positively valenced if the animal were hungry, but negatively if it were satiated. Brains thus became essential as central control units, integrating sensory information to make coherent movement decisions — whether to move forward or turn. **Affect** evolved 550 million years ago in bilaterians to make better steering decisions. Affect is about the internal state of an organism, and it has two dimensions: **valence and arousal**. In steering, valence is about whether to stay in the current location or move elsewhere, whereas arousal is about whether to expend energy or not. The evolutionary advantage of affective states is that they can **persist** for longer than the internal or external stimuli that triggered them. Animals can make more effective choices about movement when guided by lasting affect rather than transient stimuli. Affective states are created through the release of **neuromodulators** such as dopamine and serotonin. Different neuromodulators create different affective states, leading to different behaviors such as approaching food, escaping predators, resting and digesting, and preserving energy. **[[Associative learning involves learning the relationship between two events that occur together|Associative learning]]** and its basic mechanisms emerged in early bilaterians. Its function is to **adjust valence** (i.e., the perceived value of stimuli) based on **experience**. It involves recognizing and remembering **relationships** between environmental events, such as a specific chemical signaling the presence of a predator. Since environments are dynamic, organisms need to **continually modify** these learned associations.^[Mechanisms to update associations include extinction, spontaneous recovery, and reacquisition.] Also, they must discern which environmental cues are most predictive of certain events — i.e., how to **assign credit** to predictive cues.^[Early brains solved this problem using four rules: Eligibility traces formed associations with predictive cues that occurred 0-1 seconds before the event. Overshadowing prioritized stronger cues over weaker ones. Latent inhibition ignored cues that were frequently experienced and focused on novel ones. Blocking ignored overlapping cues once an association was made with a specific cue.] This process is underpinned by **changes in synaptic strength** between neurons, which happens by many different mechanisms that are remarkably consistent across bilaterian species. These synaptic plasticity mechanisms laid the groundwork for more complex applications of learning. In this process, learning evolved from a tool to support navigation to the central function of the brain. #### Vertebrates and more sophisticated brains During the Cambrian period, which lasted from 540 to 485 million years ago, the rate of diversification of life forms increased rapidly.^[Feldman Barrett, 2020] The jawless fish emerged during this period as the earliest known **vertebrate**, with a vertebral column or backbone as the defining feature. Vertebrates developed a more complex brain with **distinct structures**: the cortex, basal ganglia, thalamus, hypothalamus, midbrain, and hindbrain. These structures served different functions, which allowed these animals to process information more effectively and interact with their environment in more complex ways.^[Howard-Jones, 2018] **Reinforcement learning** is the ability to learn through **trial and error**. It emerged in early vertebrates and is shared among their descendants today. Reinforcement learning involves taking random actions and adjusting future actions based on **valence outcomes**: Positive valance reinforces recently performed actions, whereas negative valance un-reinforces or "punishes" them. A key challenge in reinforcement learning is **temporal credit assignment**, which involves linking actions to consequences that are temporally separated. To this end, **dopamine** was repurposed from a signal that good things were nearby in early bilaterians to a signal for temporal difference learning in early vertebrates to **predict future rewards and punishments**. The **basal ganglia**, a brain structure in vertebrates, plays a crucial role in reinforcement learning. It learns to repeat actions that maximize dopamine release. The basal ganglia works in conjunction with the **hypothalamus**, which houses valence neurons and determines actual rewards. Initially, the basal ganglia learns from the hypothalamus but eventually develops its own judgment, shifting dopamine responses toward predictive cues. %%To what extent is reinforcement learning as described here the same as [[Instrumental (or operant) conditioning involves learning the relationship between a response and the consequence that follows it|instrumental conditioning]]?%% — *Due to time constraints, I had to stop at this point. Hopefully I'll get to complete this note at a later point.* %% Notes from [[Evolution of the Learning Brain]]: **Vertebrates** developed **distinct learning systems** served by different brain regions.^[Howard-Jones, 2018] These systems included sub-cortical regions like the thalamus and striatum, which play crucial roles in various types of learning, from basic sensory processing to complex motivational and emotional responses. The cerebral cortex, especially in mammals, became a grand library of the brain, organizing information for future use. **Primates**, including humans, evolved sophisticated **social learning abilities**, allowing them to learn from the behavior of others.^[Howard-Jones, 2018] These abilities facilitated cultural transmission and cooperation. With the emergence of **Homo sapiens**, learning took on new dimensions. The flexibility in response to environmental variability and the development of social competencies, such as cooperation and negotiation, became paramount.^[Howard-Jones, 2018] These skills were essential for the survival of Homo sapiens, driving the expansion of the brain and the enhancement of general intelligence. ==Add and edit based on [[The Secret of Our Success]]== **Language and culture** played pivotal roles in the evolution of learning in Homo sapiens.^[Howard-Jones, 2018] The ability to transmit abstract ideas using symbols and the development of material culture and symbolic language enhanced the efficiency of social learning, propelling cultural evolution. This cultural intelligence hypothesis suggests that social learning, rather than mere social behavior, was key to the evolution of the genus Homo. ==Add and edit based on [[The Secret of Our Success]]== ### Research strategy ### Sources - Main sources: - [[Evolution of the Learning Brain]] - [[Birth of Intelligence]] - [[The Secret of Our Success]] - [[A (Very) Short History of Life on Earth]] - [[Brain Plasticity and Human Evolution]] - https://www.cambridge.org/core/journals/biological-reviews/article/abs/evolution-of-learning/73DAA46F16EFFD46F55CBB5CDBD06B60 - https://www.sciencedirect.com/science/article/pii/B9781483214481500147 - https://psycnet.apa.org/record/2002-00351-011 - https://books.google.ch/books?hl=en&lr=&id=yBtRzBilw1MC&oi=fnd&pg=PA3&dq=the+evolution+of+learning&ots=2N2yWt93o_&sig=pPOPXbijR3pN_pOPKFEJ5167jJ0#v=onepage&q=the%20evolution%20of%20learning&f=false - https://www.sciencedirect.com/science/article/abs/pii/S0065345408600467 - https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-1428-6_302 - https://www.routledge.com/Evolution-of-the-Learning-Brain-Or-How-You-Got-To-Be-So-Smart/Howard-Jones/p/book/9781138824461# - https://bmcecolevol.biomedcentral.com/articles/10.1186/s12862-017-0889-z ### Notes - This is an example of the fourth of [[Tinbergen's Four Questions]]: What is its evolutionary history? (evolutionary, across time) - #todo I don't know how to distinguish this question from [[What is the evolutionary function of learning?]]. Find out. - #todo search for all notes which reference [[Evolution (Index)]] - Judson Brewer on the Making Sense podcast says that Eric Kandel had gotten the Nobel Price in 2000 for identifying a mechanism for reward learning in sea snails. This suggests that this type of learning is very old from an evolutionary perspective. ### Literature - [ ] ==[[Evolution of the Learning Brain]], chapters 4 ff.== - [ ] *Maybe move these notes to [[What is the evolutionary history of learning?]] at the end.* - [ ] Chapter 4: the social primate - [ ] Social brain hypothesis: There has to be a reason why primates evolved to have bigger brains than other animals. The reason appears to be the increasingly social nature of their niche. As groups of individuals grow larger, the amount of mental processing required increases steadily – e.g., to build complex patterns of social trust. - [ ] Social learning refers to the ability to learn from the actions of others – even when these actions aren't intended to teach or communicate. - [ ] A critical decision for any learner is who to learn from. Key factors include the social status of the other person and the number of people who are already learning from them. - [ ] Social learning uses the same processes that the brain requires for other types of learning. In other words, there is probably no specialized brain module for social learning. Instead, **primates have a bias towards social inputs from their environment** – to pay more attention to the behaviors of others relative to other factors. - [ ] Chapter 5: the cooperative social learner - [ ] At some point in our evolutionary past, we developed a wider and more flexible approach to life, including to sourcing food. This meant greater social cooperation with more decisions to make, increasing the complexity of life and favoring additional brain tissue. In this course, our ancestors evolved greater ability to respond to change. - [ ] Ecological challenges and variability may have contributed to this increasingly flexible and social lifestyle. As a result, the genus Homo carved out an ecological niche based on overcoming challenges with teamwork. - [ ] When environmental conditions were changeable enough, the benefits of increased brain size outweighed its metabolic cost. - [ ] Oxytocin is a neurotransmitter and hormone involved in social bonding in mammals. It leads individuals to pay more attention to the behaviors and needs of other in-group members and to have more positive interactions with them. This can support a more cooperative style of social learning, in which individuals follow the attention of others, which leads to more diffusion of information (or transmission) within the group. In this way, a cultural, non-genetic "ratchet effect" would have begun. - [ ] The combination of general intelligence and social tendencies may have made Homo good at social learning. - [ ] Chapter 6: speech - [ ] An abstract symbol does not depend on context. It can bring to mind a concept that is not related to anything that's present in the current environment. Full abstraction is achieved when the behavior (e.g., vocalized sound or gesture) becomes so strongly associated with an idea that it can bring this idea into another person's mind irrespective of the situation. - [ ] A shared set of symbols whose meaning is culturally inherited across generations had a significant impact on learning. - [ ] Advances in the use of symbols potentially improved learning and transmission of the learning across the population, making it more likely that such learning would be conserved and accumulate. - [ ] Early primitive forms of language would have fostered cooperation and the transfer of information and skills. This would have resulted in a more social world that would in turn have exerted pressure to improve social skills and language, thereby improving language and cultural learning. - [ ] Chapter 8: the emergence of the written word - [ ] Tools such as numeracy and literacy arrived late in our evolutionary history, suggesting that they did not arrive with any genetic adaptation. Instead, they use our existing biology in new ways (cf. [[Neuronal recycling hypothesis]]). - [ ] The demands of our culture have advanced beyond our evolved genetic abilities ot meet them. I.e., there is a cultural-genetic lag. - [ ] ==[[The Secret of Our Success]]== - [ ] *Maybe move these notes to [[What is the evolutionary history of learning?]] at the end.* - [ ] Henrich defines "**culture**" as "the large body of practices, techniques, heuristics, tools, motivations, values, and beliefs that we all acquire while growing up, mostly by learning from other people." - [ ] At some point in our evolutionary past (Henrich suggests probably more than a million years ago), our ancestors crossed an important line whereafter **culture became cumulative**: They started to **learn from each** other in a way that allowed cultural innovations to **improve and aggregate over generations**. With time, this aggregate set of cultural innovations endowed people who had acquired it with significant benefits relative to people who didn't have access to this knowledge. Therefor, natural selection started to favor individuals who were **better cultural learners**. From that point onward, cultural innovations became the primary **source of selective pressure** that shaped the biology of our bodies and brains. In particular, it drove the rapid expansion of our brains, which allowed us to learn and store more locally adaptive, culturally transmitted information. - [ ] The **secret of our species’ success** resides not in the power of our individual minds, but in the **collective brains of our communities**. Our collective brains arise from the **synthesis of our cultural and social natures**—from the fact that we readily learn from others (are cultural) and can, with the right norms, live in large and widely interconnected groups (are social). - [ ] The author distinguishes between two types of learning: **individual learning**, in which the learner learns from experience interacting with their environment, and **social learning**, in which the individual's learning is influenced by others. **Cultural learning** is an important subclass of social learning, in which the individual seeks to acquire information from others, often by making inferences about their preferences, goals, beliefs, or strategies and/or by copying their actions or motor patterns. - [ ] Culture and cultural learning are a psychological adaptation; i.e., a consequence of genetically evolved psychological adaptations for learning from other people. In other words, natural selection favored genes for building brains with abilities to learn from others — to effectively and efficiently acquire information from the minds and behaviors of other people. - [ ] Given the evolutionary context above, how has natural selection shaped our psychology to effectively learn from others? There are several subquestions related to this: - [ ] How does an individual figure out who to learn from? I.e., who possesses information that will increase the learner's survival and reproduction? - [ ] What should an individual attend to and infer? - [ ] When should input from cultural learning overrule an individual's direct experience or instincts? %% --- Topics: - [[Learning (Index)]] - [[Evolution (Index)]] Related notes: - [[What is the evolutionary function of learning?]]