Inserm Research Director
Inserm Unit 960 Laboratory for Cognitive and Computational Neuroscience
“Human reinforcement learning” team
When we implement complex cognitive processes, for example when making decisions, we are subject to cognitive bias. But what about simpler processes, such as those involved in the most basic learning? In a new study analyzing data from all previous research in the field, researchers from Inserm and ENS-PSL show that not only are positivity and confirmation biases present even in the simplest human and animal cognitive processes, but also that incorporating them into learning algorithms would enhance their performance. This research, published in Trends in Cognitive Sciences, suggests that these biases could initially have been a very old evolutionary advantage.
Cognitive biases, such as positivity and confirmation biases, are known to influence our beliefs and decisions. Until recently, it was assumed that they were specific to so-called “high-level” cognitive processes, namely those that come into play when we reason about complex and uncertain proposals. For example, it is well known that people overestimate the likelihood of desirable events (France winning the World Cup) and underestimate that of undesirable events (a marriage ending in divorce).
In a study published in Trends in Cognitive Sciences, Stefano Palminteri, Inserm researcher at the ENS-PSL/Inserm Laboratory for Cognitive and Computational Neuroscience and Maël Lebreton, researcher at the Paris School of Economics, challenge this conception of the involvement of positivity and confirmation biases.
The researchers drew on the various existing data in the scientific literature on “reinforcement learning”. This is a basic cognitive process of learning through rewards and punishments that humans share with many animals. This literature review reveals that very simple reinforcement learning tests can reveal behavioral signatures of positivity and confirmation biases in people who are subjected to them. These biases appear to be much more common than previously thought and are even present in the simplest cognitive processes such as learning to make good decisions through trial and error (reward and punishment).
What is more, these biases do not seem to be exclusive to humans: the behavioral signatures also appear in similar tests conducted in animals. This suggests that these biases may have emerged during evolution in a common ancestor, long before the appearance of Homo sapiens, raising the question of why evolution has selected and maintained what may at first glance be perceived as processes that can generate seemingly irrational behavior.
Palminteri and Lebreton believe that they have identified part of the answer to this question through the findings of studies based on computerized simulations. These studies compared the performance of reinforcement learning algorithms – with some algorithms incorporating positivity and confirmation biases and others not. These simulations show that the presence of a confirmation bias in the algorithm actually enables it to learn more effectively in a wide range of situations. These biases could therefore actually promote survival, which would explain why they have not been corrected during the course of evolution.
The article opens up new avenues of research that would allow us to deepen our understanding of the cognitive biases and processes related to reinforcement learning. In particular, the researchers suggest exploring the role of these biases in the development and maintenance of pathological states, such as addiction and depression. These findings also suggest that including these biases in artificial intelligence algorithms could paradoxically improve their performance.
Lorsque nous rêvons, nous sommes à première vue coupés du monde, incapables de recevoir des informations de l’environnement et d’y répondre. Pour la première fois, une collaboration entre des chercheurs de l’Inserm, de l’AP-HP, de Sorbonne Université et du CNRS avec plusieurs ...
While great progress has been made in recent years in the understanding of autism spectrum disorder (ASD), its underlying molecular mechanisms remain fairly poorly documented. Several hypotheses have been put forward regarding the possible dysfunction of certain neurotransmitters in the brain, but ...
The computational roots of positivity and confirmation biases in reinforcement-learning
Stefano Palminteri(1,2,3) and Maël Lebreton(4,5,6)
(1) Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et Recherche Médicale, Paris, France
(2) Département d’études cognitives, Ecole Normale Supérieure, Paris, France
(3) Université de Recherche Paris Sciences et Lettres
(4) Paris School of Economics, Paris, France
(5) LabNIC, Department of Fundamental Neurosciences, University of Geneva, Geneva, Swiss
(6) Swiss Center for Affective Science, Geneva, Swiss
Trend in Cognitive Sciences : https://doi.org/10.1016/j.tics.2022.04.005