Personalized Algorithms Linked to Cognitive Errors
It’s the promise of the modern internet. An experience tailored just for you. But new research suggests this very personalization, the feature meant to help us, may actually be making our decision-making worse.
A study published in the Proceedings of the National Academy of Sciences (PNAS) provides strong evidence that when we believe an algorithm is personalized, we become more susceptible to classic cognitive biases. We make poorer choices. The very act of telling a user that a system is “for you” appears to short-circuit the brain’s critical thinking faculties, leading to predictable errors in judgment.
The core of the experiment, conducted by researchers at the University of Konstanz and the University of Deusto, focused on a well-documented cognitive trap. The decoy effect. This bias occurs when a person’s preference for one of two options changes when a third, asymmetrically dominated option—the decoy—is presented. Imagine choosing between a small coffee for $3 and a large for $5. Many might choose the small. But if a medium coffee is introduced for $4.75, the large suddenly looks like a fantastic deal, and sales for it increase. The medium coffee is the decoy; it’s not meant to be chosen, only to make the large coffee look better.
This isn’t just a marketing gimmick. It’s a fundamental flaw in human reasoning. So, the research team wanted to see how algorithms interact with it.
The researchers designed an experiment where participants were asked to choose between different dating profiles. Initially, they chose between two. Then, a third decoy profile was added, specifically designed to make one of the original two (the “target”) seem more attractive.
Here’s the critical part. The user base was split into two groups. One group was told the recommendations came from a “standard algorithm.” The second group, however, was told the choices were presented by a “personalized algorithm” that had supposedly analyzed their previous 30 choices to understand their preferences. In reality, both algorithms were identical and not personalized at all. Only the label changed.
The results were stark. Participants who believed they were interacting with a personalized system were significantly more likely to fall for the decoy effect, choosing the target option that the decoy was designed to promote. The mere suggestion of personalization, the study indicates, was enough to amplify a known cognitive bias.
A Switch to “Blind Trust”
Why would this happen? The researchers, including Helena Matute from the University of Deusto, suggest it has to do with cognitive offloading. When we think an intelligent system understands us, we tend to switch from effortful, analytical thinking to more intuitive, automatic processing. It’s as if our brain decides, “The algorithm has this covered,” and stops scrutinizing the options so carefully.
This creates what the paper describes as “a kind of ‘blind trust’ in the algorithm.” We don’t just trust it to find things we like; we trust its presentation of the choices themselves. The frame becomes more important than the picture. This trust, however, is easily exploited, even unintentionally.
The implications for the digital ecosystem are significant. E-commerce platforms, streaming services, and social media feeds all deploy sophisticated recommender systems designed to maximize specific outcomes, whether it’s a sale, viewing time, or ad clicks. These systems, which are constantly being refined to better predict user behavior, could be leveraging the decoy effect and other biases at a scale impacting billions of users.
A platform trying to sell a specific smartphone, for instance, might not just promote that phone. Instead, its algorithm could learn that showing the target phone alongside a slightly worse but almost-as-expensive decoy phone results in a higher conversion rate. The user, believing the recommendation is a genuine reflection of their “taste profile,” is nudged into a decision that primarily benefits the platform.
This dynamic changes the conversation around algorithmic transparency. It’s not just about knowing *what* data an algorithm uses. This research, led by co-authors Christoph T. Weidemann and Jonas De keersmaecker, suggests we also need to understand *how* the simple framing of that algorithm as “personalized” affects human psychology.
The trust we place in these systems isn’t necessarily earned through performance. It can be manufactured with a single word. And while the experiment used dating profiles, the underlying mechanism of cognitive bias is universal, applying to how we choose our news, our entertainment, and our consumer goods. The study’s most unsettling finding, perhaps, is that a powerful psychological effect was triggered even when the technology itself was a fiction—the algorithm wasn’t actually doing any personalizing at all.
