When Personalization Goes Wrong (And How to Fix It)

When Personalization Goes Wrong (And How to Fix It)

Does this sound familiar? 

You head to your favorite eCommerce retailer to do some shopping, and you find  something you’d like to buy. For this example, let’s say that item is a green shirt. 

You purchase the shirt, and all is seemingly well. Right? 

When you wake up the next day, however, something’s not right. 

Green shirts are appearing all over the ads on your favorite social media sites. You’re receiving emails about sales on green shirts from the retailer you purchased yours from. And when you go back to the site to see what’s happening, green products are appearing everywhere. Green pants, green sunglasses, green keychains, and more. 

From the outside, it’s obvious that the retailer made some sort of mistake. Just because you bought one green shirt doesn’t mean you want an entire outfit in green. 

But as the popularity of personalization has grown, retailers have started making the same series of mistakes over and over again: 

Overpersonalization: When Personalization Goes Too Far

The idea behind personalization is great. Customizing your website towards your users’ likes and preferences is sure to give them a better experience. 

Many retailers, however, take the idea and turn the dial up. We call this exaggerated version of personalization “overpersonalization.” And examples are easy to find:

Sure, there’s a chance that the shopper in question might want to start a humidifier collection, but it’s unlikely. 

In a study conducted by the Nielsen Norman Group, one participant reported a similar experience when shopping for toilets: 

“If I look up toilets on Taobao, there will be information about bathroom appliances everywhere. The downside of this is that if I already bought this thing, the ads still keep showing up. It’s annoying, and I’ll feel bad if I see a better deal after I bought it.”

It’s important to note that overpersonalization doesn’t always happen during shopping experiences either. And sometimes it’s just creepy:

Here’s the thing: 

What looks really cool in a personalization vendor’s manually curated demo is often the same thing that looks stupid or creeps customers out when applied to real people. 

Sometimes, even the smartest computers are kind of stupid, and even the savviest merchandizers miss embarrassing edge cases. 

Without constraints, it’s easy for unsophisticated algorithms personalizing at all costs to index too far in any one direction, and it’s even easier when merchandisers are told to do the algorithms’ jobs for them. It’s almost never correct to take small bits of data and make wholesale massive changes to a user’s experience because of them. It looks cool when a personalization vendor clues in on a user’s purchase of a humidifier, but it looks stupid if the algorithm suggests they start a humidifer collection.

(You also don’t get any points for displaying the fact that you know so much information about your customers). 

Over the years, both Constructor and our friends at Simon Data have talked with many prospects and clients about this issue, and in almost every case, a bulk of the issue lies in one place:


There are two unifying characteristics of the examples we shared above: 

For one, they’re all bad. That’s a given. But more importantly, the data used to drive those personalization engines was complicit in some way. Whether it’s over-indexing on too small of a data set or something else, you can’t have a comically bad personalization failure without also having a data failure. 

So, how do you fix this? 

There are two solutions, and first is (somewhat) straightforward: 

Collecting More (And More) Good Data

Irrespective of how you look at this personalization problem, one of the best solutions is simply having an incredible amount of data available — and more importantly, a mechanism to distill that data into its essential elements. The overarching goal is to make your data something that can provide value to your business. 

In reality, data exists on a spectrum: 

All the data that possibly exists

All the data you capture

All the captured data that’s unified and usable

All the data that’s being used

Your goal is to close the gap between each — capturing, connecting, and operationalizing your data. 

The best, most data-driven organizations understand this, and they dedicate themselves to closing the gaps between these steps. 

They’re always looking to capture more data (and to incentivize their customers to make useful disclosures). They’re doing the yeoman’s work of unifying and integrating that data such that it can be operationalized for driving business metrics and improving user experiences. They’re being creative and strategic in their use of all the data they have.

If that sounds like a lot of work, well, it is. 

In fact, both Constructor and Simon Data exist to help businesses solve the massive challenges associated with getting the most out of your data.

But whether you partner with us or tackle things on your own, one fact remains: you’re going to need as much data as possible to support great personalization, or you’re going to need to build an entire business around catering to vintage humidifier collectors and rabid garage door enthusiasts.

Having comprehensive data access is only the first step, though. You also need to be sure of how you’re leveraging it.

Audit-ability and “Magic Solutions”

In late March and early April of 2020, people across the world nearly stopped buying airline tickets altogether. It wasn’t safe to travel, and some governments even restricted it. 

The majority of pricing in the airline industry is set by algorithms, and those algorithms are trained to set prices based on customer demand. If demand goes up, the algorithms will increase prices (& vice versa when demand drops).

There certainly was a drop in demand in this case. The only problem? 

That drop had nothing to do with ticket prices — but the algorithms still ran with what they knew. 

Flight prices nearly bottomed out. And since there was no simple fix, most airlines halted their algorithms completely and began setting prices based on pricing in the past year.

Airlines, at least, can diagnose the problem. Their algorithms are heavily based on historical demand for given flight routes, dates, and times, but the pandemic rendered all of that data unusable. Now airlines must get creative with the data they have left in order to rebuild a fair, flexible pricing structure that works.

But imagine if the airlines didn’t know about that reliance on historical data. What if they had no way to even diagnose the problem?

There’s no doubt about it: algorithms are complex, and they’re fascinating. It’s easy to see them as a “magic solution” to all your problems. But it’s not.

We’ve seen the “magic solution” pitch many times before. A personalization vendor might sell you AI and promise an amazing, cutting-edge algorithm behind the scenes, but don’t let you see it. How do you know it’s actually an algorithm and not just some person manually curating the example? You buy their offering, see no more examples, and begin to suspect there was no algorithm in the first place.

Buying an algorithm that’s all blackbox and promises isn’t much better than buying snake oil. There must be a way to understand what’s happening behind the scenes — because in some cases (especially the one above), an inability to understand and audit what your algorithms are doing can result in serious damage. 


We touched on some of the more focused “dos and don’ts” of personalization in this article, but there’s still one question that needs to be addressed: 

What is the real goal of personalization, and how is it best implemented? 

The retail industry as a whole is in the process of moving from a channel-centric perspective to a customer-centric perspective. 

As a consumer, I think about brands much differently than the way brands currently think about me. The brand is a singular “thing,” with its own unique attributes. I may see their advertisements, read their emails, and like their social media posts across different channels, but each of those things all connect back to the brand itself. 

But how much do your emails, advertisements, social media posts, on-site experiences, etc. connect back to your consumers as individuals? 

In the past, it has been difficult for brands to reciprocate in this way. But that’s the beauty in personalization. 

Personalization allows you to finally think of your customer as an individual who has a complex relationship with the business throughout multiple touchpoints. And the only way to deliver these experiences is if personalization exists throughout all those touchpoints. 

The overarching goal of personalization is to simultaneously increase the business KPIs you care about most, while also turning siloed conversations into one overarching conversation where the things your users do on your website searching, browsing, and interacting with recommendations are connected experiences with what they see in email, on social media, in advertisements, and across your other channels you communicate with them. 

Personalization has the ability to deliver experiences to customers that make them feel understood and tailored to, but only when enough of the right data is collected, and only when the algorithms built on top of that data are simultaneously sophisticated and glass-box. 


Aaron Marsden — Aaron is the Head of Marketing at Constructor, leading all inbound/outbound marketing efforts.

Brian Thompson — Brian is the VP of Strategic Initiatives at Simon Data, an enterprise customer data platform that empowers brands to deliver data-driven, personalized customer experiences.