Toxic Optimization: Too Much of a Good Thing?

Modern tech companies have become obsessed with the optimization of growth and profits, and this can lead to worse products and undesirable societal impacts. This trend has been enabled and accelerated by the massive increase of data at companies’ fingertips. There are now endless metrics about customer behavior and how customers search for and use products, and the product manager is often charged with mining this data to maximize the financial value of their product.

This quest for optimization also applies to our own lives. In recent years, many of us have tapped into the endless data of the internet in the fruitless pursuit of perfection. Find the perfect restaurant, buy the perfect vacuum cleaner, plan the perfect 3-day weekend. It may benefit both companies and ourselves to take a step back, reconsider what we are actually optimizing for, and allow more room for instinct our decision making.

We all want things to be as good/fast/fun/easy/lucrative as possible, but there are always trade-offs. It would be quicker to get into a car and drive without bothering with a seatbelt, yet most of us take the extra time to buckle up in order to ensure we arrive safely. Likewise, companies should balance profit maximization with customer satisfaction and other non-monetary factors.

How can being data-driven be a bad thing? Isn’t this akin to the scientific method? Following the data is indeed sensible, but only if you are quantifying everything that matters. What you choose to focus on is of great importance. In recent years, this hyper-optimization by companies is done using computers and algorithms. However, algorithms only work with quantifiable metrics, which means less tangible, real-world concerns are easily ignored.

For example, a company selling a product on a website can leverage usage data to figure out exactly what color button gets clicked on the most, or what text “converts” more browsers into buyers. This makes sense, albeit to an extent. After all, you want to build a digital store that gets as many visitors as possible to complete the transaction. But what if the optimization analysis tells you that deceptive text leads to more sales? It might be “optimal” from the website’s point-of-view to “trick” customers into buying things, but this doesn’t mean companies should do it. It’s not ethical to deceive customers, and it could be ruinous for the long term perception of your company.

Sometimes, optimization on a massive scale can lead to far worse outcomes than customers buying things they don’t want or need. If you apply computational optimization to maximize, say, advertising revenue, you get search engines and social media networks that optimize for “user engagement.” This means that they aren’t optimizing for their stated purpose (connecting users with the world’s information and with other people). This generates massive wealth for those services, but at what societal costs? One only has to look at recent headlines to see how over-engagement with social media can sow division and negatively impact mental health.

How can companies do better? Start by considering what you are optimizing for. Is this goal a worthy one? What are the potential downsides? What unintended consequences do you need to consider? The sociologist William Bruce Cameron said “Not everything that can be counted counts, and not everything that counts can be counted.” In other words, don’t only substitute what is measurable for what is meaningful or important. 

I’ve experienced this myself at companies both large and small: faced with do-or-die short-term financial targets, it’s easy to lose sight of the customer and the big picture. Real-world financial pressures do exist: Wall Street expectations, investor demands, and simply being able to pay your employees and cover your expenses. Companies are in business to make money, but you can still establish guiding principals and “guardrails” to prevent “toxic” optimization.

Companies and managers should draw lines in the sand that they won’t cross. For example: you should never purposely obfuscate costs to customers. Similarly, you should never create a user interface that is deliberately designed to get a user to do something that they do not want to do (so called “dark patterns”). These deceptive practices might be profitable in the short run, but may be damaging to your brand and reputation in the end.

Product management is challenging. You are simultaneously trying to maximize profits while advocating for the customer. It’s difficult, but you can strive to balance company financial targets with doing right by the customer. Try to match customers with products that will provide them with the most value (as those matches will likely maximize your profits as well). If you don’t feel like you can meet the company’s financial requirements while truly advocating for the customer, then that company may not be the right place for you.

By all means, leverage the incredible data available to better understand your customers, their behaviors, and their desires. This data can help improve your product, get it in the hands of more customers, and generate greater profits for your company. However don’t forget to consider your gut instincts and the best interests of your customers; this will often illuminate a slightly different path than the data alone.

See my other article in this series: "Masked by Mastery: The Curse of Knowledge"