data-informed and not data-driven

Uzma Barlaskar posted an article titled Why you should be data-informed and not data-driven.

“To understand what is the difference between being data driven and data informed, let’s look at an example. Say, you are a news publisher and you are deciding how to frame the headlines of your news reports. You try a few different variants, and find that the one with clickbaity titles gets the most clicks.”

“Data driven decision making: Clicks are up. Visitations (the topline metric for this publisher) are up. Revenue (since they run CPM ads) is up. Great! All our key metrics are up. Let’s ship headings with clickbaity titles.”

“Data informed decision making: All our key metrics are up. That’s good. What are the counter metrics ? Bounce rate is up. That doesn’t seem like a good experience if users are bouncing. What about long term counter metrics? Do we think clickbaity titles are good for our users ? Would it hurt sentiment towards our publishing brand ? Btw, why are click-baity headlines working? Content that piques users’ curiosity does well. Instead of writing click-baity headlines, can we integrate this insight it into our content strategy. Write about topics that people are curious about, but may not know enough.”

“In data driven decision making, data is at the center of the decision making. It’s the primary (and sometimes, the only) input. You rely on data alone to decide the best path forward. In data informed decision making, data is a key input among many other variables. You use the data to build a deeper understanding of what value you are providing to your users.”

“In a data informed culture, you try to understand the behavior that’s behind the data.”

“When you consider data sacrosanct, you are more likely to fall for the fallacy of measuring a lagging indicator.”

“User empathy, design and product sense should always be your guide, along with data, when solving problems for your users. Ask if this solution is providing value to users? Does this fit their mental model ? Does the entire system that this change is part of make sense?”