Jesse Frederik and Maurits Martijn wrote an article for The Correspondent (originally in Dutch) titled The new dot com bubble is here: it’s called online advertising.
« The story that emerged from these conversations is about much more than just online advertising. It’s about a market of a quarter of a trillion dollars governed by irrationality. It’s about knowables, about how even the biggest data sets don’t always provide insight. It’s about organisations and why they are so hard to change. And it’s about us, and how easy we are to manipulate. »
« When Tadelis pressed them he realised that “proprietary transformation functions” was only a clever disguise for your garden-variety statistics. You take the weekly expenditure on ads, combine it with the weekly sales, and voila! Fold the mixture into a scatter plot and see what happens. Easy as that! »
« “This is garbage,” Tadelis thought. Correlation, as any Statistics 101 class will inform you, is not causation. What do these impressive numbers mean if the people who see your ad are the exact same people who were going to use eBay anyway? … Economists refer to this as a “selection effect.” It is crucial for advertisers to distinguish such a selection effect (people see your ad, but were already going to click, buy, register, or download) from the advertising effect (people see your ad, and that’s why they start clicking, buying, registering, downloading). Tadelis asked how exactly the consultants made this distinction. »
« Together with his team, he carefully analysed the effects of the ad stop. Three months later, the results were clear: all the traffic that had previously come from paid links was now coming in through ordinary links. Tadelis had been right all along. Annually, eBay was burning a good $20m on ads targeting the keyword ‘eBay’. »
« The economist was given a free hand: he was permitted to halt all of eBay’s ads on Google for three months throughout a third of the United States. Not just those for the brand’s own name, but also those targeted to match simple keywords like “shoes”, “shirts” and “glassware”… The experiment continued for another eight weeks. What was the effect of pulling the ads? Almost none. For every dollar eBay spent on search advertising, they lost roughly 63 cents, according to Tadelis’s calculations. »
« Ebay lost 63 cents on every dollar they put into Google search advertising, but that’s actually an imprecise estimate. If the experiment were to be replicated infinitely (and another ad stop, and another ad stop, and another ad stop…), in 95% of all ad stops the loss would fall in the range of negative $1.24 and negative $0.03. This is what statisticians call the confidence interval. In advertising research, the confidence interval tends to be huge. »
« Now we arrive at perhaps the most fundamental question: what, in the end, is there really to know in advertising? Can advertisers ever know exactly what their ad brings in? »
« Advertising rationally, the way it’s described in economic textbooks, is unattainable. Then how do advertisers know what they ought to pay for ads?
“Yeah, basically they don’t know,” Lewis said in one of those throw-away clauses that kept running through my head for days after. »
« When these experiments showed that ads were utterly pointless, advertisers were not bothered in the slightest. They charged gaily ahead, buying ad after ad. Even when they knew, or could have known, that their ad campaigns were not very profitable, it had no impact on how they behaved. »
« Perhaps what’s driving this phenomenon is something much more profound. Something that applies not just to advertising. “There is a fear that saying ‘I don’t know’ amounts to an admission of incompetence,” Tadelis observed… For that reason, companies hire overconfident people who act like they know what they cannot possibly know. »