The author doesn’t provide and data or specific info on what they did, but a few general points:
1. Seasonality. It sounds like they’re comparing a few months on to a few months off, which likely is affected more by the time of year than anything else. Year over year data would be a better comparison.
2. Attribution windows. If I serve you an ad, then “turn off my advertising”, then you convert, depending on how you look at the data my ad either did drive the conversion or did not. Likely he’s just looking at google analytics with default settings, so he’s missing post-view data and assists.
3. It’s not that difficult to run an actual test, and most advertising platforms give you tools and documentation on how to do so.
4. The author doesn’t understand advertising from a technical perspective and most likely is not doing the analysis correctly. The big red flag is when talking about buying a display ad and it only driving 2 conversions. It sounds like a prospecting ad, which is top of funnel and is not supposed to drive post click conversions. I highly doubt they had an impression tracker on the ad and looked at post view data with different attribution models. Or even multichannel funnel reports in ga. Those would show the actual impact of the banner, not just 2 post click conversions.
I don’t mean to bash the guy but this post is rehashed all the time here and I end up reposting the same stuff.
Thanks for the detailed reply. I didn't interpret the parent as bashing, I think that it's good to be critical of these kinds of posts especially when the analysis is shallow and lacks data.
