
Advertisers scramble as ‘non-human traffic’ eats up online budgets - pastycrinkles
http://www.theglobeandmail.com/report-on-business/industry-news/marketing/advertisers-scramble-as-non-human-traffic-gets-better-at-eating-up-online-budgets/article19965043/?page=all
======
dangrossman
Back in July, hundreds (thousands?) of sites noticed a surge in traffic in
their analytics. Hundreds to thousands of additional visits per day above
their normal traffic rates. Curiously, all this traffic identified as Internet
Explorer 7 on various version of Windows, but each visit came from a different
residential IP, and viewed only one page per session.

Eventually the common thread between all these sites was identified -- they
had retargeting tags from sites like AdRoll:
[http://www.thesempost.com/adroll-retargeting-bot-attack-
behi...](http://www.thesempost.com/adroll-retargeting-bot-attack-behind-
ie7-traffic-surges/)

For months now, huge botnets have been producing huge amounts of fake web
traffic, picking up retargeting cookies in order to later be served ads and
have whoever's behind the network paid for those views. That surge has never
ended. It's only grown (including becoming less easy to spot), yet nobody
seems to be talking about it anymore, including the retargeting companies.

I'd love to know who's behind it and how they control so many computers.

~~~
throwaway20148
Sometimes this stuff isn't even bot traffic, it's traffic purchased from
someone who fulfills by incentivizing people who play online games with
virtual currency for viewing videos or clicking through to sites that serve
the buyers ads. I presume they've also got into the retargeting capture game,
too.

~~~
JeremyMorgan
Yup. Many facebook games have a "watch this video" or "visit these sites" in
an iframe in exchange for extra points, more seeds, a better pool stick, etc.
Just as useless as bot traffic.

------
vinbreau
I envision a future where a significant share of the advertising market is
comprised of automated ad-bots pitching services and wares to automated web-
crawlers. There's something obtusely philosophical about that and what it says
about us, though I am not sure what that is.

~~~
arbitrage
William Gibson could probably give you some ideas on what that says about us.
He started writing about this fictionally decades ago.

~~~
wernercolangelo
Or Stanislaw Lem

~~~
readerrrr
Really? Could you share the titles.

~~~
wpietri
I'd say start with The Cyberiad. It's fun and approachable.

------
arbuge
I've been in the online advertising world for 10 years and it still befuddles
me to see that most ad dollars still haven't switched to cost-per-action
(CPA), i.e. only paying for actual conversions. One big benefit of advertising
online after all is that tracking conversions and their referrers is much
easier than with offline.

~~~
eli
CPA simply doesn't make sense for all ad campaigns. When Lexus buys ads on a
financial news site it's for branding -- they want to get their logo in front
of their target demographic. The ads aren't optimized for clicks or leads --
sometimes there isn't even a clear call to action.

~~~
hyperpape
Yup, but then why bother with CPM? Just buy display ads for a fixed price in
publications they believe their customers might read. Sure, it's not an
improvement over what they did in the 50s but they don't have to worry about
fraud.

~~~
brandnewlow
Those ads are freakishly expensive. Like 10x more expensive than buying ads on
FB/Google/Twitter.

~~~
eli
My very biased opinion is that you get what you pay for.

~~~
brandnewlow
In that case you're paying the commissions on the expensive sales people
needed to take your order over the phone.

------
etchalon
"Half the money I spend on advertising is wasted; the trouble is I don't know
which half."

[http://en.wikipedia.org/wiki/John_Wanamaker](http://en.wikipedia.org/wiki/John_Wanamaker)

~~~
solistice
Whilst that quote is about targeting and not fraud, it fits rather nicely.
It's actually attributed to a number of people.

------
roymurdock
Advertising in its current form is broken.

I was recently in a class where we performed a case study on Google. The
professor asked if anyone had ever clicked on an ad. One kid out of 60 had
clicked on ad for cheap protein supplements, but hadn't bought any. Everyone
else scoffed, pointed out their use of AdBlock, and cited their gullible
parents as the only people they knew who might ever possibly accidentally
click on an ad. Furthermore, the strong majority of our class viewed targeted
ads on Facebook and Gmail as extremely creepy and unconvincing.

There are two ad models that work in my opinion:

1\. Native advertising: Ads disguised as useful content. This works for
changing people's perceptions, especially if its done through official or
generally impartial channels, such as news outlets. People think they're
getting honest information about BP's charitable acts and Coke's health
benefits, but they're really being lied to. This doesn't generate any direct
sales, but it can really influence consumer sentiment on different brands
which can have a massive impact on sales indirectly. This method is extremely
sneaky and unethical, and that's probably why it works.

2\. Snapchat's new model: Opt-in, high quality ads. I liken these ads to movie
trailers. They are well-crafted and curated, and you only have to watch them
if you want to. This way, people who click on them are already intrigued and
have committed to watching the ad, making for a more engaged and curious user
base. This method is still being developed but I can see it become big in the
next couple of years.

~~~
Silhouette
_Everyone else scoffed, pointed out their use of AdBlock_

If your entire class was running AdBlock, then your sample is _far_ from
representative of the general population.

 _There are two ad models that work in my opinion_

The thing is, opinions don't matter in this game, facts do. Do you really
think major brands like Coca Cola or McDonalds or Ford spend such astronomical
sums of money on advertising without being pretty good at knowing what returns
they get?

You might think that no-one really buys a car because of the product placement
where the successful business owner drives that model in their favourite TV
show, but you'd be wrong.

~~~
thaumasiotes
> You might think that no-one really buys a car because of the product
> placement where the successful business owner drives that model in their
> favourite TV show

That would be an unusual viewpoint. People love identifying with their heroes
-- I don't think anyone considers that a secret.

Related: [http://basicinstructions.net/basic-
instructions/2014/10/26/h...](http://basicinstructions.net/basic-
instructions/2014/10/26/how-to-explain-why-you-want-what-you-want.html)

~~~
Silhouette
_That would be an unusual viewpoint. People love identifying with their
heroes_

Of course. My point is simply that there are more tried and testing
advertising models that work than just the two that exist in roymurdock's
opinion.

------
goshx
Besides the non-human traffic, most online ads are created to not be seen. I
was even thinking about creating a blog post to call out the marketers out
there to change this, but I am too lazy... specially to make them make more
money. I am so used to how ads that cover the entire page work that I manage
to pass by them without even knowing which brand was advertising there.

Ads that force you to watch a video for X seconds, but in those X seconds they
don't even show a logo or something, are money wasted indeed, again, IMHO.

Another example, ads that follow me. Oh god how I hate these. I visit a
website, usually for a product that I already own, and the freaking ad from
that company follows me wherever I go. This is money wasted.

Another example, I use Waze almost everyday to figure out the best route for
my comute. My 4" screen phone sits there right on top of the radio. Whenever I
stop the car, an ad pops up. But they make this ad so tiny that if I need to
know what brand or product it is for, I have to pick my phone and look closer.
This is money wasted.

The online advertising system is pretty broken... can I fix it?

~~~
chadgeidel
I will never understand the YouTube ads that show 3-4 seconds of black screen
and then "allow" (not sure what controls this) the "skip this ad" after 5
seconds.

You (the advertiser) have to capture my interest in the first 10-20 frames of
video, and pitch me in 5 seconds. Anything else is a waste of your money.

~~~
kefka
And YouTube only has one chance as well. He fact I'm seeing this annoying
video ad means I have already installed Ad block edge and haven't reloaded the
page.

Your move.

------
cvburgess
I wouldn't be surprised to find that the larger firms are crawling their own
ad platforms. When ads comprise 90% of your revenue, it's hard to imagine that
someone hasn't gamed the system.

------
justcommenting
there's a certain irony when advertisers--whose intention is to manipulate the
preferences/behaviors/purchases of consumers--cry foul about their
manipulation systems being manipulated

------
aruggirello
I have an (unfinished) anti-bot PHP project that does customizable automated
bot tests and blocking (both via 403 forbidden, and iptables). A custom
version of this script is running at one e-commerce site since 2012. The 403
page has a "ticket ID" so the (very few) real users caught can still contact
the website admins to be unblocked.

The (almost feature-complete) version running at the site above was able to
automatically identify and successfully block a number of bots found scraping
(or, attempting to scrape) the site since 2012. Though it wasn't an easy task,
it has been quite successful at blocking most of them since their second
(often first) request.

In fact since the beginning, and largely upto now, most bots and botnets can
still be easily spotted. Some by their user-agent, some falling into bot traps
("honeypots") which are invisible to regular users, some failing to load
images (something that a regular user wouldn't do), some crawling the site at
an unreasonable pace... most usually don't provide an http referer, some even
appear to have a number of cookies set for your domain.

It's not always easy to spot them though - some botnets can only be undone by
blocking their full IP subnet, or permanently banning their (sub)domains, like
.cn, .ua or .br (if you don't have legitimate traffic from those domains).

Unfortunately, I haven't had enough spare time to complete my project (which
tests all of the above conditions to spot bots, but is not ready for a
release), but I could if there is some market - or I could sell the project if
somebody is interested.

~~~
jacquesm
It's not hard to make a 70% effective bot blocker. It's _extremely hard_ to
make a 95%+ effective bot blocker without having a large number of false
positives.

It is (probably) impossible to make a 99%+ effective bot blocker without
having a very large number of false positives.

I've been working on this (and abusive user blocking) for a long time now and
I'm roughly where you are, I have a product, it _could_ be launched but I'm
very wary of launching a half baked product that would eventually end up being
disabled. I've had the help of a friendly corporate entity that gave me
millions of datapoints to play with allowing me to look into their kitchen
both on the support end and on the incoming traffic end as well as verified
abusers.

This identified various fraud such as ad click fraud, affiliate scams,
repeated harassment by the same users under different names, spam of all kinds
and so on. It's been working there for more than 4 months on a fairly high
level of confidence. And yet, I'm still wary of rolling it out on a larger
scale.

------
Animats
So now 25% to 50% of impressions go to ad bots. It's known that most ad clicks
from about 10% of human users, who will click on anything but buy little.

The real winner in this is Amazon. Amazon doesn't pay anybody when someone
clicks on a product listing. They have to buy. Same for eBay. So there's no
point in 'bots clicking on those sites.

Online ad rates have been dropping for some time now. Most of the ads run by
Google Adsense (the ones on third party sites) are there because Google pushes
advertisers hard to use that feature. Most of the conversions from Google Ads
come from the ones on search results, which are displayed when the user is
looking for something. Those are useful. Most other forms of advertising are
interruptions for users interested in something else.

The classic tactic ad-based companies try as ad prices drop is increasing the
ad density. That's called "pulling a Myspace". It didn't work for Myspace,
which tanked. Twitter seems to be headed that way - usage up, profits not too
good.

The way this is going, online advertising is going to have the conversion (to
a sale) rate of spam - one in thousands.

------
jbob2000
"Wah, I can't advertise using old methods and might have to come up with
something new. Waaah"

~~~
nkozyra
While I understand the downvotes, the sentiment here is valid in the sense
that there's been so little innovation in advertisement for the web - even
mobile ads are anachronistic.

The entirety of innovation has been sizes & styles and tracking and not on
presentation and usability that's complementary with content.

It's not a 'you made your bed' situation, but not wholly removed from it,
either.

------
blatherard
There's a lot of room for innovation in this space. Some friends (well,
clients) of mine have a startup (Unlockable,
[https://www.unlockable.com/](https://www.unlockable.com/)) that embeds
advertising-based games into other games. The level of interactivity is way
more than a bot could handle.

~~~
goshx
So are bots playing games as well? I mean, are they also having ad fraud
inside games?

------
SixSigma
> Do you hate how easy it is for your computer to become slow and riddled with
> malware?

No, I can't say I do. If such crime is so rampant, one would think it was ripe
for law enforcement to do more about it. I wonder if reporting it to the
police every time one visits a relative and removes crap from their computer
would start a disruption.

~~~
andybak
Do pop back here and let us know how that works out for you.

~~~
SixSigma
Excellent sarcasm. Did mummy help you?

------
perlgeek
There are two pretty obvious approaches: only pay per conversion, or accept
that not all views are human.

Just like when you advertise in a classical news paper, you only know how many
copies are typically sold, not how many humans actually look at your ad.

~~~
shostack
If only CPA were that simple.

Case in point...

\- You work with 3 CPA networks

\- Each has their own tracking tag that their terms require payment be based
off of

\- A conversion comes in and each network registered an impression (or heck,
even a click) before the visitor in question signed up.

Who gets the credit?

Well, if you don't have an agreement that states billing is done off of
numbers from YOUR ad server, where you can dedupe and set an attribution model
that makes sense for your particular business, the answer is that all of them
will.

So you are paying 3x when realistically odds are that not all of them should
get full credit. Hell, that doesn't even factor in other channels like
organic, social, etc. that all likely played a role as well.

------
tn13
This is a problem I have seen first hand. But this is not really as big a
problem as it is made out to be. The real issue here is that it is hard to
predict of the given impression is a bot impression or a genuine impression.
As a consequence any CPM based dealing is difficult. Difficult only means
falling CPMs. That is what I have observed.

The advertisers will not have to put in more efforts to measure conversions
and accordingly bid for the traffic.

~~~
mianos
I am also working in ad yield analytics, so while I wait for a DFP report to
run, I'll also say, it's also in the network's interest to address this issue.
In the meantime we'll just see more CPA line items and a drop in price for CPD
and CPM. Being an auction, I'm sure Appnexus already sees this. When the issue
is addressed we'll see a rise in CPM that reflects the increase in value with
less bots. There is money in fixing it so I'm not calling the end of the on-
line advertising world like the article, which is written by a general
reporter not someone in the field (Cost Per Mile? oh really?)

~~~
shostack
Just wait until cross-channel attribution (particularly dynamic attribution
vs. fixed model which is more commonly available) really takes off. CPA-based
buys are going to come under a LOT more scrutiny when Joe Advertiser suddenly
realizes that his retargeting campaign didn't actually add a ton of value to
the path for the vast majority of his conversions, especially if he's giving
<gasp> 100% credit to view-throughs.

~~~
brandnewlow
It's funny how much business we've won just by making it easy for clients to
take control over their view through attribution settings and counting.

~~~
shostack
That's awesome that PerfectAudience does that, and more and more DSPs are
definitely waking up to that as well which is great.

What do you do to help inform what the settings should be? Unless an
advertiser has a solution like VisualIQ, Convertro, Adometry, etc. (and
they've all been bought up so who knows what will happen to them), or GA
Premium, they likely don't have a dynamic attribution solution in place to
help inform them as to what those values should be and how they are shifting
over time (because in reality those weights are anything but static).

I really wish DSPs did more in the way of thought leadership on how to
properly value view-throughs and other ways of measuring contribution as view-
throughs are the majority of what they drive, and that metric is coming under
increasing scrutiny (as it should).

------
userbinator
This reminds me of
[http://dhowe.github.io/AdNauseam/](http://dhowe.github.io/AdNauseam/) , which
was on HN recently:
[https://news.ycombinator.com/item?id=8515398](https://news.ycombinator.com/item?id=8515398)

------
bthornbury
I've wondered about this before if advertising was just designed to be a big
negative sum game for advertisers.

Interestingly if these same bots were capable of making real purchases and
were run by ad companies this behavior may be sustainable because some people
are making money. Almost like wall street.

~~~
meric
Assuming an average consumer will spend a constant number of dollars per year
- advertising would indeed be a big negative sum for advertisers. Advertisers
would make the same number of dollars, except having had to spend on
advertising. The problem is those who do not advertise generally lose out to
those who do.

------
walterbell
WhiteOps is working on ad-fraud detection:
[http://blogs.wsj.com/cmo/tag/whiteops/](http://blogs.wsj.com/cmo/tag/whiteops/)
& [http://www.whiteops.com/](http://www.whiteops.com/)

------
ForHackernews
Good. Maybe they'll stop throwing so much wasted money at Google.

------
rebootthesystem
We stopped all advertising on Facebook. It's a mess.

~~~
jsonne
Really? I've had quite a bit of success with facebook for my clients. What
didn't you like about it?

~~~
rebootthesystem
Fake non-responsive non-engaging likes to pages and more. See this, it's very
true:

[http://www.youtube.com/watch?v=oVfHeWTKjag](http://www.youtube.com/watch?v=oVfHeWTKjag)

~~~
jsonne
I have seen that. That mostly has to do with boost posts though. We only use
custom tailored audiences for our ads and it works quite well. Albeit there's
a lot of data scrubbing.

~~~
rebootthesystem
It really goes beyond that. Facebook wants to own your audience and that is
fundamentally wrong.

Let's say I spend $100K advertising on FB to get people a my FB page or group.
In order to now be able to reach them I have to constantly pay FB through
boosts and further ads.

Why? Because they've installed a mechanism through which they limit the reach
of every single post you make on your page. And, if a percentage of your likes
are fake or non-responsive for various reasons (they liked the picture on your
ad, couldn't care less about your content) it is very likely your posts will
be to a great degree wasted on a useless audience.

In addition to this they have NOT installed a mechanism through which I can
engage with my entire audience. I understand that people don't want their
timeline spammed. At the same time, they liked the page. If they liked Tesla
it is likely they want to hear from Tesla. Facebook provides no mechanism
through which you can do this (message your entire audience) unless you pay
them for every single point of contact on every single message/post.

This, to me, is fundamentally wrong. They don't own my customers. I do. A fan
page or group owner should not have to pay FB over and over again to reach his
or her audience. It's a scam.

In sharp contrast to this, if I spend the same $100K advertising on Google and
get my customers/likes/fans to register on my site I can then build a real
relationship with them over time. If they don't like what I am saying they can
opt out of the emails they receive. If they do, they keep receiving them. And
I don't have to spend tens of thousands of dollars every time I need to
communicate with them.

A more concrete example is a product we tested on FB. It was aimed at medical
professionals. With custom audiences you'd think you can reach a good number
of your prospects. That isn't necessarily true. The heuristics they use can
deliver anyone who, for example, likes pages with medical information. I am
being simplistic. I think you get the point. Still, a good argument could be
made that one could do worst outside FB. In terms of targeting a narrow
audience this might mostly be true depending on the approach taken.

Reaching medical professionals on FB is expensive. Likes can easily cost you
more than a dollar when all is said and done. And, generally speaking, there
isn't a 1 to 1 relationship between ad spend and actions or page likes. So,
you spend $100K to reach medical professionals and you might get 1,000 to
5,000 legitimate likes. In other words, you could actually spend a very real
$20 to $100 per prospect. And then you have to spend money again every single
time you need to reach them. We all know that a marketing message must be
received multiple times before action is produced. You could very easily spend
another $10 per prospect before you get a conversion. That's why I say this is
a huge scam. If I spent $20 to $100 per effective member to a page or group,
from that point forward they should be my members, prospects or customers, not
Facebook's. And extorting money out of me in order to reach all of them with
every post is just plain wrong.

The same scenario plays out far better if you avoid FB, use Google and other
channels to reach your desired audience and bring people to your own
environment for engagement. We've tested this with various audiences and also
had plenty of prior experience before advertising on FB. We can consistently
produce better results out of direct engagement with an email list as small as
200 people than by spending almost any amount of money on FB.

This is why I don't understand when I see TV ads where companies are
subverting their brands to FB by telling people to go to their FB page rather
than their own landing pages. They spend millions of dollars creating and
airing these ads and then they have to go back and shovel more money at FB to
reach the very people they spent so much money to target. That's just insane.
More people doesn't mean better results.

In the end I think it is about a basic business formula: Is there a more cost-
effective and higher-converting approach to reaching customers and building
relationships with them than using FB ads? At the moment my answer to that
question is: Yes. Absolutely. I could be wrong but my experiences so far tell
me otherwise.

