On: Residential proxies detected on White House network

[cpburns2009]

This isn't a theoretical problem either. My employer's website was being hammered by residential proxies a month ago. It wasn't until we setup WAF captcha rules that they mostly subsided. It wasn't even the typical exploit scanner. I don't know what the purpose was but it was effectively a DDoS.

[cowlevel → cpburns2009]

Everyone's is. Far as anyone can tell, it's probably AI companies.

[cpburns2009 → cowlevel]

It wasn't any sensible AI scraper. It kept requesting the same pages a thousand times over.

[cowlevel → cpburns2009]

Yep that's what AI scrapers do

[cpburns2009 → cowlevel]

I thought AI scrapers were generally a little more intelligent than let's spam the same request 1k times (without any obvious permutations).

[cowlevel → cpburns2009]

Nope

[mrguyorama → cpburns2009]

These so called "Best of the best industry experts" don't understand what "Air gapped" means, and still don't even after 3 perfect examples.

They do not respect prior knowledge in any way, and think it has no value.

Of course they don't do standard, table stakes things like making their crawler smart. They don't have to. They're better than that or something.