I block all Amazon AWS/EC2 on my servers because it's never humans and I've yet to see a useful bot from there - they just suck bandwidth and cpu time. Since they have free, unlimited inbound, there's a bunch of nonsense going on.
Now I suspect silk is going to use the same IP range as amazon aws, so if you block aws, you block silk?
So no more using iptables to stop the traffic - maybe I can do it on another layer, allowing the ip via user-agent but of course bots will start spoofing that too.
Anyone know if silk will cache and serve content that is not fresh while ignoring no-cache headers?
Bonus points if anyone as access to a Fire and can test the ip range and header obedience (as well as pre-fetching aggressiveness).
Good question and I haven't seen a response yet. This may be a serious problem for Amazon since you aren't alone in your blocking of AWS/EC2 IP ranges. If users use the tablet and find that the Silk browser doesn't show them all the sites they want, then people will be disappointed in Amazon. Hopefully they'll separate/segment IPs to prevent users from not using the browser.
More likely, if AWS fails to fetch a page it will fallback on the browser to do the work. So the user will still get to the page they want, and your site will be blamed for being "slow" due to blocking AWS caching.
Why wouldn't they assume that? Amazon surely knows that some people block AWS servers (AWS customers sure know!).
You also have to realize that Amazon gets the advantage of crowd sourcing--in short order they should know who's blocking what and not even have to make the original requests to be blocked.
Not only that, but what's to prevent them from caching content after the first client makes a successful direct connection? After that, they can serve the cached content quickly to other Silk users. Circumnavigating such blocks may even be one of the driving forces behind developing Silk in the first place, since it allows Amazon to access content they can't easily reach otherwise.
I also block AWS/EC2 on my servers because it's a relentless source of badly behaved bots, freeloading SEO scrapers, and poorly misguided startup ideas (no, I don't want you to proxy SSL connections for my users--just think about it for a second, for crying out loud).
Now with Fire & Silk in the mix, how do you even go about protecting your content from Amazon, if they can get it directly from the browser, cache it, and make it available to their own paying customers (who might have been blocked by your firewall, for whatever reason)?
I can accept that since it's technically possible, they'll just do it, and resistance is futile. But if you want to locate bad netizens, AWS/EC2 is an excellent starting point.
Remember, this isn't just about publicly-available content, it undermines the entire trust model of TLS/SSL (and exposes some of its underlying weaknesses, hopefully spurring development of a better solution in the long run).
Silk is supposed to be dynamic and work on AWS when it helps and locally when it doesn't. In your case I'd assume Silk users would have a slower, but successful, time using your site.
Perhaps they'll still make it easy to identify so that you can give Silk users the best experience possible, but there's no way it's just not going to work.
If the Silk servers add an an X-Forwarded-For header to the request, you might be able to do it that way. I'd be pretty disappointed if they don't, actually.
I block all Amazon AWS/EC2 on my servers because it's never humans and I've yet to see a useful bot from there - they just suck bandwidth and cpu time. Since they have free, unlimited inbound, there's a bunch of nonsense going on.
Now I suspect silk is going to use the same IP range as amazon aws, so if you block aws, you block silk?
So no more using iptables to stop the traffic - maybe I can do it on another layer, allowing the ip via user-agent but of course bots will start spoofing that too.
Anyone know if silk will cache and serve content that is not fresh while ignoring no-cache headers?
Bonus points if anyone as access to a Fire and can test the ip range and header obedience (as well as pre-fetching aggressiveness).