For example I've been following the development of an online manga reading site (mangadex.org) that is now pushing over 1PB/month of images. If they had built it on aws, even at the lowest rate of $0.02/GB with cloudfront/S3, it would be $20,000 per month. But they ended up paying nothing by using cloudflare who gave them unmetered bandwidth.
(well, until they got throttled earlier this year but they got out of that by upgrading to a measly $200/month plan)
A previous site I built (https://hearthstonejson.com/) is pushing on the order of 100TB / month in images and JSON data, with 96% cached requests and 99.9% cloudflare-cached bandwidth. Almost nothing comes out of S3 itself.
Cloudfront is the ridiculously-priced offering IMO. I don't really understand why people use it.
"2.8 Limitation on Non-HTML Caching
The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Use of the Service for the storage or caching of video (unless purchased separately as a Paid Service) or a disproportionate percentage of pictures, audio files, or other non-HTML content, is prohibited."
"disproportionate percentage" seems wide open
For those of us not in the know, what else would you use it for?
No enterprise contract required. Just for video, obviously.
Even then, just a few terabytes of traffic will make Cloudfront reach the thousands.
This is probably customized per client and could be more for big companies.
Companies that get their highly regulated product(health insurance, hospitals, government contractors) "certified" for use in the AWS ecosystem. Once they do that, they are likely to use it for everything, and pass the cost on to the customer.
There’s also engineering around the network involved in your application. A lot of architectures internally reflect traffic around (moreso if you’re also accepting some ridiculous amount of traffic rather than just serving it). In those cases you can save a lot of money by figuring out ways to pass traffic through unmetered connections like internal albs and amazon hosted dbs. Otherwise you can either rack up a lot of cross-az traffic charges or be on the hook for engineering your way to minimize those costs.
Cloudfront’s pricing can absolutely bite you if you are not careful. Which is one of the reasons why I prefer Cloudflare over it.
OVH Public Cloud offers unmetered bandwidth (250-500 Mbps) in all regions, apart from Asia-Pacific.
Hetzner Cloud offers 20 TB (1 Gbps) for each cloud instance, but has locations only in Europe.
Both of them offer dedicated servers with up to 1-3 Gbps unmetered bandwidth that could be used as exit nodes.
Let's say you spin up 10 instances with OVH at the cheapest level (ie, $200/month). That is supposed to give you dedicated 3Gbps in addition to compute / storage etc or around 1PB data per month + compute and storage.
This compares favorably to google standard egress at $60,000/month. But as soon as you build a business model around this - poof - rate limiting -> some TOS violation claim pops up.
"Oh, we meant unlimited or unmetered, but only if..."
Seriously, with unlimited / unmetered at $20 these would make the great bases of things like CDN networks, image / static asset hosts for big properties etc. But it generally turns out to be total BS.
In contrast - paying for bandwidth with AWS / Google etc -> no one has ever complained to me (though my current usage is minuscule in the distance past had high usage experience)
If bandwidth was at the core of my business model, I would certainly want to pay for it separately to avoid possible interruptions. In case of OVH, I assume, that's what "Bandwidth upgrade" with a "limit of 20 Gbps per customer, and per datacentre" is designed for. It's not unreasonable to consider "spinning up to 10 cheapest cloud instances to avoid bandwidth limits" as a service abuse.
The current price per TB is 1 EUR + VAT if applicable, so even if we assume you'll have to pay the German VAT, it's 1.35 USD/TB.
That's true for Hetzner Cloud, but Hetzner's dedicated servers have been unmetered (with a guarantee of 1 Gbps) since October 2018.
But there’s always a “secret limit”—the point where your bandwidth usage looks like a DDoS attack, and the tier-1 exchange feeding the cloud provider’s DC decides to blackhole your traffic for the sake of the network.
Why not? It's deliberately misleading, no?
That’s useful to me so I can just not worry about that thing.
Am I missing something? I'm not seeing another side to this. Secret fair-use rules are exactly as dishonest as when the telcos lie about 'unlimited' data plans, no?
That's true, European providers work best for Europe.
But how many global locations does a service really need, if it's sitting behind a CDN? Three locations (East Coast, Western Europe, and Singapore) alone are enough to be within 100-150 ms of the most of the world's population.
OVH probably wouldn't be the best choice for projects focused exclusively on South America or China, but it already covers the rest of the world pretty well, including three locations in North America (East Coast, West Coast, and Canada).
I've looked at them to use for my email/small web server and like their prices and features, but am not sure of the GDPR implications.
Currently I use a US cloud provider, and I'm in the US, and so am a controller or processor not established in the Union, and all my data processing takes place outside the Union. All my GDPR obligations, if any, are those that arise under the extraterritorial jurisdiction provision of Article 3(2).
If my server was hosted in the EU by an EU company, would that still be the case? Or would GDPR now apply via the in Union jurisdiction provision of Article 3(1)?
This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to:
a) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or
b)the monitoring of their behaviour as far as their behaviour takes place within the Union.
So it basically says that the GDPR applies, I don't think that hosting in europe would change anything at all.
It can't actually accomplish that goal, because the EU doesn't have the jurisdiction for that.
For controllers and processors that are in the Union, GDPR applies to their processing of personal data of people regardless of where those people are or what entities they are citizens of.
So, for example, as a US citizen residing in the US who has never set foot within about 7000 km of Europe, but has bought things from vendors in the EU, those vendors need to obey GDPR when dealing with my data.
For controller and processors that are not in the Union, the EU lacks the authority to enforce such a broad requirement on them. Instead, the requirement is that if the person whose data you are processing is "in the Union" and you are offering goods and services to them or monitoring their behavior as far as their behavior takes place within the Union, GDPR applies.
(Whether or not they can actually enforce that is still an open question).
Putting this all together, if I'm in the US, with users in the US, but having my server in the EU makes me count as being in the EU for GDPR purposes, then I have to obey GDPR when dealing with US users. If having my server in the EU doesn't do this, so that for GDPR purposes I'm in the US, then GDPR does not apply to my dealings with people in the US.
Yes, it was not appropriate for situations where you had to pay for upload bandwidth. We were very careful in designing the gui to disclose the fact that it would use your upload bandwidth, explain the possible costs and benefits, let users opt-in if that was ok with them, monitor the download status, and switch the BitTorrent feature on and off.
TomTom real time traffic prediction system depends on users trusting them enough to opt-in to uploading anonymized trip measurements, so it was very important not to do something that broke their trust by abusing their network connection.
The first thing I tried was to make an XP/COM plugin out of the LibTorrent library. That worked ok, but the idea of increasing the size and complexity of what was already essentially a web browser with a whole bunch more complex code that does lots of memory allocation and networking all the time, didn't seem like a good design. This was long before Firefox supported multiple processes, so the same app handing bittorrent would bloat the app and degrade the user interface responsiveness.
However RedSwoosh, FoxTorrent and BitTorrent DNA all ran in separate processes that just handled all the BitTorrent stuff, and that you could talk to via https (to stream the file with a special url, and retrieve progress telemetry on another url). And it's incredibly easy to integrate xulrunner or any web browser with those servers via http, so no C++ XP/COM plugin required.
Another issue is that you don't want every application to have its own BitTorrent client built in, or you will trash the network and disk, since otherwise they would compete for resources. It needs to be a centralized a system service, shared by any app that needs it.
BitTorrent DNA worked nicely that way. And it could fall back to the CDN to download at full speed if there weren't enough seeders.
P2P-with-failover-to-CDN is what Spotify famously used to do.
But as far as I know, kissmanga.com is still abusing blogspot.com/tumblr.com by using them as free image cdns.
i thought this was obvious to everybody: public clouds want to lock you in, so they make it very cheap and easy to get data in, but charge stupendous amounts for getting your data out.