
Image hoster pushes 1.8 PB per month through Cloudflare CDN - sheraz
http://postimage.org
======
ChrisNorstrom
Yeah unlimited does have it's limita....OMG you're funneling 1.8
petabytes?!?!! That's like 1,800,000 gigabytes or 1,800 terabytes every month.
EVERY MONTH. Unlimited does not mean you can serve the bandwidth of a small
country through 1 ISP.

 _" Please contact us if you have a CDN that is capable and willing of serving
1.8 Petabytes of outgoing traffic per month free of charge," -PostImage_

Is the company run by spoiled entitled millennials? It takes a lot of
equipment and staff to create a network that can serve that and they just want
it all for under $1k? You know data centers have costs too. Other startups
spend years of their lives putting together a team that can assemble
infrastructure to handle that and they want it for free? For their barely-
cash-flow-positive project that doesn't earn them a decent living...

I can't believe they just asked that. Sorry but PostImage sounds like the
biggest moocher in the web right now. If your ISP provider says "unlimited
data" does that mean you can funnel the traffic of Russia through your
internet connection? You want for "free" what others are paying thousands and
thousands of dollars for?

What is wrong with you? Seriously? It's like you're trying to run a top-tier
web company on pocket change.

~~~
spiderfarmer
I agree, but to be fair, I think "unlimited anything" is almost always
misleading, Why not advertise with a generous free tier with clear limits?

~~~
marktangotango
Services offer this because if you don't have any users, it's easier than
building in and enforcing limits. If your service gets traction then it makes
sense to spend the time and money on development, not before. I personally
wasted a lot of time on building limits for a service that has yet to gain
traction.

------
mark242
Equivalent pricing:

Amazon Cloudfront: $59770 per month

Google Cloud CDN: $50000 per month

Fastly: $144200 per month

etc etc. If they can't get together $12000 per month for Cloudflare, they sure
as hell don't have a viable business model.

~~~
zzzcpan
Or like $1000-$1500 per month for a dedicated server with 10G unlimited
bandwidth. This is as low as it gets.

~~~
mark242
Could you run the service off a few OVH dedicated boxes with 3Gbps? Sure, you
could, but it's going to be painful, and at that point imgur is the better
choice anyways.

~~~
zzzcpan
No, it's not painful at all. I think multiple boxes on different ISPs is the
way they should go, but it's going to be more expensive, than a single
cheapest 10G box.

------
codexon
The solution is really simple, don't host images through a CDN. Though it
might not be as fast, it will be way cheaper.

I'm really suprised they didn't look at all the stories of people before
trying to start image or video hosts on Cloudflare and getting booted, as if
all the other image hosts were too stupid to use Cloudflare for $200/month
instead of paying CDNs tens of thousands a month.

~~~
idp
Of course we were puzzled by CloudFlare's pricing structure. However, all of
their advertising material kept reiterating that they were making money off
low-tier customers by collecting lots of data required to properly serve their
business and enterprise customers. So our sense of alarm and disbelief was
somewhat suspended until CF contacted us about our bandwidth consumption, but
even then we didn't realize how much trouble we were facing, as they first
told us that a $1k subscription would do the trick.

Watching competing projects also added some degree of assurance. Without
giving out any specific names, we are aware of a number of competing projects
that even now keep heavily relying on CloudFlare (although I am obviously
unaware of how much they pay), so we didn't consider it an abusive behavior.
In contrast, we were also aware of at least one competitor that kept pushing a
considerable portion of their uploads to imgur and basically parasitizing on
them for a long time until imgur's abuse team finally took notice and brought
down the hammer.

~~~
jgrahamc
_However, all of their advertising material kept reiterating that they were
making money off low-tier customers by collecting lots of data required to
properly serve their business and enterprise customers._

This makes it sound like we are doing some monetization of low-tier customer
data. We are not doing that with any data (low-tier or anything else). That
would be hella creepy.

We do track abuse (DDoS attacks, etc.) whoever they hit and use that
information to protect other customers. So, in that sense low-tier customers
help our overall business, but it's a common lie from our competitors that we
are somehow monetizing traffic. Cloudflare's business is pretty simple: work
out how to operate our services as cheaply as possible, charge web
site/application owners more than that amount.

~~~
idp
Apologies, that was a poor choice of wording on my part. While I do entertain
the theoretical possibility that CF silently uses low-tier customer data in
interesting ways (such as data mining and tracking user behavior), my actual
expectations of the way CF actually uses this data exactly matches your
description (DDoS mitigation & things like that).

------
Namrog84
Cloudflare claims unlimited bandwidth then says you are using too much
bandwidth? Isn't this one of those blatant false advertising claims?

Also it's super unfortunate if they go down breaking so many images hosted so
many places.

~~~
mdasen
Cloudflare's terms of service specifically deal with this: "Using an account
primarily as an online storage space, including the storage or caching of a
disproportionate percentage of pictures, movies, audio files, or other non-
HTML content, is prohibited."

If they put their website behind Cloudflare, but not their image serving
domain, presumably that would be fine. Cloudflare is made for a site that
serves up web pages with some content like your logo, a few images, etc. Those
images can be cached and it's a small amount of data since something like the
YC "Y" image is the same served for everyone on HN. A site like Postimage
serves image content that's hard to cache since it's mostly different for each
visitor.

Cloudflare is pretty explicit that it's not meant for a site that will be
mostly serving up loads of pictures.

~~~
derefr
> Cloudflare is made for a site that serves up web pages with some content
> like your logo, a few images, etc.

For an interesting counter-example: 4chan (including its image-CDN subdomain!)
is served through CloudFlare. It seems that CloudFlare really _is_ willing to
host and distribute multiple petabytes of image content per month, as long as
it's for the sake of something else (e.g. pages of threaded conversation)
that's not an "online storage space."

Though, I think in 4chan's case it might _also_ help that pages and their
image resources both get expired off of the site quite quickly. You can create
a page that hotlinks image URLs from 4chan's CDN subdomain, but the
correspondent resources at those URLs won't be there five hours later, so
there's little point to doing so. Unlike most image-hosts one could name.

------
_ursu
1.8PB/month is an average of 5.48Gb/s. Assuming you can make do with peak
capacity of ~4x average, you can buy 20Gb/s of transit for ~$2k (these guys
for example [https://www.fdcservers.net/ip-
transit.php](https://www.fdcservers.net/ip-transit.php)). You can probably get
a decent deal from somewhere more well known if you ask for a quote, with that
kind of traffic, especially if it get a little publicity.

~~~
dismantlethesun
You'd get something, but it won't be a CloudFlare level CDN.

At best, you can get AdvanceHosters. They're Russian based but have 7 CDN pops
across the US and Western Europe, they'll let you push 1.2PB for 7500 $/month

You won't get asia/oceana/australia traffic for those prices, and you
certainly won't get S. American of S. African traffic for those prices.

------
LusoTycoon
Building your own CDN is not an option? Something small only to reign in the
costs

~~~
codexon
Building your own CDN these days requires you to own a large ip block and ASN,
and colo at several locations so the DC will actually bother routing to you,
in order to deploy anycast.

Certainly too expensive for a small deployment which is why people buy from
CDNs instead of setting up their own.

If it was as easy as just doing dns geolocation (which is awful due to
geolocation failing and ISP caching), few people would bother buying from
CDNs.

~~~
toomuchtodo
You shouldn't be anycasting TCP, only UDP. DNS is sufficient.

~~~
codexon
If anycast TCP didn't work, then cloudflare wouldn't exist. DNS is not
sufficient. It takes forever to handle downtime because ISPs don't respect
TTL.

~~~
zzzcpan
You are incorrect, I'll try to explain.

Anycast is essentially an SPOF. Well, not only anycast and not anycast per se,
but a single AS it is under. It breaks from time to time because of various
mistakes, bugs, etc. and brings down every server as a consequence. This
occurs roughly every couple of years and takes hours to resolve.

So, with anycast, if you have 10 servers in different places, you get hours of
downtime for 100% of users from time to time.

With DNS, on the other hand, if one server goes down, it affects only 1% or so
of users of a particular server, that have incorrect TTL in the resolvers they
use, others see change in DNS right away and use working server. But, those 1%
of users don't all go to that server at the same time, only small percentage
of them does and also sees the old record. Leaving us with let's say 10% of
that 1% on 1 out of 10 servers, or 0.01% of all users unable to see the new
DNS record for an hour or so. If a typical server on some random AS goes down
five times a year, you get 0.01% * 5 * 10 or 0.5% of users affected for an
hour per year. Now if you use round robin and let users see multiple records
nothing is even going to stop working in the browser for them, just going to
make them wait longer until they see a set of working records.

To summarize, anycast is 100% of users not able to reach any server for hours
every couple of years, while DNS is 0.5% of users experiencing slowness for an
hour per year. In other words: anycast alone cannot be reliable enough for a
CDN.

~~~
codexon
I never said you can't use dns on top of anycast.

~~~
zzzcpan
You have to understand that anycast comes from ISP/networking people, who are
biased towards network level solutions for anything and don't care about hard
numbers, no matter how badly anycast looks there.

EDIT: Anyway, resilience, just like security, needs a threat model to avoid
wasting resources on things that don't actually work, otherwise it's all just
hype.

~~~
codexon
As someone who has looked into deploying a DNS based CDN, I have frequently
found ISPs ignoring TTL.

Routers handle bgp updates much faster. Anycast routing is much more superior
to geolocation which gives a completely incorrect location as high as 10% of
the time. Anycast ips are also the best way to handle DDoS attacks. Attackers
can easily shut down ips listed in DNS while people's ISP and browser
repeatedly try to access the same dead IP for hours.

If DNS was an acceptable option, I would have gone with that instead of paying
a CDN for an anycast solution.

These are all from my personal observations, they are not something I just
heard from "ISP/networking people".

------
bluedino
Never heard of them. Does imgur say how much traffic they have per month?

------
winteriscoming
Never heard of them before but apparently they have been around since 2004 and
they support 450k websites and their only revenue model is advertising. I
doubt they had a solid technical and business plan behind this service.

