
RawGit is now in a sunset phase and will soon shut down - marvindanig
https://rawgit.com/
======
cmroanirgo
Kudos:

"Unfortunately, RawGit has also become an attractive distribution mechanism
for malware. RawGit was meant to improve people's lives, but jerks are
increasingly using it to hurt people.

Since I have almost no time to devote to fighting malware and abuse on RawGit
(and since it would be no fun even if I did have the time), I feel the
responsible thing to do is to shut it down. I would rather kill RawGit than
watch it be used to hurt people."

~~~
ken
Where does one draw the line? The internet is used for that, so should we shut
down the internet? Is the content of the internet the responsibility of its
creators, or operators?

I completely sympathize with this position for a one-person service, but I
think the real problem is that we still haven't figured out how to fix
incentives so we can distribute the work of keeping the jerks in line. On the
internet, the jerks are winning.

~~~
pmcjones
> On the internet, the jerks are winning.

Arguably, it's not just on the internet. But I try to be optimistic.

~~~
ken
Sure. I guess I'm old enough I still see the internet as a thing that we could
hypothetically choose to shut down (without killing half the population) --
unlike, say, agriculture and civilization. Maybe that's not realistic, though.

------
klenwell
I figured this day was coming. Excellent writeup. Sorry to see it go. Thanks
for all the raw fish.

Sounds like the cost of hosting this were not insignificant. Curious how was
this financed.

EDIT:

To answer my own question, I found the FAQ page, which is also great:

[https://github.com/rgrove/rawgit/blob/master/FAQ.md](https://github.com/rgrove/rawgit/blob/master/FAQ.md)

Sounds like donations helped (maybe). Oh and Stackpath. Kudos to them, too.

~~~
rgrove
Hi! I'm the guy behind RawGit.

I've actually never accepted donations. I paid the meager cost of the
$10/month DigitalOcean droplet for the origin server, and StackPath (formerly
MaxCDN) sponsored the CDN, which handled the majority of RawGit's traffic.

Once, I worked out how much the CDN bill would have been if StackPath hadn't
sponsored me and my head nearly exploded. So yeah, I definitely couldn't have
done it without them!

~~~
wumms
How much would the bill have been? Rough estimate: 176TB per month: $3.500 to
$4.200.

~~~
rgrove
It was a while ago now so my memory is hazy, but that sounds about right.

------
JCharante
RawGit was an amazing service, it's truly the end of an era for when you want
to render the HTML page for source code you're viewing by just changing the
domain.

------
tlrobinson
There are ~1M RawGit URLs embedded in open source projects on GitHub:
[https://github.com/search?q=rawgit.com&type=Code](https://github.com/search?q=rawgit.com&type=Code)
I'm curious to see how many there are a year from now.

~~~
Ayesh
How many of them would have SRI hashes added to them. If this domain
eventually expires, and whoever buys it next can make trillions!

~~~
rgrove
I'm not going to let the domain lapse for exactly this reason.

------
sdegutis
> _" Unfortunately, RawGit has also become an attractive distribution
> mechanism for malware. RawGit was meant to improve people's lives, but jerks
> are increasingly using it to hurt people."_

This has been my experience too, and I'm now convinced that hosting any online
services can only be done by corporations whose business model entirely relies
on it, and that a small full-time staff is needed to combat malicious users
effectively and keep the service usable for everyone else.

~~~
pjc50
This is something the decentralization fans need to take into account too: if
successful, ipfs will be full of malware being distributed from compromised
routers etc.

------
jimaek
It was a really cool project and I think the first to offer this kind of
features. Sad to see it go.

But right now jsDelivr supports both GitHub and npm as source for CDN files.
So here is an easy tool for migration
[https://www.jsdelivr.com/rawgit](https://www.jsdelivr.com/rawgit)

~~~
Ajedi32
Not really a full replacement, unfortunately:

> For security reasons, we serve HTML files with Content-Type: text/plain. We
> recommend using GitHub Pages if this is a problem.

------
geerlingguy
This will be interesting... I just did a grep of my local codebases and found
11 instances where CSS or JS resources were being downloaded from rawgit.com.

I have a feeling when it's turned off, a lot of sites are going to break.

~~~
rgrove
Yeah, I really, really don't want to break things, so I've tried to be very
careful about how I'm doing this.

In the shutdown announcement I committed to keeping the site running in sunset
mode for at least a year. Hopefully that's plenty of time for everyone who's
aware of the shutdown to migrate, but I expect there will be stragglers.

My unofficial, subject-to-change plan for dealing with that is that at the end
of the sunset year, if there's still a significant amount of traffic, I'll
start throttling requests to make RawGit slower. Hopefully people will notice
their websites are slow and will investigate. I'll also try to notify
stragglers individually by filing issues against their GitHub repos if
possible.

~~~
mhils
Instead of throttling, you could also consider doing incremental brownouts
where you drop requests for the first ten minutes every hour. PyPI did this
recently when they phased out TLS 1.0, which worked really well IMO.

~~~
rgrove
Thanks for the suggestion! This does seem potentially more effective.

~~~
Klathmon
IIRC Google also uses (or used) a similar system for deprecating old APIs.

Start with failing 1% of requests randomly and slowly ramp up from there.

------
superasn
I've never used rawgit for my own projects, but it did make life a lot easier
to check out demos and test run examples in JS libraries hosted on github.

Totally understand that the effort required to fight malware is tedious. But
instead of shutting it down you may want to try a $5/mo plan. The money is not
for making money but kind of like how Google chrome store required a $5
payment to publish your first chrome extension.

Spammers hate paying it and I think as soon as money changes hand there is
verification and a trail which makes them nervous too (many forum owners do
this as well).

~~~
madrox
I got the impression the author would rather not deal with the headache
anymore, and charging money is trading one set of headaches for another.
Building things is fun. Operating things is stressful. I don't blame them.

~~~
superasn
The stats he has quoted on the FAQ it wouldn't be surprising if it starts
making $10K/mo (just a wild guess) and the malware reduces as a side-effect
too.

But I guess you're right. Not everyone is driven by money and it would
certainly turn a hobby project into a full time job. Sounds fair to shut it
down.

------
flanbiscuit
It was great while it lasted. I currently have a small extension that depends
on it. Oh well, I knew the risks of using a free service so no complaints.
Thanks for the free service!

------
jopsen
By now you can configure GitHub to publish your master branch as github pages.
So no need to use rawgit either.

The only sad part, is all the links that are breaking.

------
piotrkaminski
Is there a service that can replace RawGit for Gists, specifically serving
them with the correct MIME type? This was a convenient feature for users of
[https://reviewable.io](https://reviewable.io) to hack up a quick custom
stylesheet or keyboard shortcuts override. It's not clear to me whether any of
the suggested replacements support Gists.

~~~
ic4l
try replacing "rawgit.com" with "rawgit2.com"

------
icebraining
Congratulations, it was a great gift for the web. Nothing lasts forever
anyway.

------
alpb
Precisely why I never use free CDNs or stuff like this to serve static
content. Just host your own thing. It maybe costs a little bit more but at the
end you'll have websites that will run for decades.

Rawgit was useful, I wish it stayed up for a little longer, perhaps someone
like Cloudflare could sponsor this?

~~~
Ayesh
CDNJS is from CloudFlare, running with what I believe is better DNS and more
POPs.

------
lima
Maybe Red Hat (who use Rawgit a lot) or, for that matter, StackPath want to
take over?

------
ravenstine
Thanks for everything! Are you going to open source the Rawgit codebase? I'd
be interested just for educational purposes.

~~~
rgrove
RawGit has been open source from day one:
[https://github.com/rgrove/rawgit](https://github.com/rgrove/rawgit)

------
tokyodude
oh well, a few hundred of the working answers I posted on stackoverflow will
stop working. Not sure how I can fix them

~~~
Ajedi32
Thankfully, it's pretty easy to find the GitHub page for the linked source
with a bit of URL surgery.

~~~
Drdrdrq
And SO will probably fix this automatically for all answers.

~~~
Ajedi32
They might, depending on how widespread it is. It certainly wouldn't be
without precedent. Alternately, there might just be a manual effort to fix
them all.

------
admay
"You either die a hero, or live long enough to see yourself become a villain"

------
enimodas
Why not sell or give away the project to someone who has time to fight the
abuse?

~~~
rgrove
I did consider this, and several people have offered to take over, but
ultimately I feel that the thing that was most useful about RawGit — it made
serving HTML at scale dead simple for anyone with a GitHub repo — is also the
thing that made it most prone to abuse.

It came down to a simple equation: fighting abuse on RawGit will _always_ take
more time and effort than spreading abuse via RawGit. One persistent jerk
working a few hours a day could do so much damage so quickly that mitigating
it would require multiple people working full time.

There's just no good way to scale that and retain the functionality that
actually made RawGit useful.

I talked about this a little more here:
[https://github.com/rgrove/rawgit/pull/191#issuecomment-42831...](https://github.com/rgrove/rawgit/pull/191#issuecomment-428310526)

~~~
ilaksh
Look at how enthusiastic people are about moderating forums. Would it not be
possible for someone to add a web application where volunteers could go to
manage blacklists or something? Like prescreened volunteers.

~~~
wild_preference
It’s not free. You will have to now spend time managing people.

------
ic4l
For the time being I threw this up.

You should be able to replace rawgit.com, with rawgit2.com.

~~~
freddie_mercury
Are you going to self fund the $50,000/year in hosting costs and devote time
to fixing the malware problem?

~~~
ic4l
If the time comes, sure.

Also it can be hosted for much less.

------
fenwick67
I used this for a lot of Codepen stuff - sad to see it go.

~~~
rgrove
You might find this useful: [https://github.com/mallendeo/cdp-rawgit-
fix](https://github.com/mallendeo/cdp-rawgit-fix)

It'll scan all your pens and try to suggest jsDelivr URLs wherever you've used
RawGit.

------
minhoryang
I loved your product and enjoyed it. Thanks always!

------
samat
This was very useful service, sad to see it go.

------
platz
> 176 terabytes of bandwidth

How did he pay for this?

~~~
dguo
It looks like he didn't, for the most part:
[https://github.com/rgrove/rawgit/blob/master/FAQ.md#can-i-
do...](https://github.com/rgrove/rawgit/blob/master/FAQ.md#can-i-donate-
moneybitcoinpie-to-help-you-out)

"It's super nice of you to offer, but I don't need any donations at this time.
RawGit's server costs are minimal, and the lovely people at StackPath provide
RawGit's CDN service free of charge. Thank you though!"

------
scruffyherder
Yet another trendy hip service folds with little to no notice. Meanwhile
sourceforge keeps on going.

~~~
fastball
... what?

