We've been running GitHub Enterprise Server (first on-prem, now in a Cloud VM) for many years. Size: low thousands of repos, less than a terabyte of total data.
Pros: Not affected by GitHub.com downtime; stability has generally been good for us; you have full control over the org and user names on the instance (instead of having to pick globally-unique ones on github.com); can sync licenses between GHES and github.com if you want to use some private repos on github.com; can raise your API rate limit as high as you want; probably cheaper to provide your own storage.
Cons: New features from github.com can take months (or longer) to be released to GHES; many third-party integrations only work with github.com not GHES; if something goes down, it's now your fault; Actions requires bring-your-own self-hosted runners; autoscaling your self-hosted runners is non-trivial.
The last point is probably the biggest negative for us. We're trying to use GitHub Actions more, but I don't really want to have to manage a pool of autoscaling runners. (I'm doing it, though.) If GitHub provided an add-on service to connect GitHub-managed cloud runners to GHES, I'd use it. (Needing to always keep code private and on-premises is not a requirement for us.)
Have you considered the "Enterprise Managed Users" feature? In my understanding this is where GitHub runs a separate instance of GitHub for you (i.e. it's not on public github.com) and one of the benefits (in my understanding) is things like globally-unique org names and such no longer applies.
Just be forewarned that GHE is not at feature parity with .com; back before I jumped ship to GitLab we were waiting forever for some of the most basic features to be backported, and that's not even getting into the mess that is "tools which claim to work with GitHub but haha have github.com hardcoded everywhere
The sibling comment about needing to use a partner may speak to this, but unlike GitLab GHE is also not `docker run ...` and tada, so you're right to be wary of the pain
It happens my company partners with GitHub to deliver with this kind of thing. Drop us a line over at [1] and we can give you a breakdown of the different migration/deployment models.
On one hand GitHub seems a little more proactive than most when it comes to raising incidents and updating status pages, but we’re getting a little too used to pausing our deploys in the middle of the day.
Again? Last time an incident like that happened was Packages which was less than 24 hours ago. [0] Before that weeks ago [1] At this point, it is clear that GitHub is essentially falling apart and is becoming extremely unreliable.
It's not too late to join the serious projects like Blender, ReactOS, RedoxOS, etc who are happily self hosting and are not going all in on GitHub.
I posit that they even have better uptime by just self hosting for one year vs GitHub's uptime record since they got acquired by Microsoft. Looks like 'centralising everything' to GitHub did not age well at all, as I have warned years ago. [2]
> why always repeatedly write the same comment about not going “all in on github” in these threads? what is goal of writing this hundred+ times?
Because I can. How many more times does a service need to go down in less than a month to deem it unacceptable for any use? Unless you think paying for downtime is acceptable?
> i dislike github quite alot but this does not achieve anything.
That is not my problem since I saw the problem years ago and chose to self-host instead. Rather than others complain to GitHub, they might as well self-host instead for reliability.
If you can setup a simple SaaS website, you can easily self-host a Gitlab / Gitea instance for 1 to 5 people. It's not that hard.
> no one else is saying to go all in on github so what is this even reply to?
So you didn't read this then?
>> We're all in on Github actions these days but there was a time when we used to maintain our own Jenkins instance. [0]
>> At this point, our self-hosted CI and code review tooling has had significantly better uptime for the past year. [1]
The schadenfreude for GitHub's unreliability and some of its fans celebrating monthly down time only makes it hilarious to watch.
so, because two people said they go all in on github on an hn thread TWO YEARS AGO , you feel the need to post public i-told-you-so hundreds of times ???!???
i am amazed @dang is ok with that, it seem completely deranged man, borderline harassment
I already gave the solution years ago [0] and each time GitHub goes down, hundreds of comments continue to complain when the solution is right in front of their eyes.
> i am amazed @dang is ok with that, it seem completely deranged man, borderline harassment
What harassment exactly? That whatever helpful advice I said in [0] and not 'centralizing everything to GitHub' seems to have aged well?
There is nothing 'deranged' about explaining preventative solutions like 'self-hosting' to those still using an unreliable service. In fact, it is more 'deranged' to believe that GitHub has gotten any more reliable after Microsoft acquired it.
You can continue to use GitHub's services, but don't complain when something from GitHub goes down again in less than a months time when the solution has been in front of you for years.
Seems like you're the one that is showing signs of 'derangement' and episodes of 'madness'. I'm just laughing at you and also at the ones that still continue to rely on a service that evidently goes down each month with the solution given to them, years ago.
> ok? and who in THIS THREAD is saying anything remotely like that?
Whatever someone said about exactly “all in on github” verbaitim is a 'minute detail' you decided to focus on to deliberately miss the forrest for the trees and to make an excuse for an argument. I also include using any of GitHub's services, which as the evidence suggests is beyond poor and comes with folks here complaining about them. Especially GitHub Actions.
So you attempting to narrow the scope to who said what verbaitim from the start is essentially missing the wider point. In fact, it is a distraction, derailing the general and entire point of GitHub's years of unreliability.
> who are you responding to??!?? comments makes no sense, derangement!!!
So far, you for showing yourself as a prime example fitting the definition of 'derangement' and to everyone else who is complaining about GitHub going down despite its track record, assuming you have read the links. There's no need for you to continue embarrassing yourself here and in your next reply, since you continue screaming and 'shouting' in caps-lock whilst attempting to put forward distractions that fails to address the main issue of GitHub's history of downtime and incidents.
you are spending time creating multi-year linked list of hn comments every single time a website you dislike and dont use goes down
i am “failing to address main issue of github’s downtime history”? indeed sure i am not addressing this, because it is not my care to address! why care when website i dont use and dont like goes down? not my problem!
do you see anyone else on entire hn creating redundant toplevel comments that then link to previous comment on same topic, spanning hundreds of comments across multiple years, all to post told-you-so to some random other commenters from 2-3 years ago? no one else do this, is deranged!
Do you want git hosting, CI/CD, etc. to be among your core competencies? Because that's what this entails. You need a full team to staff it and be on call.
I guarantee you that those who self host will have just as many breakdowns. Five nines is hard, especially when it's not your main job.
Edit: Must be something bigger than just the MS/Github hosted agents. Can't trigger builds or deploys on my self hosted agents either. Control plane issues perhaps.
Oh no! I was working on my side project; about to test the integration with Azure Computer Vision and this happened. Since I have no idea how to deploy my code to Azure without GitHub Actions and don't have the time to learn I will call it the day and see it tomorrow.
Tony Soprano : [over the phone] It's a bad connection, so I'm gonna talk fast! The guy you're looking for is an ex-commando! He killed sixteen Chechen rebels single-handed!
Paulie 'Walnuts' Gualtieri : Get the fuck outta here.
Tony Soprano : Yeah. Nice, huh? He was with the Interior Ministry. Guy's some kind of Russian green beret. This guy can not come back to tell this story. You understand?
Paulie 'Walnuts' Gualtieri : I hear you.
[the telephone connection is lost - Tony swears, and Paulie hangs up]
Paulie 'Walnuts' Gualtieri : [turning to Christopher] You're not gonna believe this. He killed sixteen Czechoslovakians. The guy was an interior decorator.
Christopher Moltisanti : His house looked like shit.
There you go and OpenAI is likely going to get the same share of outages - Like GitHub.
Just don't 'centralize everything to GitHub' like someone else said before since we now know that with this foresight made years ago [0], it is a very bad idea in the long term.
Funny how my troubleshooting mind works - I assumed I had borked the config, then assumed they had introduced a bug, then found the real reason is an outage.
Azure DevOps pipelines are experiencing issues and we can't trigger jobs against our self hosted agents currently, previously they did. I assume what ever is occurring here is more control plane related than host specific.
Dozens of HTTP 500s and so forth each week. I have built us quite a large library of actions with transient retry due to this. I'm out of options for the remaining daily failures, so the "automatic retry failed actions" sledgehammer is up next.
Keep the f$!# away from it. I have suggested moving away from GH entirely, but my peers aren't too keen on migrating to a different CI system yet again.
If you rely on Actions and want proper monitoring to alert you of these kinds of things, Cronitor has pretty a nifty monitoring plugin (which, itself, is an action, naturally)
