Hacker News new | past | comments | ask | show | jobs | submit login
GitLab sees huge spike in project imports (monitor.gitlab.net)
1736 points by AgentK20 on June 3, 2018 | hide | past | favorite | 455 comments

For people wondering what makes GitLab any different, the answer is that GitLab is an open source product at its core. This means that anybody can run their own instance. If the company ends up moving in a direction that the community isn’t comfortable with, then it’s always possible to fork it.

There’s also a proposal https://gitlab.com/gitlab-org/gitlab-ee/issues/4517 to support federation between GitLab instances. With this approach there wouldn’t even be a need for a single central hub. One of the main advantages of Git is that it’s a decentralized system, and it’s somewhat ironic that GitHub constitutes a single point of failure.

In theory this could work similarly to the way Mastodon works currently. Individuals and organizations could setup GitLab servers that would federate between each other. This could allow searching for repos across the federation, tagging issues across projects on different instances, and potentially fail over if instances mirror content. With this approach you wouldn’t be relying on a single provider to host everybody’s projects in one place.

GitLab is a lot like Firefox, or "Linux on the desktop" in that way. It's what a lot of us want to use, but the less-open but more-polished option has always seemed the more pragmatic choice.

But that can change.

Recently (just as most of the world has apparently moved on from desktop computing, haha) Linux is pretty much fine for the traditional desktop computing. I have current Mac, Windows, and Ubuttnu on my desk, and they are essentially interchangeable except for a few special-case purposes (say, Final Cut video editing, or opening one of those weird wonky Excel sheets that only open on Windows).

Firefox, too, is suddenly performant and I've switched to it as my main browser (for default website use) — something absolutely, utterly unimaginable two years ago.

I hope that GitLab is reaching the same kind of transition point. I've heard horror stories from people that used it 2-3 years ago and are happy to be back on GitHub. But I don't hear much recent grumbling. I moved a toy project to it and it seems nice. As fast as GitHub for me (though I am in Japan, and GitHub is slow in Japan). More features. A nice, sort of adult/professional aesthetic. And — yay! — the open source core it's always had.

I might be wrong, since I haven't used it for reals, but it looks like they might have hit that critical usability threshold?

Open source and worse isn't a very compelling sales pitch. But an open source tool that is broadly equivalent to a closed source one is generally more attractive, especially when you're talking about services that will be used as part of your core infrastructure.

As a faithful Fx user since 1.5, I'm not sure about Firefox analogy.

Its market share was actually much higher in the past and only went down (the introduction of Quantum didn't really help market share wise).

And in term of polishness.. Firefox was MILES ahead of any browser for a very long time. When Chrome launched, it was as horrible as you can think in term of functionality, and Firefox at that time was already as solid as today. But Chrome advances very fast. People often attribute Chrome's success (merely) to Google's push, but Chrome's technological development plays a bigger role IMO.

I'd argue Firefox was easily far more fleshed out, but Chrome was simply faster when it first came out, and I made the switch. It turned out all those features didn't mean jack when a faster option was out there. Of course, Chrome has added a lot of features now... and Firefox has gotten vastly faster than it used to be.

When I left Chrome a few years back, the sluggishness of Firefox was quite palpable. I eventually moved to Edge, which, while crashier, overall performed better. But after both the Electrolysis project, and Quantum, Firefox won me back quite solidly.

Long story short: Speed > features when it comes to web browsers.

I share the same thought. I use Edge on Windows when it came out simply because it feels so much faster and smoother. However, it has such a high tendency to glitch out and freeze sometimes I wonder how can Microsoft release such a glitchy browser and expect users to use it as their main browser.

In 1998 I purchased a Windows computer for the first time in 14 years. My thoughts were "They've had 10 years. Surely they've worked their bugs out". As it turned out, Windows 98 and Internet Explorer 4 actually did have some stability problems, to put it mildly. So I would say that releasing very buggy software without blinking is not a new concept to Microsoft.

> "They've had 10 years. Surely they've worked their bugs out".

That's quite a weird thing to think in 1998 with respect to a browser, given that they'd only had any sort of a browser for less than 3 years at that point.

The question was posed "I wonder how can Microsoft release such a glitchy browser and expect users to use it as their main browser."

To which I gave a related anecdote and responded

"So I would say that releasing very buggy software without blinking is not a new concept to Microsoft."

Since they released an entire OS that was massively unstable (win98), while claiming their browser was an integral part of it, it doesn't surprise me that they would release a buggy browser at a later date and expect people to use it.

I wasn't only addressing browsers, but rather my perception of the general quality history of Microsoft products at that time. It affected my perception of Internet Explorer, which at that time was not as popular as Netscape Navigator.

He clearly must have had a buggy experience elsewhere before.

Bear in mind, the previous default web browser was IE. ;) Raising the bar is not hard when the bar is lying on the ground.

When Edge actually works, it's wickedly fluid. That's IF it actually works though. I haven't been using Windows much lately to know the state of Microsoft Edge

It usually works. When it doesn't, it really doesn't though. And there's some annoying quirks: For instance, Edge always restores tabs after a crash, and you have to literally create nonexistent registry keys to disable that functionality.

The reason this is a problem is because when a malicious webpage hijacks your browser, and you have to forcibly close it to escape... Edge helpfully reopens the malicious webpage each time you relaunch Edge until you find out you need to hack on the registry to fix it.

I often try to use Firefox but there are certain things that just became irritating...that were entirely Google's fault. Like Google Meet ONLY working on Chrome.

You click enough links in Slack that open in Firefox, don't work and then you have to C&P the meet link into Chrome...eventually your commitment wears down and you just start using Chrome as the default again.

And I hate that. It's a very IE6 type move on Google's behalf. Short of applications / my system giving me a clear way to say "always open links on this domain in this browser" it makes the workflow a pain.

Maybe the Facebook stuff will make that type of thing more popular. If I could make Firefox my default browser, but always open Facebook and Google links in Chrome I'd be pretty happy. Currently running Linux Mint.

Yup. The entire backslide into "best viewed in IE6^WChrome" is ironic, considering how Chrome was launched as "the open, standards-compliant browser".

Chrome made a big thing about unit testing at the beginning of their development. I'm not sure if their definition and my definition of "unit test" matches up, or even if they still have that drive, but I've often wondered how much that has made a difference in their development process. My own experience has been that an early commitment to unit testing can make a massive difference to sustained development pace (pays off more as time goes on). My inner confirmation bias would love it to be true :-)

I think the true advantage you're referring to is automated regression testing, which isn't necessarily just unit testing (or at all)

The interesting thing is (and without meaning to denigrate your point of view) I actually think that automated regression testing is not particularly valuable when you have unit testing. I actually want unit testing for the ability to help you reason about the code. One of the things that I would dearly like to discover is whether or not my view holds any water. It certainly feels that way from my perspective, but I'm biased ;-)

However, I think you are probably correct in that I think it is far more likely that Chrome has a good automated regression test suite than it has a good unit test suite.

Libraries benefit very heavily from unit tests, built applications or frameworks derive more benefit from regression tests.

For a small unit of code, or a library, the unit tests effectively prove that the code/library does what it says on the box.

For a continuously worked on application, regression tests holds the guarantee that the application continues to work correctly for whatever thousands or millions of use cases built up over its lifetime - even when the implementations, algorithms and libraries used change underneath.

Good regression testing is a nice way to keep unit tests honest. If you never catch anything with your unit tests but boat loads in regression testing - you are inclined to pause and ponder.

I actually test my unit tests by changing the behaviour and measuring if the tests fail :-). A great metric is fuzzing the behaviour and measure the number of unit tests that fail. If it's a lot, then you can improve your unit test game :-)

The problem with unit tests is that they test things you know are problematic. The larger issues is then ones you were never aware could be a problem in the first place.

A larger problem is they make you feel better about bad code.

I highly recommend reading Michael Feather's Working Effectively with Legacy Code. He has the best description of unit tests that I've seen. Briefly, he describes it like a clamp. When you are working on a wood working project, you clamp part of your project so that it doesn't move. Then you work on the bit you are interested in. Later you clamp that part and work on another part. The purpose of unit tests is not to test the behaviour (unfortunate nomenclature aside) -- it's to immobilise it. This allows you to work on another part of the the system and be alerted if you've caused something to slip.

Acceptance tests are incredibly important. They tell you if the system is working. No amount of unit tests are going to help you with that. Once you have accepted the behaviour, what you're really interested in is whether or not the behaviour has changed. You do not need your acceptance tests for that -- your unit tests will tell you.

I'll write it a bit more concisely because I think it is important: acceptance tests tell you whether or not the code is working correctly. Unit tests tell you whether or not the code is doing the same thing it was doing the last time you ran the tests.

The reason I don't favour a large suite of acceptance tests is because they are unwieldy. It's fine for a few months, but once you get a few tens of thousands of lines of code, you will end up with a lot of acceptance tests. These acceptance tests are extremely hard to refactor. It's extremely hard to remove duplication. Over time, they get more and more problematic until you are spending more time trying to figure out how to make your acceptance tests pass than you are trying to figure out how to make your production changes.

Unit tests, when written in specific ways, have less problem with this. Some people think about a "unit" as being a class. But really a "unit" is anything that you might want to isolate in your clamp. It can be a function. It can be a class. It can be a module. Your unit tests should probe the behaviour in the unit (and by "probe" I mean, expose the internal state). Michael Feather's has a great analogy of a "seam" which runs through your code. You try to find (or make) that seam and you insert probes to show you the state in various circumstances.

IMHO, you should write unit tests the exact same way you write any code. Your "circumstances" (or scenarios, I guess) consist of creating the data structures to give your initial state. Your "tests" consist of probing the state along the seams and comparing it to expected values. This is simple code. You should be able to maintain this code using the same tools you use to maintain any code. You should write functions. You should write classes. You should write modules. You should use all the tricks of your trade to reduce the complexity of your "test" code. Your goal is to create specificity when tests "fail" (the probe detects behaviour different than your expectation -- or the clamp detects that your wood has slipped). When behaviour changes, only a few tests (ideally one) should "fail". It should report the "failure" in a way that immediately describes the difference between the state you expected and the state that you probed. It should be easy to change the probe when the behaviour is intentionally changed (ideally changing only one place). It should be easy to probe new behaviour (just build your data and add an expectation). Finally, it should be easy to reason about the behaviour of the code by reading the "tests". Refactoring your tests and removing duplication is very important here.

As for acceptance tests, like I said, they are incredibly important. What I don't find particularly useful is a large suite of regression acceptance tests. The unit tests already tell me when the behaviour has changed. When written well, they even tell me exactly where in the code the behaviour "slipped". I often write manual acceptance tests. Once I have tested it, it is not necessary to test it again (as long as I have a good unit test suite).

My personal opinion as to why people find automated acceptance tests suites important is because they have never worked with a good unit test suite before. There is a general lack of experience in the industry with these concepts. Quite a few people's experience with well tested code is with green field projects. Often these people leave after a year or so. It's not until you have a lot of experience working with the legacy of various testing techniques that you can understand the advantages and disadvantages. I think this is why Michael Feathers is so respected -- as far as I can tell he specialises in legacy code.

Having said all that (and I'll be surprised if you make it to the bottom :-) ), I do value a small automated acceptance test suite. It's my canary in the mine. If it ever fails, then I know I've really stuffed something up and I launch an immediate investigation. Also, there are some things that can't be unit tested effectively (for example testing a web application across both client and server) -- you end up faking the boundaries, which leads to the possibility of skewing. Again, in those cases, I try to find a few end to end tests that will hit the majority possible problems.

I hope you found that interesting. I've typed up essentially this same message in at least 10 different threads of the past couple of years. I think it's slowly getting better, but I think I still haven't managed to explain the concepts as well as they need to be explained. If you've made it this far, thanks for reading :-)

You can use mutation testing[1] to automate this. E.g. pitest[2] for Java, or golang[3].

[1]: https://en.wikipedia.org/wiki/Mutation_testing [2]: http://pitest.org/ [3]: https://github.com/zimmski/go-mutesting

It felt like for a while, FF was lagging behind. But it’s great these days. So far my only problem with Quantum is that they killed grouped windows.

I'm not sure if it covers the use case you're looking for, but they've already introduced the API's necessary in Nightly (or Beta?) that allows the Panorama Tabs extension to be re-introduced :)

I have been using gitlab heavily for the last year and since then nothing terrible has happened. The worst thing was for one day it was pretty slow but was fixed the next day and no data was lost.

GitLab today in my opinion is now a better piece of software than github.

I certainly disagree with this comparison.

I've been using linux on the desktop for over 15 years with no intention to ever stop the near future.

I had been using firefox since its first release under the name phoenix then firebird then firefox (I had been using mozilla suite before that) and I've stopped with no intent to ever come back to it when mozilla finally killed what made firefox useful to me after a lengthy agony process (BTW the claim of firefox being not performant enough 2 years ago is totally unsubstantiated as I used it daily with over 250 tabs open concurrently without a single hiccup despite having 35 extensions loaded as they were required to put back the useful features mozilla had removed, to remove the unwanted cruft mozilla has added and to add the necessary features mozilla refused to add). I have now switch to waterfox, and its name says it all.

So really comparing gitlab to linux on the desktop means gitlab will never happen, and comparing gitlab to firefox means it will be mishandled into irrelevance by a shady finance operation aiming for market domination.

To me Gitlab seems a better alternative to proprietary and centralized github that will be bought at some point in the not so distant future, has been my stance on this matter. That Microsoft is the one buying would not have been my first bet but is not a huge surprise either considering their change of PR to jump on the opensource bandwagon as an attempt to extend their agony further.

> Recently (just as most of the world has apparently moved on from desktop computing, haha) Linux is pretty much fine for the traditional desktop computing.

It's been fine for the past decade, it's just the trope of "Linux on the desktop" that's slow to die.

Eh... it's mostly fine. It's just that when something doesn't work, you're screwed if you're not technically confident enough. Emphasis confident - you have to trust your instincts when digging through internet fora for a solution. Something that I can never see my mother doing, for one.

>Eh... it's mostly fine. It's just that when something doesn't work, you're screwed if you're not technically confident enough.

As opposed to Windows or OSX where you're just screwed.

There are rough edged in Linux on Desktop but people seem to be completely house-blind* about Windows and OS X. If you spend a lot of time on Linux and go back to Windows or OS X the rough edges in those platforms become immediately obvious.

* If English isn't your first language, house-blind is where you get so used to something being out of place that it becomes part of the decor. e.g. a jumper on the back of a chair that stays there for a week+.

Oh, I agree with everything you say!

But we have to put this in the context of an elderly lady who is terribly frightened of breaking the computer and always believes it's her fault when software screws up, because the feedback loop of computers is terrible (either absent, or opaque jargon, or marketing lies). That is, my mother.

And keep in mind that I live abroad. Helping her out remotely is a very difficult and slow process - if I lived close by the story would be different. In that context, when it comes to Windows or OSX, my mother has a lot of people who can help her out other than me - my sisters, my father (they're separated but still get along), some of her friends.

Now, my younger sisters are getting into programming (because all professions need it) so maybe I can get them into Linux too - they're definitely capable but the question is whether they consider it worth the investment. But still.

I'm not convinced. Chromebooks seem to be popular enough and simple enough for luddites. It's certainly a flavour of Linux; despite being a bit locked down afaict.

Maybe we're so used to expecting Year of the Linux Desktop to mean Year of the FOSS Linux Desktop that we ignore the successes.

Well, I agree that I was thinking about the FOSS Linux Desktop experience, yes.

> As opposed to Windows or OSX where you're just screwed.

Let’s not be disingenuous. I don’t know about Windows, but macOS has had absolutely wonderful critical failure recovery for a while now: There is the recovery partition, which acts like a mini-macOS and lets you do various things like drop into Terminal.app, use Disk Utility for drive scanning and repair, or do an ‘archive and install’ (extremely useful for the technically challenged) which keeps all your files but sets up a fresh macOS install. If even the recovery partition is borked you get the option of ‘Internet Recovery’, which connects to WiFi and automatically downloads and installs a fresh copy of macOS (with the aforementioned archive function, if an old install is detected).

Compare this to Linux, where you either get dropped into GRUB or a bare shell..

I'm English and I've never heard the phrase "house-blind" before. You're probably better off using phrases which don't require an in-line definition.

I've never heard of it but it's a useful idiom, I approve!

I agree but I don't know a better shorthand to describe the phenomenon.

I have quit Microsoft for Linux 8 years ago. I do the remote administration of the computer I have installed for my old mother since 5 years. At home, there are 3 Linux computers, one of them has a dual boot to Microsoft. My cloud website is on baremetal Scaleway running ubuntu. During last year, I have spend more time for system administration on windows (that I use once a month to use an application during 5 minutes) than for administration of the 5 Linux systems.

If someone asks me to help for the administration of his Linux machine, I would accept because it is so easy and so little work compared to windows. I think Linux is perfect for the noob who accept to delegate administration.

I'm happy for you and your mom! But you cannot compare that to the situation of my mother - I'm trying to make her more confident but it is a very, very slow process.

It's not that I'm not willing to help, but I live abroad. If I lived close by, I would gladly install something like Ubuntu or KDE Neon on her machine (probably Ubuntu though - the mainstream would make it easier for her to find things on her own).

Whenever I'm home I help her with her computer. The whole thing is very educational for me as an interaction designer as well. It often shows how modern interfaces make her feel like she is the dumb one, when honestly it's often the arrogant UI designers who think everyone is on board with modern UX paradigms. Or worse, abuse dark UI patterns for evil purposes.

Maybe try Elementary OS? New version coming soon. Really focuses on simplicity and UI/UX and being more intuitive for Mac / Windows users.

I'm more worried about overcoming technical issues.

UI wise I would have helped her switch to one of the distro's ages ago, and I agree that Elementary is a good candidate.

Ubuntu gnome is more similar to the "simplicity" of windows XP than vista, win7 and win10. The changes of windows are too big and too fast for old users. My mother is still on ubuntu 14.04 to keep this stability.

Honestly, I just came home to see that somehow Bing search had installed itself over DuckDuckGo (my mom loved the search engine for the name alone), and that her Chrome browser had turned into a touch edition which hides the mouse cursor. And that was the least offensive change.

Trying to fix her Lenovo Yoga I had to navigate a forest of dark UI patterns, with pre-installed apps trying to trick me into sharing private data every step of the way.

I really fucking hate this user-hostile attitude that can only be explained by greed. I've "fixed" her computer one more time, but I think the next time I'll let her try a bootable Linux distro, and see if she likes it enough to be willing to give it a try.

"Fine"... that seems to have little to spare already.

I switched from Windows to Ubuntu last December, and it has given me a whole new appreciation for Windows. The polish (things just working well) of recent Windows versions is just amazing, in comparison to Gnome/Ubuntu.

PS. Will stick with Ubuntu though. PPS. Gitlab is an awesome product and company.

>GitLab is a lot like Firefox, or "Linux on the desktop" in that way. It's what a lot of us want to use, but the less-open but more-polished option has always seemed the more pragmatic choice. But that can change.

I wonder if this move by GitHub is motivated by them seeing this writing on the wall.

No, it’s motivated by 5 billion dollars (or whatever the real number is). That’s why people funded GitHub.

a16z, Sequoia and friends got their liquidity event from $350+ M invested. Looks like a (probably $3.5+ B) 10x or better ROI, which should help VC’s IRR numbers.


Your numbers are a bit off. The fundings were done at roughly $750M and $2B valuations. Still going to be great for investors, but most likely not 10x or better.

We deploy GitLab locally and it’s a great experience.

Same here. The CI is easy to use and makes sense, though it lacks some features - for instance being able to automatically run manual jobs as soon as their predecessors complete. Now I have to wait (!) and then click so the manual job starts...

But all in all, good product, I hope they succeed!

wouldn't those "automatic manual jobs" just be normal jobs?

I think GP means that you know that for this one pipeline related to commit X you want manual step Y to proceed once previous steps are ok, and you know it right now, so instead of waiting for previous steps to complete you want to trigger the step in a delayed fashion. Kind of like "merge this as soon as tests pass" on MRs.

Exactly. (& sorry for late reply)

Same, I've had it running on a proxmox VM on a nuc in my room for years. It's considerably better than github, and damn that CI is good.

> Linux is pretty much fine for the traditional desktop computing

Yeah, on the desktop things are getting better. But ... everybody is moving to the smartphone. And there things are getting worse. For example, my Banking app works only on 2 platforms, which are not open.

But I assume their online banking website works on all/most modern browsers? Do you need to use the app?

> Recently (just as most of the world has apparently moved on from desktop computing, haha) Linux is pretty much fine for the traditional desktop computing.

It's exactly because the world has moved away from desktop computing that Linux on the desktop has become viable: collaboration tools are increasingly web-based (or is at least web-enabled for the 80%-usecase), and those tools are exactly what tends to anchor an organisation on a single platform. These days, even Outlook has a perfectly usable web-interface that works fine in Chrome and Firefox.

Off topic but I couldn't get hibernate to work on Ubuntu Linux. I purchased a laptop with Linux pre-installed; but apparently, if a manufacturer claims to support Linux on a laptop, that does not include hibernate support. Now that we've reached the milestone of Linux on a desktop, I'm looking forward to the Year Of Linux On The Laptop.

People here are deluding themselves. Gitlab, without gobs of VC money and the promise of a big IPO payday, is an abandoned open-source project with a tiny fraction of the team that built the code. So yes, you can fork the code, but without the money and resources of the parent company, good luck keeping it up to date! Worst case, you get another mysql: neglected by an acquiring company, with lots of bureaucracy and infighting and IP tangles to slow things down.

Gitlab is essentially salting the earth for dev tool startups. I had my issues with Github, but at least they had built a business around a dev tool, behaved ethically and gave back generously, and so I wished them well. To see so many people dropping them for a fauxpen-source competitor whose primary selling point is “it’s free!” just makes me sad.

If you want nice things, you have to pay for them. If you aren’t, I guarantee you that someone else is, and they’re the ones with control.

Sounds like you've got an axe to grind.

> I had my issues with Github, but at least they had built a business around a dev tool ...

That's a strange claim given that in the current top story - of Microsoft buying Github - the following claim is made:

"The acquisition provides a way forward for San Francisco-based GitHub, which has been trying for nine months to find a new CEO and has yet to make a profit from its popular service ..."

Perhaps it comes down to your definition -- can something be non-profitable for a decade and still be called a business?

The only “axe” I have to grind is the one I clearly stated: Gitlab is salting the earth for dev tools. You can’t build a business competing with someone who is using VC money to give away their product. This is all going to end badly when the music stops.

”can something be non-profitable for a decade and still be called a business?”

This is such a bizarre talking point...do you honestly believe that Gitlab is a better business? Their model is “just like Github, but with even more stuff given away for free!” And let’s not forget that Github has to compete with Gitlab cannibalizing the low end of the market. I’m sure that hurts margins.

Someone has to pay those Gitlab engineers who are writing the bulk of the code. At the very least, as soon as the dumb money dries up, the velocity of development on Gitlab will drop like a rock. In the worst case, you’ll get an conflicted corporate hydra, like mysql.

I understand that you're claiming gitlab is salting the earth, but still don't understand why / how.

You write:

> You can’t build a business competing with someone who is using VC money to give away their product.

This is delightfully worded, given it could apply to both github and gitlab.

Remembering that github started in 2008, while gitlab.com started four years later (first commit to their codebase was 2011).

Github is running on $350m of VC funding.

In response to my question 'can you call a 10yo company that still isn't profitable, a "business"?', you avoided the question, called the matter bizarre, and tried to distract from the question by claiming github is a _better_ business.

Your claim that github has 'built a business and .. gave back generously' is also weird in that gitlab has released the source to their core product, but github hasn't. This also speaks against your claim that you're more likely to be abandoned if you commit to gitlab than github.

Finally, the idea that the 'low end of the market' is where all the money is does not match any other tech startup's experience, is belied by the pricing structure of both companies, and further invalidated by the fact that gitlab is not swimming in cash from their cornering of the frugal user segment.

If you're a HN reader, I highly recommend Jason Lemkin's writings. Gitlab is a classic case of a bottom feeder late follower: https://www.saastr.com/dont-confuse-room-at-the-bottom-with-... .

And what that means is, yeah, either they keep burning $$$ every month and selling more of the control to VCs to feed the war chest until they maybe buy 2nd place, find an acquirer (and with that much ever-increasing VC control, a likely push), or yeah, layoffs will happen. Gitlab is extra interesting because their definition of innovation is biting off even more surface area (e.g., CI), and therefore even more burn.

Keep in mind.. all this says zero about how nice the product quality is or how friendly the people are. But just in the same way you don't get mad at what happens if you stick your hand in a lawn mower (https://www.youtube.com/watch?v=-zRN7XLCRhc&t=34m7s) ... there are financial forces at play from being a high-spending bottom feeder that are hard to escape. Possible, and I wish them luck, but that's a real bet.

AFAIK, Github went for growth. Gitlab went for cash flow. Gitlab is profitable and, imo, their product is comprehensively superior to Github.

>Keep in mind.. all this says zero about how nice the product quality is or how friendly the people are.

Then don't use the term bottom feeder since that means the people are making a shitty product with no ethics to really innovate. It says the people are shameful hacks and the quality of the product is bad.

In reality Gitlab is a better product and the people involved should be proud of their work.

I don't think their official statements match that? They say their fundraising approach is 2yr runway, which is only 6mo longer than the advice for a regular VC-backed startup, and they've been raising increasing amounts ~annually.

Based on that, having 275+ employees, and their stated IPO targets, I ran the numbers recently. My guess was their costs are ~$40M year (admirable: I expected way higher but they focus on non-US hires and pay only 50% percentile in _local_ markets: super low!). Likewise, their stated IPO and growth targets make me guess they make ~$20M/yr. So two different reasons to believe they're burning... ~$20M/yr. The positive thing for them, which they're not public about but I'd guess, is while they're probably growing OK in regular accounts (hard competition vs bitbucket, github, etc.), they're probably Super Great on retention + internal expansion, so net negative churn, compounding factors, etc. I think they _can_ stop hiring and let revenue catch up, though other forces take hold then: so it does look like they're on the classic growth-over-control VC treadmill (despite saying they're not), and will keep ceding control to VCs.

I think you may be correct and my information was out of date. According to the strategy documents that Gitlab publishes they seem to have changed direction towards growth via SaaS:


""" During phase 2 there is a natural inclination to focus only on on-premises since we make all our money there. Having GitHub focus on SaaS instead of on-premises gave us a great opportunity to achieve phase 1. But GitHub was not wrong, they were early. When everyone was focused on video on demand Netflix focused on shipping DVD's by mail. Not because it was the future but because it was the biggest market. The biggest mistake they could have made was to stick with DVDs. Instead they leveraged the revenue generated with the DVDs to build the best video on demand service. """

The term bottom feeder refers to going after the "leftovers" that premium market leaders leave on the table: lower-paying, more demanding (e.g., requires open source), higher acquisition cost (closeted international markets), etc. Good B2B companies often raise prices as they deliver more value and build brand trust, and as they establish the market, bottom feeders will pop up and spot the missing chunks. However, they are forced to play catchup in terms of features and with less $ (or a LOT of VC $). Says nothing about being nice, smart, and high quality, just the market & financial pressures.

No label is ever 100% accurate, but a lot of that dynamic has played out here pretty clearly..

> Github is running on $350m of VC funding.

Gitlab is also running on a bunch of VC money.

> In response to my question 'can you call a 10yo company that still isn't profitable, a "business"?'

Gitlab also likely runs at a loss. Gitlab has certainly never claimed to be profitable and some estimates are that as few as .1% of their customers pay for Gitlab.

> I understand that you're claiming gitlab is salting the earth, but still don't understand why / how.

It's pretty clear to me at least that neither Github nor Gitlab have sustainable business models. The OSS community is crazy to think that either business will continue to subsidize OSS development while losing millions of dollars a year. All of the anger against Github and the new "faith" in Gitlab is pure delusion. Both these companies subsidize OSS development while losing millions of dollars. This will go on until it stops. It certainly can't go on forever.

Personally I suspect the absolutely best thing to happen to both Github and Gitlab would be being bought out by real companies that heavily depend upon OSS and, you know, actually make money.

It came up before and now the chatter has started up again around Gitlab. I think it still makes a lot of sense for AWS to purchase Gitlab. There's a fundamental strategy alignment there (both Gitlab and Amazon aim to be a "one stop shop"), Gitlab offers the potential to lure a bunch of developers into the AWS platform with a free offering and, ultimately, Gitlab offers the same computational economics as other Amazon products because it is just another hosted product that requires a database. Wouldn't be surprised at all to see such a transaction in as little as 2-3 years.

Wouldn't a company like Gitlab be able to sustain a decent engineering team by just selling a few dozen top-tier subscriptions for their on-premises offering to top Fortune customers who are often still too afraid to have their crown jewels hosted in the cloud?

> Gitlab is also running on a bunch of VC money.

That is what I said.

I would say gitlab is more closely aligned with Google, at least technically, with their auto DevOps targeting kubernetes, and Google cloud having the most 'turnkey' k8s offering.

That would align with Red Hat, as would the open source.

”Your claim that github has 'built a business and .. gave back generously' is also weird in that gitlab has released the source to their core product, but github hasn't. This also speaks against your claim that you're more likely to be abandoned if you commit to gitlab than github.”

Uh...Gitlab is built upon libgit2, rugged and github-linguist. In other words, the core parts of Gitlab — the ones that interact with git are built, maintained and open-sourced by GitHub. And these are just the obvious dependencies. Github people contribute heavily to open-source projects that most Ruby websites use.

If you’re going to fanboy all over the place, fine, but at least know what you’re talking about when you do it. And don’t try to weasel out of it by talking about “core products” —- without GitHub’s substantial technical contributions to the infrastructural code that interacts with git, it’s a safe bet thst Gitlab wouldn’t exist. That’s core.

Wow. I feel you've certainly made a point.

Originally you declared:

> If you want nice things, you have to pay for them.

And I don't know how that fits in with people releasing / maintaining free software.

I responded to your first rant because you appeared to be 'going all fanboy' over github, declaring them a successful, superior business. I asked you if a company that hadn't turned a profit despite first mover advantage and a decade of trying could be termed a business ... and you weaselled out of that question.

I answered you, you just ignored the answer: yes.

If you believe Github isn’t a business, then you’re going to be sorely disappointed by Gitlab, whose business model is worse.

I’m done talking to you now.

> If you believe Github isn’t a business, then you’re going to be sorely disappointed by Gitlab, whose business model is worse.

The challenge discussing this with you is all your comments about Github are based around comparing them (favourably) to Gitlab.

> I'm done talking to you now.

This is a shame, as I'm consumed with curiosity on your take of today's news that Microsoft spent US$7b buying github.

From what you've described it sounds like they should have just cloned libgit2, rugged, and github-linguist, and rattled up their own gitlab over the weekend.

MySQL is a great example. Bought by Oracle, still a good product, but also forked by some big players as well as some open source groups. I'm sure it is still the most commonly used database on the web today and Mariadb and percona both maintain great MySQL forks as well.

The MariaDB story is a bit of a fuck you though - cries about how oracle will close mysql (which hasn’t happened) but then adopts a bullshit license for its own software.

But the damage is already done - people think MariaDB is some bastion of good intentions and open source software now, because they very rarely look deeper.

> The MariaDB story is a bit of a fuck you though - cries about how oracle will close mysql (which hasn’t happened) but then adopts a bullshit license for its own software.


There was strong precedent for fearing what may happen with MySQL. Knowledge of what happened to Hudson, OES, OpenOffice, Solaris ... this would concern the stewards of any bit of software that got swallowed up by Oracle.

(Edit: Also I recall some worrying stories coming out from Monty and other key developers.)

What's this 'bullshit licence' that MariaDB has? I thought the source was (L)GPL all the way down?

Lookup MariaDB MaxScale.

> Lookup MariaDB MaxScale.

This is like a scavenger hunt.

I've looked up MariaDB MaxScale ... and found an optional / add-on product that is aimed at Enterprises, seems to require an Enterprise support licence for the Enterprise edition of MariaDB ... and I completely fail to see how any of this demonstrates that the 'MariaDB story is a bit of a fuck you'.

A DuckDuckGo search for 'mariadb maxscale license' gave me this in result #3: https://www.infoworld.com/article/3109213/open-source-tools/...

Basically - their formerly GPL proxy for doing HA deployments is suddenly not open source.

They can of course make this decision - it's their code to do with as they wish. But it's quite fucking rich for Monty to claim Oracle will close source MySQL, create a fork and company which then uses that fear to grow in popularity, only to do the very thing he accused oracle of doing: closing an open source product.


Also, if you think only "enterprise" customers need database clusters that survive individual node's being offline, you're in for a big shock.

I personally have not and will never use MySQL again because Oracle owns them. That is a company where software goes to die. Plus their atrocious security record.

Indeed the most extraordinary story of the last ten years is how Google, Oracle, Redhat, Microsoft and Facebook have funded open-source software to tune of billions. This is likely the greatest act of charity the planet has ever known. And while a lot of holier-than-thou types (particularly here on HN) imagine these tech giants as not to be trusted or even the enemies of OSS, the numbers don't lie. Look closely at who actually funds and writes the vast majority of OSS and the same five companies pop up over and over and over...

Why would that be charity as opposed to a sane shareholder value maximizing business decision?

> Indeed the most extraordinary story of the last ten years is how Google, Oracle, Redhat, Microsoft and Facebook have funded open-source software to tune of billions.

Definitely not the most extraordinary story over the last decade.

And trumped by IBM's famous first $1b spend on 'Linux' just shy of twenty years ago, and their subsequent announcement that they'd recouped that money within a year.

Coincidentally this speaks to your claim:

> This is likely the greatest act of charity the planet has ever known.

These guys aren't in it for the charity. There's doubtless plenty of positive PR spin from contributing to free software -- but don't mistake pragmatism or happenstance for altruism.

Your confused if you think just because one benefits from charitable actions that somehow invalidates them.

And IBM's contribution was, frankly, marketing. It does not compare to the volumes of high quality technology that the companies I mentioned have simply given away for free.

Many on HN and others are perhaps too close to it but I think people will look back upon this extraordinary corporate charity as a decisive event of the century.

> Your confused if you think just because one benefits from charitable actions that somehow invalidates them.

I think you're being overly charitable to think these tech corporations had charitable intentions when they contributed resources to tech projects that happened to improve their tech business prospects.

> And IBM's contribution was, frankly, marketing. It does not compare to the volumes of high quality technology that the companies I mentioned have simply given away for free.

Bizarre you didn't mention that up front when you named 'the big five contributors'.

On what do you base your bold claim that IBM's contribution was marketing, and the other corporations weren't?

> Many on HN and others are perhaps too close to it but I think people will look back upon this extraordinary corporate charity as a decisive event of the century.

IBM announced their first billion spend last century.

>can something be non-profitable for a decade and still be called a business

Just ask Amazon

> Just ask Amazon

Maybe, we should ask ourselves first if it's a fair comparison. Amazon kept investing the profits into themselves. I don't think that is the case with Github though

Excactly, Amazon could have been profitable

Free cash flow is king

You do realize Github didn't take their first outside funding until they had been in business for 4 years, and had paying customers?

I use it for reals at my job. Enterprise version. I love it.

> But that can change.

That's because change is the only constant.

It should change and suspect quick. The big tech companies will have to move off of GitHub and go somewhere.

So either they do it themselves which is a risk or looks like GitLab.

Yes yes yes, federation is needed.

People on GitHub is not just to sharing codes, but also to get in touch with each others. GitHub succeed because it's not only just a free online Git repo services, but also a developer + user community where you can put your code on, share it, and 'earn' stars & forks as feedback. And stars + forks can help you stand out in a job interview and many other occasions.

Bitbucket is another Git repo service, but it sort of failed to build it's community. Result? It received less attentions compare GitHub.

So, while GitLab is also trying to be 'yet-another' Git repo + you can host it on your own, the benefits of become community can't be ignored. And federation can help that by connecting all the GitLab instances together to form a bigger and global community.

Even better, the federation protocol itself can be an open-sourced public standard, so all the other Git repo software can implement that in their product. The potential is huge.

Unlike GitHub though Atlassian isn't just working on GitHub but many other tools for developers. This includes JIRA, Confluence and so on. GitHub is mostly focused on repositories. This is why devs had to write a letter to GitHub to tell them hey can you guys seriously work on the Issues system? With the resources of Microsoft it will not surprise me to see a much more open and capable GitHub.

Except that Atlassian aren't really working on Bitbucket any more. There's lots of issues with the system that are not being resolved. Last time I had a problem with Bitbucket I found they had known about it for months and done nothing, with no plan to do anything either.

But then, as I understand it, Bitbucket was an acquisition rather than a brainchild of the Atlassian team, so you can expect a certain amount of neglect.

An omen for github?

Atlassian has a lot of people working hard on Bitbucket. They shipped pipelines, deployments, code aware search, new UI, git lfs, embedded Trello and a lot more while improving uptime and performance, getting soc2 type 2 certified and a massive amount of other stuff.

They might not be working on the bits you care about, but they're definitely working on it. I agree, a few more resources should be invested by companies to fixing bugs and not just adding new features instead.

> With the resources of Microsoft it will not surprise me to see a much more open and capable GitHub.

Devil's advocate: why would Microsoft invest in improving, say, the issues functionality of GitHub when it could instead integrate and push users towards its existing products and tools for project management, like SharePoint?

Because they develop some of their core open source products right on top of GitHub. Visual Studio Code, .NET Core (and dozens of core libraries for .NET / .NET Core) as well as the less popular things they work on in the open.

See their GitHub for more:


Also their focus is now on developers. They made VS Code completely free, MIT Licensed and not for themselves, but for developers.

Yes, beacuse they we're losing out as web development exploded, and realized no one wanted to pay for an editor when there we so many great free alternatives. They did it out of necessity, not out of innovation. Just like everything they do when it comes to their "open source movement"

I think this needs to go even further. If federation is done using an open standard like ActivityPub, then any services using the protocol would be able to federate. At that point it wouldn't matter what people are running, they'd all be able to talk to each other.

I've been reading into Activity Pub over the last week, and I don't think this would really be the case, (but I'm happy to be wrong).

For example, if a Gitlab instance posts a Pull Request object over to a Mastodon instance, what is the latter supposed to do with that? presumably it won't have any UI widgets to display the content, and no way of acting on it semantically.

As far as I can tell, Activity Pub is a way of federating instances of the same application, with the same semantics. But on the internet I'm seeing some dialogue which seems to presume that AP would make it possible to federate instances of all sorts of different applications and have them all Just Work™

(Apologies if I've misunderstood your post, I'm kinda rambling at this point)

ActivityPub is a very loose spec, and obviously it doesn't just magically work across random services. You would have to implement support for the specific types of notifications for them to be meaningful. An example of this is PeerTube federation with Mastdon as seen here https://peertube.cpy.re/videos/watch/da2b08d4-a242-4170-b32a...

For example, GitLab projects could publish their activity feed to Mastdon. You could follow it to see what commits they make, issue discussions, and so on. Meanwhile, federating things like pull requests would happen across Git based services. So, if Gogs decides to implement compatible ActivityPub protocol, then it could integrate into the federation of GitLab servers.

Yes, federation would be great for git. I just hope that an open protocol is designed rather than something GitLab specifc. GitLab takes a lot of resources to self-host.

The current talks are about basing it on ActivityPub which is an open spec but of course it would only work on software which supports it.

Would be nice if Microsoft gets onboard for such a protocol. Especially once they own GitHub. Would also be funny if Microsoft Open Sources GitHub much to everyone's surprise.

I think it's important to clarify that even though GitLab's core is open source, its own business model is built on the idea of an enterprise version that includes proprietary code. Many useful "advanced" features, such as "squash and merge" are only included in the EE.

Interestingly, exactly the feature you mentioned is just being open sourced due to high community demand and it being free in its competitors: https://gitlab.com/gitlab-org/gitlab-ce/issues/34591#note_57...

I based myself on what is available on their "features" page:


In any case, this highlights once again the issue: the community will have to put enough pressure on the parent company to release certain improvements as Open Source. And their interests are most likely not aligned.

But doesn't the fact that the feature you mentioned has actually been made open sourced, that those business models might well be aligned? IIRC, GitLab's open sourcing policy is something along the lines of "if it's useful to small or open source projects, we open source it, whereas if it's primarily useful to large enterprises with >100 employees, it goes into enterprise edition". That sounds like a pretty viable way to both have a useful open source project and raise the money required to build and maintain it.

If they don't it can always be forked though. I think that's why they eased up - they realised that GNOME would just fork GitLab if they didn't release this issue.

GNOME already planned to self-host and have enough of a community to maintain a close fork.

I think the bigger picture here is code hosting. You cannot fork that, and TBH the value of moving from Github to gitlab.com is hosting alternative which is not really open-source.

Gogs is another great foss git alternative, and easier to run for small sites. We use it at ZeroTier.

I've also heard about gitea, which is apparently a fork of Gogs with the only difference being more maintainers.

Does anyone know of any more substantial differences? Which should I choose?

I've setup gitea for myself. Both a fairly similar, just that gitea doesn't have a single point of failure. Gitea has a feature comparison chat on their site[1] if you wish to compare the two directly. The fork arose from the single maintainer not working to give some control to the community.

[1]: https://docs.gitea.io/en-us/comparison/

That is a pleasantly honest comparison table, even listing the features they don't (at least currently) have. A refreshing change not to need a third party assessment for that.

You can see when Gitea forked from Gogs at


The following shows how active the two products are:


gogs was more of a one-person-in-command project, not sure its status now, gitea is more of a team work. I use gitea, works great all around, perfect for small to middle-size needs, totally resource light when you self host,a $5 VPS will serve well, or a small local VM


https://rhodecode.com/ Dual licensed AGPL / commercial

https://kallithea-scm.org/ GPL3 fork (pre-AGPL)

There is also Gitbucket - https://gitbucket.github.io/gitbucket-news/gitbucket/2018/05...

Now with LFS, code review, etc features. Built on the JVM - blazing performance.

All these alternatives seem to use Github as the main code repository. That's a really bad smell. Why don't they host their codebase in their own service?

Gitlab's main codebase is in Gitlab, though.

Yeah no, they really need another name. Bitbucket is known. Everybody talking about gitbucket will have to make it twice clear that it's not Atlassian's bitbucket.

there's a pinch of humorous irony in this website url

This is also an interesting read on alternatives if you want to host on a RaspberryPi: https://gitbucket.github.io/gitbucket-news/gitbucket/2017/03...

For me, the one thing keeping me using Github for most of my public projects have so far been discoverability. I use Gitlab for all of my private projects.

To the extent that I think really the only thing I'd want to see to not need Github any more would be some sort of federated social/discoverability/search layer similar to what you're describing.

> For people wondering what makes GitLab any different

nobody wonders that, as every week there’s a post on HN prising gitlab.

Federation would be amazing

How is GitLab funded? How real is their commitment to open source?

GitLab has taken VC money too. Including from a Google-connected investment group. There's no guarantee they won't be bought out etc. some day. They too are looking for a big exit for investors.

Their commitment to free/libre/open values is not 100% but is far more than GitHub. The fully-FLO CE part is truly a completely usable product. They have zero proprietary client-side JavaScript. They publish the source for their proprietary stuff (last I checked anyway) but have a restricted license. And they've actively worked with the FSF and volunteers to meet the GNU ethical repo criteria https://www.gnu.org/software/repo-criteria-evaluation.html

So, GitLab is not quite Mozilla (which is itself not quite Wikimedia or GNU level dedication). But GitLab is still a standout in FLO-commitment compared to the mediocre norm.

I wish it was Gogs that became this popular instead. Spreading Java and Oracle's influence even further is going to be a bad idea for the future.

Is GitLab Java based? How does Oracle feature in this?

GitLab is built on Ruby on Rails and a sprinkle of Go.

We have no relation to Oracle.

It doesn't matter when people are wanting the community and aren't wanting to host their own solution.

If you don't want to host your own solution, then just use GitLab's hosted service. It is free.

The whole point is that some people do want the control that comes with self-hosting, and GitHub does not provide a way to do that.

> The whole point is that some people do want the control that comes with self-hosting, and GitHub does not provide a way to do that.

But they do though. It might be significantly more expensive than GitLab and also only sold in packs of 10 user licenses and also not allowed to be run in a public-facing capacity. But they definitely do have a self-hosted option.


actually unless something recently changed, GitHub does, it's the enterprise edition and last time i checked it was like $10k a year. Yes, not reasonable for most people but it _is_ an option for people that want the control that comes with self hosting. Its feature set is also always lagging behind normal GitHub. The place i work used to use it (don't ask why). I think it's per-seat licenses but i remember it working out to ~$10k for us and we've got < 40 devs.

If they want self hosting, why are they importing to gitlab.com?

Easier and faster to evaluate gitlab.com and determine if they like the product or not before setting up a self-hosted version.

In fact I have been literally considering migrating our internal GoGS install to GitLab for the last week or two. The end of my day on Friday was downloading Gitlab and figuring out where to host an evaluation install.

Migrating my personal account over from Github to GitLab.com is a good chance to get some hands on time. Plus I can consolidate my personal CI setups at the same time, and I don't have to pay monthly for private repos.


PS: Always interested when your content comes up in my RSS reader. I do wish it was easier to share links without the unsafe content warning though. :P

That sounds like something just waiting for copyright abuse and its ensuing enforcement hell. "This GitLab instance has been shut down permanently by the FBI and ICE"

GitLab's recent performance has been abysmal. We recently moved from a self-hosted git solution to GitLab, and while the CI, 'namespacing' and issue tracking are truly great and well thought of, we've had entire days where the team was unable to deploy because the CI workers did not run (even though we host the workers), and therefore the artifacts for deployment were never generated. And nearly every day, pushes take minutes to complete, as opposed to a few seconds with GitHub.

If anything, I hope that Microsoft's acquisiton of GitHub means that GitHub is going to keep growing in features for varied enterprise uses, and that we're going to see even more competition in this area.

I'm sorry that you had a bad experience with GitLab.com self-hosted runners and pushes. I can't place the CI runners not working entire days. Pushes to GitLab.com should not take minutes. They do take longer then to GitHub.com and we're working on performance improvements, including deprecating NFS for Gitaly and more performant size checks that just got merged.

A big problem seems to be stability/error reporting and averaging of statistics. I've frequently had the following experience:

- I can't push or something in general goes wrong with one of my repos (but not others).

- Gitlab's status page is green

- Other people are having issues on Twitter and tweeting @gitlabstatus about it but there is not general across-the-board outage

This seems to indicate that Gitlab tolerates (and very often has) a reasonable amount of instability and error rates across its platform, but just takes the average of these as a baseline of performance: i.e. it's a very spikey graph with a reasonably high average line fit.

This tweet supports this impression:


"Errors should be down to normal" - the idea that there is an non-zero error rate that is openly described as "normal" is worrying. Not that I'd expect a constant zero error rate, but at least aiming for it should be a consideration.

It sounds like you've ever worked on a global scale service.

Services at this scale will have errors for all sorts of strange reasons, it doesn't mean the service is poorly engineered. In fact, if users don't notice these problems it usually means the service is resilient and robust when it encounters strange situations.

Consider a really simply example such as making a breaking API change to your service API. Now what happens when a user doesn't refresh their web browser and continues running javascript that doesn't work against new API. This can happen with smaller services but the odds of this happening are much higher when you are a global scale.

There are other strange problems that come with large services which means all components should be fault tolerant if possible.

You’re conflating two separate things: internal and user-visible errors. While it’s true that errors are inevitable, robust systems try to handle the latter gracefully with minimal disruption. If the person you replied to is accurately describing their experience a system which has significant unrecovered user-visible errors which aren’t acknowledged has serious robustness issues.

Also, please don’t make disparaging comments about other people’s experience unless it’s highky relevant. It doesn’t add anything to the conversation and will likely derail the conversation.

OP's post indicates that the metrics are poorly engineered.

As per the really simple example: generally you'd be better off rolling out a second endpoint for the new api and then stop serving responses that use the old one. First this doesn't break everyone who had your page up, and second you can stop rollout safely if you find a problem with the new api.

> Services at this scale will have errors for all sorts of strange reasons, it doesn't mean the service is poorly engineered.

Of course, and as I said, zero errors is not a practicably achievable in this type of context. The issue is with metrics though: the idea of taking averages instead of looking at troughs is still problematic.

> In fact, if users don't notice these problems it usually means the service is resilient and robust when it encounters strange situations.

True. But in the case of Gitlab, users are noticing these problems. Constantly. It's just Gitlab's own metrics that could be (I've not done more than browsed their Grafana instance a bit, so my comment is generally a bit speculative) ignoring the problems because they're focused on averages instead of specifics or thresholds.

> Consider a really simply example ...

lallysingh has already pointed this out, but I'll reiterate that this is a very apt bad example. You're right that ideally components should be fault tolerant if possible, but frankly that's a big ask. Especially for highly-scaled services supporting many many components of various types - ensuring that all of those components are completely fault tolerant is much more difficult than simply ensuring the old API continues to operate for a grace period while the new one is served from elsewhere.

I think your example is apt, because it's indicative of a common excuse for bad engineering: the assumption that downtime or disruption is necessary because of necessary software upgrades/improvements and poorly planned orchestration.

Do you publicly document your performance improvements? It would be cool to have a chart showing time to push or something, and let people see that trend go down as you are working on it. It would inspire confidence. Like others have said, you have had dealbreaking performance issues for a long time now.

I like your idea. However, few performance problems are global. We have a public monitoring dashboard at https://monitor.gitlab.net/. Embedded in this dashboard are various metrics which will often show a drop in response time if we improve performance on a particular item. We usually find a page or set of pages that hit a particular bottleneck and improve that one point. Also, you will usually see mention of specific performance improvements in the changelog (https://gitlab.com/gitlab-org/gitlab-ce/raw/master/CHANGELOG...) and in our release blog posts.

I'm getting intermittent 502's\Bad Gateway errors here on your Grafana dashboard.

Other comments further down are showing other's are too. Hacker News Hug of death?

It's not a great look.

To be fair this is probably the first time the page has been hit by HN / Reddit simultaneously...

Yea, I'm working on that, will be deploying this nice caching proxy to speed up the dashboard.

Thanks to Comcast for creating Trickster.


Never heard of Trickster till now, that's great.

Hope my post didn't come across as snarky as some others have... HN are like the Spanish Inquisition. No one expects.

Yea, I haven't used it myself, but the reports are that it works better than the original PromCache proxy. It's been on my TODO list for a while, but way lower on the priority list.

But you know, when the internet decides it's time for everyone to look at your site, some random new stuff might be better than serving 5xx all day. :-D

New HTTP error code:

HTTP 512 - Social Media induced DDOS due to media related frenzy :)

Good luck!

Anyone with a brain in their head understands this is because people are either considering or already moving to gitlab.

In this very minute my team is unable to deploy (and therefore accumulating blockings) because of issues with Gitlab. We have a plan on-hold to migrate off Gitlab (even though we just migrated to it!) and while I'd love to stay on Gitlab it's becoming very hard to justify.

Why not use plain Git? It's fast. And it's not difficult to build your own automation on top of it, e.g. using cron for nightly builds, etc.

Is there any documentation on Gitaly? I am exploring different filesystems and it will be helpful to learn about Gitaly.

Sorry to say it like this but you’ve been working on your performance problems for years now and you’re still at the same place. I think your problems run much deeper than that.

Their gitlab website is much faster than a year ago. A year ago I moved all my repos from GitHub to gitlab because I had to cut some personal costs. I remember it took a while to load pages when navigating around the site. A week or so ago I logged in for the first time in a Long Long time to setup a project to share with someone to test some ideas. I was surprised that I wasn’t waiting for pages to load. It was much faster than it used to be. Still room for improvement but I did notice it was much faster.

So while they still have improvements to make it would be a lie to say they haven’t improved at all.

Can confirm. The website is much faster and much nicer than it was a year ago.

Also, I don't think GitLab has had a long downtime recently. At least not for any of my projects.

Even with the influx of users due to this I was able to not only setup a repo, but push all code up, and deploy via GitLab CI all within minutes... Speed is very good. I don't notice a difference between it and GitHub.

> Also, I don't think GitLab has had a long downtime recently.

That mostly depends on whether you're using CI/CD I'd think, that's had some day-long outages/problems lately. Of course, GitHub doesn't even have it's own CI/CD, and GitLab's is amazingly flexible, so it's still the better product. But it'd be nice if it were more stable.

(Note: all this is on GitLab.com. If you self-host, it's presumably much better.)

I use GitLab.com's CI/CD extensively. I guess the downtime was when I was asleep or something because I've never seen a day-long outage.

I don't encounter all of them myself - it depends on what I'm working on and perhaps also the timezone. That said, April 26th was the most recent occurrence for me where I was very happy I wasn't in the middle of a production deployment that I would have had to roll back. See the status updates on that day on https://twitter.com/GitLabStatus

(I am using the free tier though, so this is more informative than that I'm complaining.)

Oh wow, it has been over a year since the GitLab database outage. That still feels like the other month to me. I'm getting old way too fast ...

I first tried migrating to GitLab when the public cloud first came out and abandoned it due to performance.

However, I re-valuated and did migrate about 2 years ago and it has been fine during that time. There have been a few hiccups, but not for more than an hour or so. I've had a team of 4-7 devs working in it all day for the last two years and we have not had performance problems. We run our own CI runners as well, and while the cloud runners do often have delays, I've never had issues with delays to my own runners unless they were all busy.


I love GitLab and it’s UI, but recently the performance of the hosted version is awful (not sure why - just being overloaded?).

In fact, even their own status page reflects it: https://status.gitlab.com/ - the current “project HTTP response time” is around one second which makes me cry when using the UI.

I wish them the best but would be moving to a competitor (or maybe a self-hosted GitLab) in the meantime until they sort it out.

Noticed this too recently, makes me more wary to import my projects there. Just browsing a repo is painfully slow. Although I've noticed the same on github for large source files or large repos.

A viable workaround is to browse the repo on your local checkout. It is decentralized git after all.

Ironically, the status page itself takes 5-6 seconds to load!

> we've had entire days where the team was unable to deploy because the CI workers did not run (even though we host the workers)

I find it boggling that a commercial team chooses to accept this kind of external dependency. What do they offer which makes it worth the extra risk?

It reduces the complexity of your ops environment. Not the OP, but we do the same thing (though not with GL). When you only have a couple of developers, it makes sense to keep everything in house because your cost is essentially a couple of hours keeping things up and running as well as having an extra development machine somewhere. When you are a large organisation it also makes sense because you have a whole bunch of ops people keeping things running. Somewhere in between there is an awkward point where you've got enough complexity that you'll need to hire an ops person to handle it, but you don't have the organisational infrastructure to deal with that hire. Outsourcing is actually less risky because you're essentially piggy backing on somebody else's large organisation. A single bad hire isn't going to sink you, for example.

What's the alternative? Writing your own CI/CD system from scratch? You're going to be relying on some external dependency for important things anyway, you just have to pick one that is dependable.

I'd think a happy medium would be using an open source CI system and testing new versions on a test server before deploying them to prod.

Then again, I come from a largely non-web background where external dependencies aren't just accepted as inescapable. I guess if your entire business is producing an add-on for some other company's web service (not saying yours is but many out there seem to be) then what's one more on the pile?

that's exactly what their CI is. An open source CI system that you can deploy on your own server (and plug to either gitlab.com or your self-hosted instance of Gitlab).

The focus is on long term freedom here, not occasional performance issues. People who are migrating projects right now to GitLab are implying that Microsoft will push the site's policies in unwanted directions. And the frogs who did not jump out in time will be boiled to death in the slowly heating water.

> the frogs who did not jump out in time will be boiled to death in the slowly heating water.

That’s a myth: https://en.wikipedia.org/wiki/Boiling_frog

Irony is that the page this post links to is 502’ing right now. Disappointing...

Sorry about that. The 502 was only on our public monitoring dashboard for a short time. GitLab.com itself is up and running and the monitoring dashboard is back online now.

Still seems down for me. Getting a 502

It went down again for a short time. We're continuing to monitor and adjust resources as needed. We weren't expecting this traffic to our monitoring dashboard, but it's great that so many people are interested in taking a look.

Is your monitoring page a recent build, is it scaling well?

I ask because it's not a particularly good luck that the viewers from HN are capable of bad gateway hug of deaths to the site?

It sounds like it's a separate, single instance. Definitely doesn't use the same infrastructure as gitlab.com itself (which is a good thing, since that's what it's monitoring), nor is it built to be scalable really. So, no great surprise that HN-level traffic overpowered the instance.

I do hope they're using Prometheus federation to expose this instance to the fickle internet and that they have one or more internal Prometheus instances that aren't directly queried by this instance. After all, that stuff is responsible for paging if something goes wrong in prod.

It is a separate instance from our internal one. We have a cron job that automatically copies the internal Grafana dashboards to the public one, so you still see exactly what we see.

We used to use Federation, but now we just have the public server scrape the same targets as our private one.

I'm adding a caching proxy (https://github.com/Comcast/trickster) to the public server now to improve performance. :-)


This link is for the monitoring page. The imports are going through.

To be fair the page worked fine until it was hugged by HN / Reddit...

I was using GitLab for about a year until a few months ago. The reason was sluggishness and how power-hungry it was. My Gitea server consumes considerably less resources and is faster.

Not to mention, it’s visually customizable.

Another performance issue is Sidekiq leaking memory, which the enterprise edition has a work around for but the community edition does not:


It appears that the workaround is open source, though, and is loaded in the open source version?



Many other reasons to be concerned about performance, but there's no evidence that they're withholding essential features like this from their free version.

Sorry, you're right, I misspoke. I should have said the CE version has the work around, but it's disabled by default.

The Sidekiq memory killer is enabled for both CE and EE by default with the Omnibus package. If you're seeing something different please let us know and we'll see what's going on.

That doesn't help in my experience, good idea though.

We are using it already in omnibus install.

The fact that it's acceptable to restart sidekiq instead of working on fixing the memory leaks in the first place is a perfect example everything that's wrong with software engineering in Ruby land.

Reminds me of the classic "The main Rails application that DHH created required restarting ~400 times/day. That’s a production application that can’t stay up for more than 4 minutes on average".

Adding to this, it now seems that monitor.gitlab.net that's linked in the top post is sending "502 Bad Gateway".

Sorry about that. We had to rescale the Grafana instance due to increased traffic and hit a snag. It's being actively worked on.

We use a self hosted GitLab, it is really fast and stable. Also the CI is very reliable and fast.

Ironically, the OP currently just waits at spinners forever for me, perhaps because so many people are trying to look at the graphs from the HN post (although one would think that page would rely on cached info and easily scale...)

How recently did you switch to GitLab?

A lot of this traffic is probably due to the opportunistic advertisement on GitLab's home page for non signed-in users: https://i.imgur.com/ZNikKcJ.png

The irony of this whole thing being that GitLab is hosted on Azure...

Post from 5th April 2018 on their blog says:

"We’re already in the process of migrating GitLab.com to Google Cloud Platform."[1]

So looks like they wont be on Azure for much (?) longer?

1 - https://about.gitlab.com/2018/04/05/gke-gitlab-integration/

Edit: Found this super-interesting architecture overview: https://about.gitlab.com/handbook/infrastructure/production-...

I think that the topic 'privacy' isn't the main factor in this case. Most stuff on platforms like GitHub and GitLab are public anyway and shouldn't include any private stuff.

In my case i simply want to move since i really dislike microsoft and their business decisions. I simply don't want to be involved in any "direct" way with this company. Microsoft doesn't have much control over GitLab, just because they use their servers. But Microsoft will have a lot of control over GitHub.

> Most stuff on platforms like GitHub and GitLab are public anyway and shouldn't include any private stuff.

GitLab offers free private repos. I expect that's a factor in many people choosing it.

That's fine, although IMO anyone that doesn't like Microsoft's decisions over the past few years is living in the past.

Meanwhile Google is GitLab's largest investor, and I've been burned more times to count. Google Wave? Google Wave?

One move from Microsoft that really disappointed me recently was to prioritize advertisement over user privacy, for example in Windows 10. I read big chunks of the terms and conditions myself and was horrified.

There are many articles covering this, for instance: https://www.eff.org/deeplinks/2016/08/windows-10-microsoft-b...

Investor != owner. Very different in terms of control and influence even if they are the largest investor. It'll be the Google VC and not corporate.

How were you burned by google wave? It was never a finished product and for the longest time (probably forever before being canceled) was a beta product.

You still have the source code. If big Google tries to take over you can always fork

I think the tweet https://twitter.com/gitlab/status/1003409836170547200 and the homepage update https://gitlab.com/gitlab-com/www-gitlab-com/commit/7c9e6703... happened around the same time. I think #movingtogitlab caused the traffic.

No, folks really are moving. I've imported about 2 dozen of my projects from github and my coworkers and friends are doing the same.

I just moved all of my repos over from github so actually I'm some of the problem haha. :)

The only people who will see that ad are those who have already decided to check out GitLab today. It’s a convincer, not a traffic driver.

Thanks for the image! But next time, please post a direct link instead of the JS blob that Imgur has become. <3

The user actually did. The website redirected.

imgur has figured out how and when it's safe to redirect PNG/JPG requests to a "JS blob" (of advertising), unfortunately. They tried to pull this a few months ago, completely bungled <img> embeds, and had to turn it off in a hurry. I think they've figured it out this time, sadly!

Time for a new image host... imgur has gone all high-level and "scale"-ey, it would seem (particularly with the new video with sound thing).

I was going to say something about toxicity, but this is sadly just a scaling problem. Now that sound - and competing with youtube - is the new "major consideration", just being a competent works-anywhere image host has been relegated to the region of rounding errors, so it doesn't matter in the same way if they get that right anymore.


Also, they're detecting that I'm on a mobile browser and forcing a redirect to a smaller version version of the file with _d appended to the filename. If this were a large file, I would not be able to see the full size without something like changing my user agent.

GitLab is awesome as a broke amateur dev with 0 way to really give back to GitLab the team was cool enough to send me a tshirt and stickers. Its come a long way since then but they seem to be awesome. I dont know if the msft acq is generally even a bad thing, but its great there are options either at GitLab.com or as a self-run service for people to change over to if they want. Congrats!

Thanks for the kind words. I'm glad you like the swag and enjoy using GitLab.

I highly recommend people consider self-hosting with phabricator. It's php-based, and if you have a ESXi or proxmox or digital ocean / linode instance, it's a very simple system to install & update, uses all the same LAMP stack everyone is used to, supports ssh/http based push/pull workflows, git,svn,mecurial repos, same type of authentication as github/gitlab/bitbucket, but biggest part of all, its PHP based, written to a very high standard through & through. Has the ability to scale insane amount of workers & cluster configurations. I've never had it once hiccup on me. Yes, if you import a project with hundred thousand+ commits, it will take a while to import, because it builds records for the entire commit history of the repo, it doesn't generate them the moment you view. So this will take time to back populate into the phabricator system, but once you have an imported repo, it has event system that is unlike any you may have ever used. It takes some time to get used to it, but organizations like CircleCI have a tutorial on how to integrate simple CI workflow using Harbomaster. https://circleci.com/docs/1.0/phabricator/ Also as of php7.x phabricator runs even faster than ever before. It also has it's own Trello Board style system, and it has a much more powerful issue system than gitlab/github. Tickets can have parent/children, and repos never specifically belong to any one project, but rather projects can have many repos. It also has integration with 2FA. It has a Question & Answers section, wiki, pre-code-commit audits. Lot of really big open source communities use it. A fairly large list of big companies using it are shown on: https://en.wikipedia.org/wiki/Phabricator

> but biggest part of all, its PHP based, written to a very high standard through & through

What?! Holy shit. Is that considered a positive thing these days?! I must be quite out of the damn loop.

PHP applications are extremely easy to install and keep running, and Phabricator is probably the cream of the crop in terms of code quality. The devs have done an incredible job with it.

So the answer is "it can be" considered a positive thing.

Well, there's nothing wrong with using PHP, and the performance is often better than with other comparable options. Personally I really enjoy PHP 7.

If you're way out of the loop then all you need to know is that PHP 7 is quite awesome.

You can write clean code in any language, and I've written some nice PHP, but the language itself has many fundamental design problems. For example, printf shouldn't have side-effects on a datetime object:


I run some OSS stuff on PHP, but in containers to keep it all isolated. Although things have gotten better with PHP7, knowing what I know about the language makes me hesitant to use it on any new project for anything except the most trivial systems.

That should definetly be fixed, but it's not something people will ever experience unless they try to write broken code by using undocumented features/sideeffects.

What exactly is it you know about the language that makes you hesitant to use it?

It's type system and automatic casting/comparing is a nightmare.

Although there are namespaces now, the idea of everything being in the global scope was insane (and still even with namespaces, much is still in the global scope).


There are no type-safe comparisons for greater or less then (You have ===, but no <== or >==).

This article: https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

This subreddit: https://reddit.com/r/lolphp

The number of bugs that are closed as not-a-bug/wontfix.

I'm not sure how much of this has been fixed, but I've moved on to different languages and career choices and don't really want to look back that way.

> it's a very simple system to install & update, uses all the same LAMP stack everyone is used to

I think that has not been true for a while now. Maybe it's just the world i live in, but putting a bunch of files into a directory on a vanilla LAMP configuration as a deployment scenario is not something i encounter a lot nowadays.

edit: specifically referring to the "everyone is used to" part.

> its PHP based, written to a very high standard through & through

I read small parts of the source code and it doesn't adhere to any of the modern standards set by the PHP community.


- The UI is lacking, esp. when viewing large diffs, compared to GitHub, VSTS-git, etc.

- `arc` CLI is awful

Batch editing a hundred tasks pops up a progress bar where you can watch it happen. I'm no MySQL fan but InnoDB shouldn't be that slow.

Can confirm, Phabricator is awesome!

GNOME turned down Phabricator in favour of GitLab partially because it is not easy to upgrade.

My own company is moving to GitLab - attempts to upgrade Phabricator have failed:

`git pull` is not a modern upgrade strategy - it ought to have an RPM.

Did you use a script like this one? https://gist.github.com/JosephKu/7a151e737c0a2de3eb3a6ec3c26... that is mostly what I do. Works flawlessly for me every time.

I'd like to point out that GitLab is not independent. They are owned by investors, including Google Ventures. So anyone who thinks they're giving GitHub/MS the finger by migrating their repos, please be ready to move everything again when GitLab disappoints you.

GitLab engineer here: our CEO has responded to exactly that concern in https://twitter.com/sytses/status/1003415290368028672 and https://twitter.com/sytses/status/1003337681668005888. To quote:

> We'll have a liquidity event. We're aiming for an IPO https://about.gitlab.com/strategy/#goals but we can't rule out an acquisition.

And it's worth pointing out that you can run GitLab on your own server, so even if GitLab.com ever ends up disappointing you, you don't need to go through the trouble of migrating to the next best thing.

Or then I could just host it myself using their open tech and everything would be fine again. :)

Google has done a lot of shit before, but I sure as hell trust them more than Microsoft at the moment when it comes to acquisitions.

Care to explain what? Satya Microsoft is day/night different between Ballmer MIcrosoft.

Meanwhile Sundar Pichai Google hasn’t been the friendliest.

Windows is still doing a lot of bullshit with ads in the start menu and forced updates. A bunch of their open source stuff which has included spyware has been less than upfront about including spyware.

I think what happened to Skype is a case in point. It doesn't matter who's running Microsoft, after that disaster I don't think anyone's going to be trusting them.

Skype has basically become unusable for professional communication. It’s getting worse with every update. I kind of expect the same for Github then. That’s my concern.

What about Windows 10?

This. Thank you for pointing this out. Also I have a hard time trusting the "community + proprietary" model...

They are actually advertising google cloud platform in self-hosted versions now:


Applications are open for YC Winter 2024

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact