
GitLab 10.6 released with CI/CD for GitHub - rbanffy
https://about.gitlab.com/2018/03/22/gitlab-10-6-released/
======
eslaught
Gitlab's CI/CD is amazing. Of all the systems I've seen, it's the easiest to
(a) deploy your own runners easily, and (b) mix private and public runners
(i.e. the equivalent of using a free plan on Travis, with the benefit that for
heavy-duty jobs you can run your own machines). That and first-class support
for docker (no more being at the whims of what Travis chooses to deploy in
their images), make it a really great system.

But we had been on Github, and that turned into something of a pain.

First we tried mirroring Github to Gitlab. This worked, but (a) it added a
minute or two of latency to starting jobs in the CI, (b) since it went through
one account, all the build emails went to a single person, and (c) the
mirroring happened on our own infrastructure, which turned out not to be so
reliable. So we got tired of this approach.

Eventually we decided to move development to Gitlab, but we didn't want to
move our users, so we left all the user-facing elements on Github (issues,
PRs, etc.). This also kind of works but has its own set of issues.

Things might have been different if we'd started on Gitlab from the very
beginning, but as things are, I think having the repository on Github and the
CI on Gitlab would be the best of both worlds for us.

~~~
eslaught
Edit2: This is wrong, see replies below.

An update, having now tried this on a toy Gitlab/Github instance.

It looks like this still relies on Gitlab's support for mirroring an external
repo, which relies on polling. This is less than awesome, because it means
we're back into the (many) minutes latency range from pushing changes to
seeing them appear in the mirrored repo (and therefore CI). When I did this
before, we had to set up a mirror on our own machines to get this latency down
into the ~1 minute range, otherwise it was unusable.

What appears to have been added was syncing the build status from CI builds
back to Github. That part seems to work as intended.

There does appear to be a Gitlab API for triggering a mirror update, but I
don't see an easy way to get that hooked up to Github out of the box. The CI
status feature would be much more aweseome with the addition of push-based
mirroring.

Edit: And the problem with mirroring manually like we did before is that there
isn't a clean way to mirror pull requests (it requires manipulating web APIs,
not just pushing a Git repo). So if we went back to that route, we wouldn't
really get the seamless experience we've been hoping for.

~~~
williamchia
Hi Eslaught - So the new CI/CD for GitHub feature should be solving the pain
points of the old manual mirroring process. When you set up a project and
Oauth to GitHub it should automatically implement 3 components: \-
Bidirectional Mirroring [https://gitlab.com/gitlab-org/gitlab-
ee/issues/3745](https://gitlab.com/gitlab-org/gitlab-ee/issues/3745) \- push
webhook to GitLab to trigger CI/CD immediately once a code is committed (e.g.
"push mirroring") [https://gitlab.com/gitlab-org/gitlab-
ee/issues/4691](https://gitlab.com/gitlab-org/gitlab-ee/issues/4691) \- GitHub
project service integration (to webhook CI status back to GitHub.)
[https://gitlab.com/gitlab-org/gitlab-
ee/issues/3836](https://gitlab.com/gitlab-org/gitlab-ee/issues/3836)

Is this not working for you? (may be a bug)

Did you set up the toy instance by creating a CI/CD project and use the GitHub
Oauth?

~~~
Snappy
FWIW, if you created the connection between GitLab and GitHub for your project
before 10.6, you'll have to remove the connection and then re-add it to get
the push-based mirroring to happen. Otherwise, yes, it will still poll.

~~~
eslaught
How do you do this?

~~~
williamchia
It seems like you should be able to do this, but there may need to be an
update to work with already-mirrored repos since the feature was designed to
have a flow that expects a new project to be created.

I've logged an issue to track down the process: [https://gitlab.com/gitlab-
org/gitlab-ce/issues/44552](https://gitlab.com/gitlab-org/gitlab-
ce/issues/44552)

------
ninkendo
Since "X adds support for Y" is nearly always ambiguous and confusing:

It looks like GitLab's CI/CD system can now be pointed at GitHub repositories.
So that means you can point your GitLab CI system at a GitHub repo and get
build/test/deployment of your code, probably integrated with PR checks and so
forth.

~~~
vortico
The original title was "GitLab adds support for GitHub". Now that GitLab
supports GitHub, I'll go to GitLab for all my support questions about using
GitHub!

~~~
tmuir
Pssh. Gitlab was obviously just tallying the individual sentiments of whether
the users were generally in favor of Github continuing to exist as an
organization. How could you have missed this?

------
chinhodado
I hate it that Gitlab is focusing too much on all these project-management
features, while neglecting the basic code review features such as commenting
on a block of code, or making a review session instead of creating individual
comments, or the ability to blame previous revisions of a file

~~~
andrewstuart2
It's open source. I'm sure they wouldn't turn down a PR if you submitted one.
I've poked around the codebase a few times; it's not too hard to contribute,
despite my lack of experience with ruby.

~~~
gtirloni
It's _open core_ , not open source. I'm pretty sure they would have a lot of
incentives to turn it down.

~~~
jobvandervoort
We don't have a lot of incentives to turn down contributions. Quite the
opposite.

We not only encourage this, we actively help people contributing to GitLab.
It's humbling to be a steward to an open source project and something we don't
take lightly [0].

And to be blunt, for an organisation as ours, with very ambitious goals, and
highly demanding users, having someone else than us contribute something that
makes the product better is the best possible deal we could do. We can
compensate for the 'lost' value in our enterprise product by spending that
time on other things that are maybe more interesting for our paying customers.

[0]:
[https://about.gitlab.com/stewardship/](https://about.gitlab.com/stewardship/)

~~~
gtirloni
That's reassuring, thanks for replying. I was commenting from a general
perspective on open core licensing but I understand not all companies are
created equal.

------
deedubaya
I really want to try GitLab, but I can't log in to their service with oAuth
using Google as a provider. I get a 422. I brought this up with GitLab's
support, and they essentially told me it was my fault and doing something
wrong with JS or something. It's just me clicking on links on their page, wtf.

The interaction with their support was enough to turn me away from their
service if that's the type of help they provide.

~~~
locnguyen
I ran into this recently. You probably created a GitLab account with your
Gmail address first and then afterwards tried logging in with OAuth and
encountered the 422. Try logging in without OAuth, then enable social sign in
under [https://gitlab.com/profile/account](https://gitlab.com/profile/account)

~~~
deedubaya
Unfortunately, it's not the same problem. I get the same results from multiple
gmail/gsuite addresses. womp womp

------
joeblau
Will this be the Trojan Horse that gets me to switch to GitLab? I've been
reading a lot of great things about GitLab's CI/CD. They also have cheaper
plans for developers and if everything is all integrated into one place, that
would make setup for OSS projects that I maintain like gitignore.io really
simple.

The main thing that is keeping me on GitHub is the community. GitHub still
feels like the main place where developers go to find open source projects.
They've done a great job at pulling all of the major corporations like Apple,
Microsoft, Google, Netflix, Amazon etc... While that shouldn't really matter
because it's just a code repository, some how it does.

------
Thaxll
I'm curious for the people that use GitLab CI/CD, what's the pro / cons vs
Jenkins and the like?

~~~
diggan
A short while ago (eh, just noticed this was over a year ago...) I did a
comparison between different testing solutions for all projects within the
IPFS organization. Remember this was a year ago, so many things have changed
but we ended up going with Jenkins and I'm now migrating projects from free
usage of Travis/CircleCI to Jenkins.

\- Brief explanation:
[https://github.com/ipfs/infrastructure/issues/100#issuecomme...](https://github.com/ipfs/infrastructure/issues/100#issuecomment-265439892)

\- Spreadsheet comparing features:
[https://docs.google.com/spreadsheets/d/11kTloM6d1CGu_ycmP3Xz...](https://docs.google.com/spreadsheets/d/11kTloM6d1CGu_ycmP3XzpteGpJtij-h6q07anM5ZuGs/edit#gid=0)

Check out those links and you should get a good overview

~~~
smulc
Nice summary. Did you test support for handling multiple changes to a single
project in parallel? The only CI Platform I've found that tries to handle this
in any meaningful way is OpenStack Zuul ([https://zuul-ci.org/](https://zuul-
ci.org/)) and
[https://docs.openstack.org/infra/zuul/user/gating.html#testi...](https://docs.openstack.org/infra/zuul/user/gating.html#testing-
in-parallel) for details of the parallel support.

Do others have this need? Or are people generally focusing on keeping CI runs
short and landing changes in a serial fashion?

~~~
diggan
No, this was not a consideration, as we often have so many PRs open at one
time, that this is probably not feasible. If I understand the process
correctly, it would merge each PR in order from the first made until the most
recent one. So the most recent one includes all the proposed changes. For
ipfs/go-ipfs, that would involve doing 104 tests + builds for each PR.

Rather, we test each branch as it was merged into master (in isolation), and
when master changes, trigger builds for PRs that now are changed.

The "Cross Project Testing" and "Cross-Project Dependencies" is something
we're deeply in need of though, and already do somewhat. The dependencies are
implicit now though (via npm or our golang package manager, gx) rather than
explicit.

If anyone have ideas for "cross project testing with dependencies" on Jenkins,
I would be very happy to hear them.

------
Avery3R
Both a sticky header and a sticky footer... This reminds me of the IE6 days
when most of your screen real-estate was taken up by toolbars.

------
jordanpg
Related Jenkins announcement: [https://jenkins.io/blog/2018/03/19/introducing-
jenkins-x/](https://jenkins.io/blog/2018/03/19/introducing-jenkins-x/)

------
baconomatic
This makes a lot of sense to me. Gives Github users a taste of Gitlab's CI
capabilities.

~~~
wyldfire
Note that today you can use the public hosted GL service and mirror a GH repo,
and trigger GL CI when new commits arrive.

But this sounds even smoother still.

------
lobster_johnson
I've never used GitLab, and I'm curious if it's something my company could
use.

Almost all our Github projects use Docker. We don't use automatic deploys at
the moment, but CI basically consists of building a Dockerfile, then running
tests with "docker-compose run", then pushing the image to GCR.

We do have some projects that consist of multiple dockerfiles, but they're
outliers. Everything else follows the same convention.

We currently use Semaphore, which has several downsides:

* Every project needs to be created and configured individually. We have many, many small projects. On Semaphore, setting up a project requires clicking through 4 or 5 screens to configure. I know Semaphore has an API (two incompatible ones, actually), but that means we have to write code for something that should be included in the box.

* To alleviate this burden, we have a single shell script, hosted with HTTPS and basic auth, that every project runs to build. It does things like log into GCR, set up secret build args, then do the build. Again, stuff that's identical across all projects. It's a bit silly to fetch this script every time rather than have it be shared across the whole organization.

* Semaphore's Docker layer caching (achieved with docker pull + docker build --cache-from) seems inconsistent. Some builds are fast, some don't get any caching.

* Semaphore is terrible at describing test failures. You'd think that a CI system would be great at doing things like show "4 failures/2 skipped/123 successful" on some kind of board. But instead you just get success/failure, and the entire build log. If the test is something like "go test", then you get 99% noise, and it's hard to zero in on what the failure was. I'd love to get the actual failing test in the Slack notification.

We've tried a few alternatives:

* Self-hosted Drone: Great, but too immature when we ran it, and development seems slow, with the organization behind apparently focused on a SaaS solution?

* Google Cloud Container Builder: Better, but requires each project to be "mirrored" within Google Cloud, which is a pain, and the mirroring is flaky. Also, no layer caching. And apparently not appropriate for running Docker Compose tests, and secrets management requires more tendrils into GCP, whereas we prefer things to be ore self-contained.

* CircleCI: Similar problems as Semaphore. Seemed a bit messy.

* Jenkins: Nobody likes Jenkins, apparently.

Based on the above, would Gitlab be a better fit?

~~~
kkapelon
> Jenkins: Nobody likes Jenkins, apparently.

That is not a good reason for not using Jenkins. You should try to see if it
covers your needs and if it does, you should not care about what other people
think about it.

Also you can discover more options here:

[http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-
com...](http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-
comparison.html)

------
jerrac
If any GitLab people are still watching this thread, has there ever been any
discussion on offering subscriptions to specific EE features? Instead of a
full per user subscription.

I work at a small community college and we have a self hosted instance of
GitLab-CE. There's no way we could budget anything for a subscription (as much
as I'd love to support GitLab's development.) So, if I could tell my boss that
we could get a needed feature for x-dollars/month, that might actually be
doable.

I was one of the people asking for the GitHub -> GitLab-CI integration. So I
am fairly disappointed it ended up in EE instead of CE.

~~~
kgradin
Thanks for your comment. We would be happy to discuss options with you if you
could submit a contact request by following the link here
[https://about.gitlab.com/sales/](https://about.gitlab.com/sales/) we will
reach out to you.

------
kuschku
Finally!

I'm running a custom GitLab instance, and I have most of my projects in there,
but I have to have some of my projects on GitHub, and having the same CI
workflow available for them is going to be amazing.

~~~
gtirloni
It seems this feature is only available in the Premium plan, if you're hosting
your own instance.

It's free if you're using GitLab.com with a public GitHub repository.

[https://about.gitlab.com/features/github/](https://about.gitlab.com/features/github/)
\-- "Large Enterprises"

~~~
kuschku
Oh fuck that. I’m a student using this to host open source projects, I’m not
going to pay more for GitLab than I pay for all my servers and software
combined.

I’ll just hack it in there myself then, like I had to put some of the other
"premium" features in myself.

Long-term it looks like I’ll have to migrate to something else then.

~~~
williamchia
Hi Kuschku - I think this is misunderstanding, we are offering GitLab CI/CD
for free for GitHub.com projects. It'll be free for private repos for a year,
and it will be free for public repos forever. See the "Open source projects"
and "Anyone using GitHub.com" sections:
[https://about.gitlab.com/features/github/](https://about.gitlab.com/features/github/)

~~~
kuschku
According to

> For self-hosted installations, GitLab CI/CD for GitHub is available for
> customers with Premium and Ultimate license plans.

It’s not.

~~~
williamchia
Edit: I see clearly now in the top-level comment you mention self-hosting

Ah, my apologies, didn't realize you were self-hosting GitLab vs using
GitLab.com. Yes, this is the case for self-hosted instances it is part of the
Premium feature. On GitLab.com is is available for free.

------
larrywright
Not really related to the story, but I just saw that you can run Gitlab on a
Raspberry Pi. Has anyone done this and care to share how well it works? Seems
like a nice way to get a Git server for the homelab.

------
IshKebab
This is great, although I'll probably stick with Travis & Appveyor for my open
source project since Gitlab.com currently only has Linux runners.

All three currently have two big flaws IMO though:

1\. They all use YAML which is an awful, unintuitive format.

2\. The only way to test a configuration is to commit it and push it. Why is
there no way to paste a config file in their website and run that?

Hopefully one day we'll get a CI system that supports Windows, Linux and OSX
and has a sane config format, and free minutes for open source projects. A man
can dream!

~~~
y4mi
Ive got mixed feelings about yaml as well, but which format would you prefer?
The Jenkins DSL ist terrible to debug and XML is way too verbose for such a
usecase.

Yaml is honestly the best option here.

~~~
IshKebab
TOML is much better.

~~~
y4mi
i don't know any CI/CD framework which utilizes it though.

and i'm unsure if it really is. There are some imo really good features
missing such as document separation, cross references and the newline to space
conversion.

toml would be great for pretty much everything thats under /etc/ though.

------
kazishariar
Is this available for Stash/Bitbucket as well?

~~~
mryall
Bitbucket Cloud PM here. We'd love to see more usage of Bitbucket (both Cloud
and Server) by GitLab users.

As williamchia mentioned above, it's possible to configure GitLab builds for a
Bitbucket Cloud or Server repository via webhooks and our build status API
today. We're open to helping build better integration, but given the APIs
already exist, it would mainly be a project on the GitLab side.

~~~
joshfriend
Is there a way to migrate a repo from Bitbucket Server into gitlab _preserving
pull request history_?

~~~
brodock
There is a lack of APIs from bucket to allow that kind of operation. You
should consider the lockin each platform have, before jumping in

------
piahoo
what makes it different than ordinary webhook ?

~~~
jobvandervoort
In short:

1\. We mirror your project

2\. We set up the integration with GitHub

3\. We report the status and link back on GH's pull requests

------
f2n
Please fix this to point to the original article, not this blogspam.

[https://about.gitlab.com/features/github/](https://about.gitlab.com/features/github/)

~~~
akerro
I'm not sure if you noticed but this "blogspam" is literally a release note,
as says in the title " GitLab 10.6 released", support for GitHub is one of
many features.

~~~
bronson
You're commenting on the changed link. The original was techcrunch blogspam.

------
sctb
We've updated the link from [https://techcrunch.com/2018/03/22/gitlab-adds-
support-for-gi...](https://techcrunch.com/2018/03/22/gitlab-adds-support-for-
github/) to this announcement.

------
avardar
We used big(6gb) git repo on gitlab. I lived to many problems and moved to
gogs now we are hapy with gogs. It is much more lightweight. Just try gogs and
gitlab.

~~~
nik736
The headline is that GitLab adds CI/CD support for GitHub and you are writing
about migrating to Gogs which has not nearly as much features. Gogs is a
lightweight git hosting solution, so basically the opposite of GitLab. I am
not saying Gogs is bad, in fact I use it myself but you are comparing two
completely different things. If you only need a simple git solution, you can
use Gogs, if you need more and like the features GitLab has you don't have any
self-hosted alternative.

You are also not writing if you use their SaaS solution or if you've installed
it on your own servers.

