Is anyone here using the Gitlab CI/CD for anything a bit more complex?
I'm especially interested in a pipeline with parallel executions of different kinds of tests, maybe some manual checkpoint in there, a more complex chain with execution on different kinds of hosts or containers.
We're got some pretty complex pipelines, so much so that we've reached quite a few limitations.
You can do all the things you've listed. But you can't build an arbitrarily complex DAG of tasks. Instead, you must order everything into a sequential list of 'stages'. Within a stage you have a list of jobs, which are run in parallel. This might be enough for your needs, but I'd like flexibility to express dependencies for each and every task, Say `t1 -> ((t2,t3),t4 -> t5) -> t6`, etc.
And it's a pain to propagate variables from one task to another, like say setting an environment variable in one task, and getting its value in a downstream task. You end up writing scripts to write them to a file, telling gitlab to make that file an 'artifact', and then reading it downstream.
And then there's the transient errors, all of which are listed on open issues. We're using the hosted gitlab.com, and on some of our CI pipelines, we have a dozen or some stages. The chances of a transient error, .e.g. a runner getting a 503 from gitlab.com, are pretty high.
No, you can define jobs and stages. Stages run one after the others and all jobs inside the same stage will run in parallel, that's it, unless you start triggering remote jobs but then it becomes a lot let maintainable.
Last time I checked (a few weeks ago), Gitlab's CI/CD system is great for most use cases as long as you can describe your pipeline as a list of non-interactive tasks -- some of which may be parallelized -- that each can run in a Docker container.
For one of the software projects I'm involved in, I have more complex needs. We release binaries for multiple platforms so our Jenkins master delegates certain tasks to slaves running on specific OSes. Then at the end the Jenkins master downloads the built artifacts from all slaves and publishes everything to our artifact hosting server. As far as I can tell, Gitlab CI does not support this.
In future CI jobs I may even require user interaction, e.g. I may ask a human to sign off a report. I don't think Gitlab CI can do this.
But if your needs aren't so complex then Gitlab CI is great. It's UX-philosophically similar to Travis and setting it up is super simple, as opposed to Jenkins which is a pain to use.
> For one of the software projects I'm involved in, I have more complex needs. We release binaries for multiple platforms so our Jenkins master delegates certain tasks to slaves running on specific OSes. Then at the end the Jenkins master downloads the built artifacts from all slaves and publishes everything to our artifact hosting server. As far as I can tell, Gitlab CI does not support this.
GitLab CI/CD can certainly run jobs on different OSes, then have a final job consolidate those artifacts and publish to an artifact server. What part can't GitLab do?
> In future CI jobs I may even require user interaction, e.g. I may ask a human to sign off a report. I don't think Gitlab CI can do this.
GitLab CI/CD has manual jobs so a pipeline can wait for human sign off before proceeding.
The issue is that the configuration format makes complex jobs difficult to define. It's good for the simple case, but versus e.g. a Jenkinsfile[0] where you get a complete groovy DSL that can interact with it's environment across tasks, pipeline, branch, etc with ease it's rough. Having fine-grained control becomes important.
You could achieve something like this with tags - https://docs.gitlab.com/ee/ci/yaml/#tags. Though you will need to define each machine individually in the CI file.
The pipelines for Flockademic are somewhat complex, and since it's open source, you can take a look at them: [0]. It's building four interrelated Node projects at the same time, running tests, deploying the back-end projects, then building the front-end project linking it to the back-end one, then running some end-to-end tests and other checks on the deployed code. And it does that for every branch.
No manual checkpoints and different kinds of hosts or containers though.
CNCF has a pretty complex CI/CD system called cross-cloud where we're deploying half a dozen CNCF projects on half a dozen different clouds. It's using GitLab and is all open source.
I'm especially interested in a pipeline with parallel executions of different kinds of tests, maybe some manual checkpoint in there, a more complex chain with execution on different kinds of hosts or containers.