
Jenkins 2.0 Beta - sciurus
https://jenkins.io/2.0/
======
teraflop
It's cool that they're promoting the "pipeline" plugin to a built-in feature,
but the devil is in the details.

Under the hood, it's implemented by taking a script written in a "DSL" (it's
actually Groovy code), transforming it into continuation-passing style, and
serializing its state. This is pretty cool from a theoretical CS perspective,
but having played with it a little bit, the implementation seems very fragile.
There are a number of long-standing unfixed bugs that cause the transformed
code to misbehave in ways that you wouldn't predict just from looking at it,
even if you're experienced with Groovy. I ran into a couple of them just
during a brief period of experimentation. For instance:

[https://issues.jenkins-ci.org/browse/JENKINS-27893](https://issues.jenkins-
ci.org/browse/JENKINS-27893) (varargs are mishandled)

[https://issues.jenkins-ci.org/browse/JENKINS-28277](https://issues.jenkins-
ci.org/browse/JENKINS-28277) (currying doesn't work)

[https://issues.jenkins-ci.org/browse/JENKINS-26481](https://issues.jenkins-
ci.org/browse/JENKINS-26481) (calling a closure in a loop doesn't work right)

[https://issues.jenkins-ci.org/browse/JENKINS-28183](https://issues.jenkins-
ci.org/browse/JENKINS-28183) (killing a pipeline job doesn't clean up its
state properly)

And there are inherent limitations to the approach; for instance, you can't
store any non-serializable values in local variables, which means simple
things like foreach loops don't work (because you can't serialize an
iterator).

I really like the idea behind the Pipelines feature, in principle. But I think
building it around a general-purpose programming language, and then failing to
support all of that language's features, is a recipe for headaches. If you
never try to do anything more complicated than what's shown in the examples
then you should be fine, but the borders between what works and what doesn't
are very ill-defined.

EDIT: Oh yeah, and there's a surprising amount of functionality that isn't
documented anywhere except for blog posts and JIRA tickets.

~~~
vorg
> Under the hood, it's implemented by taking a script written in a "DSL" (it's
> actually Groovy code)

It's Groovy code? The announcement page doesn't mention "Groovy" anywhere, let
alone "Apache Groovy" which is what the backers of Groovy are meant to refer
to it as since last November (2015) when it was promoted from the Apache
incubator by the ASF.

~~~
teraflop
Yep. You'd think this kind of thing would be considered significant enough to
mention, but apparently not. They do mention it in the "tutorial", which
AFAICT constitutes the entirety of the official documentation:
[https://github.com/jenkinsci/workflow-
plugin/blob/master/TUT...](https://github.com/jenkinsci/workflow-
plugin/blob/master/TUTORIAL.md#understanding-pipeline-scripts)

> A pipeline is a Groovy script that tells Jenkins what to do when your
> Pipeline is run.

------
jacques_chester
It's great to see that Jenkins is following the path blazed by GoCD[1] and
Concourse[2] to make the pipeline concept more central.

That said, this appears to be achieved by promoting the _plugin_ into the
default installation.

It also misses some of additional the advantage Concourse holds over Jenkins
and GoCD: build configuration is purely declarative and can be checked in with
the project. You know what version of your pipeline built the software at any
point in its history. And you have a reasonable shot at recreating that build,
because every task in every job is run in a fresh container.

These are killer features, in my view. Jenkins can be extended with plugins to
try to sorta-kinda do either or both, but it's not part of the central
organising concept of how it works. Windows can run some POSIX apps, but it's
not a nix.

Further out, the Jenkins pipelines are tricky to do fan-out/fan-in with; in
Concourse it's trivial. You have to lay out your pipeline in Jenkins,
Concourse will lay it out automatically based on declarative information about
each job. Rather than a very rich API for plugins, Concourse boils the unit of
extension down to "resources", which with three actions (check, get, put) can
model points in time, git commits, S3 files, version numbers, interacting with
a complex product repository and so on.

I used to _tolerate_ CI/CD, as a necessary and worthy PITA. Now I find myself
actively looking for regular tasks, resources, sanity checks and so on I can
put into Concourse, so that I don't have to remember them or write them up in
a wiki.

Disclaimer: I work for Pivotal, which sponsors Concourse development. But I
wouldn't rave about it if I wasn't such a convert.

[1] [https://www.go.cd/](https://www.go.cd/)

[2] [http://concourse.ci/](http://concourse.ci/)

~~~
mikehollinger
Yeah, version control of Jenkins itself has always scared me. There seems to
be a pattern that we go through.

(in the beginning, there was light...)

* Create a small, tight, single-purpose Jenkins job

* Add a small tweak to it

(repeat adding tweaks)

(realize the Jenkins job now contains MANY different configurations options
and the job itself is now a shell script in its own right)

* Sweep the "job" into a shell script. Check in said shell script

* Back up the Jenkins config, and hope no one asks why something's happened.

I now have a plugin that automatically checks in the Jenkins config to source
control, but it again doesn't solve the problem of matching up a particular
jenkins artifact to exactly what built it, and why.

~~~
wahnfrieden
We use Netflix's Job-DSL to keep Jenkins job configuration in source control
(and to allow easier reuse than offered with job reuse plugins).

[https://github.com/jenkinsci/job-dsl-
plugin](https://github.com/jenkinsci/job-dsl-plugin)

~~~
loevborg
I use [http://docs.openstack.org/infra/jenkins-job-
builder/](http://docs.openstack.org/infra/jenkins-job-builder/), which is
great as well: Jenkins configuration in a simple YAML file under source
control.

------
hidingfornow
I've been tasked with implementing a large Jenkins deployment to support a ton
of teams and I don't think I've hated a piece of software so much in such a
long time. The past few years I've been using other CI systems like Circle and
I totally forgot how much you have to fight Jenkins.

The UI is atrocious, job state is spread out among tons of crappy xml files,
and the plugin system causes tons of headaches. If you're going to have a
system that forces you to use the UI for the most part, rather than scripting
up config files that I can load with some automation, at least make that UI
nice to use.

Hopefully 2.0 is fixed up, but personally I'd never reach for Jenkins as a CI
system if it wasn't part of a client's requirements.

~~~
quacker
You should definitely check out Jenkins Job Builder -
[http://docs.openstack.org/infra/jenkins-job-
builder/](http://docs.openstack.org/infra/jenkins-job-builder/). It lets you
store job configs as nice yaml files which can be checked in to version
control. It's what openstack uses to manage jobs for the many many openstack
projects. I've been using for a year or so at work and highly recommend it.

~~~
hidingfornow
I'm not in charge of writing the jobs, just standing it up and migrating a few
dozen teams onto the system. Some are migrating from previous Jenkins systems,
so the majority have old jobs in config.xml files. The Job Builder makes
everything a little bit better, I've seen a few people using it already.

I'd still never use Jenkins on my own though, I'd just use Drone or pay
CircleCI.

------
zwischenzug
I wrote these posts recently about stateless Jenkins deployments:

[https://zwischenzugs.wordpress.com/2016/01/24/ci-as-code-
sta...](https://zwischenzugs.wordpress.com/2016/01/24/ci-as-code-stateless-
jenkins-deployments-using-docker/)

[https://zwischenzugs.wordpress.com/2016/01/30/ci-as-code-
par...](https://zwischenzugs.wordpress.com/2016/01/30/ci-as-code-part-ii-
stateless-jenkins-with-dynamic-docker-slaves/)

[https://zwischenzugs.wordpress.com/2016/02/25/922/](https://zwischenzugs.wordpress.com/2016/02/25/922/)

Does anyone know if any effort has been put in to make Jenkins more
'programmable' in 2.0? I had a quick look at the bumph recently but couldn't
see anything pertinent.

~~~
snockerton
I agree that the lack of an easy, declarative deployment capability is one of
Jenkins' biggest flaws and doesn't get near enough attention.

------
bb0wn
Very exciting -- a new major version of Jenkins has been a long time coming.

> "Jenkins 2.0 is a drop-in replacement of the Jenkins 1.x series of releases
> and fully backward compatible. There is practically no reason not to upgrade
> once 2.0 is released."

Skeptical, but optimistic about this claim.

I hope that the Jenkins configuration format is something that can be
manipulated more easily in the new version.

~~~
jowiar
I'll believe it when I see it -- the Jenkins ecosystem is one of the worst
offenders I've worked in with regards to things getting arbitrarily broken by
a point release. I shudder to think what a major release will do.

~~~
gtirloni
We look at the list of available plugin updates and cringe. You never know if
one plugin update will break something else, and then it's a game of
downgrade/upgrade until things mostly work.

~~~
slavik81
I don't update or install plug-ins without doing a taking a snapshot of the
Jenkins machine. A new plug-in version may force various other plug-ins to
update, and thereby break the many plug-ins that have incomplete dependency
specifications.

I also found that downgrade may not fix the problem, as the upgrade may change
your configuration files to use newer features, causing Jenkins to choke on
them once downgraded.

Jenkins is a spiderweb of plug-ins held together by shoestring. It's a very
useful tool, but extremely fragile.

------
Eduard
I hope Jenkins 2 will fix Jenkins 1's tendency for unclear and ambiguous user
interfaces. Many configuration input fields have weird names, and their
explanation fields can confuse me even more, and when using plugins, the whole
configuration layout can quickly become a mess - Jenkins 1 often gives me the
feeling of a mighty but hard-to-use Japanese pro tool lost in translation.

------
xvilka
Jenkins by itself is eating too much memory on our buildservers, which can be
used more wisely. Using compiled language could help that - e.g. Drone.io CI
written in Go: [https://drone.io/](https://drone.io/)

You can find it's sources here:
[https://github.com/drone/drone](https://github.com/drone/drone)

I just hope it would be able to work without docker, like jenkins.

~~~
Pyppe
Jenkins too is built on top of a compiled language. Too bad it is JVM, which
is a hog when it comes to memory... Point being that a "compiled language" is
no guarantee of a low memory usage.

~~~
xvilka
OK, I meant "compiled into native code" language.

~~~
outworlder
Still doesn't change much when said language can be JIT'ed.

------
sytse
It is clear that build stages should be first class citizens. We based GitLab
CI on the awesome work that GoCD and Concourse did in this respect. Our DSL
allows you to assign jobs to stages [https://gitlab.com/gitlab-org/gitlab-
ce/blob/43e49f52e30199c...](https://gitlab.com/gitlab-org/gitlab-
ce/blob/43e49f52e30199c9724329e71f375874eb76d554/.gitlab-ci.yml#L29) and
jenkins seems to have picked a similar syntax.

What is interesting is that Jenkins has a DSL but that this is not part of the
repository. This means that it is hard to extend when you introduce new
functionality and when you push an old branch it might not work anymore. I
think that the model that Travis CI pioneered clearly wins.

I'm torn on the plugins. One one hand it is great to be able to plug so much
in. But the plugins have access deep into the Jenkins internals, preventing a
rewrite of core functionality. Our idea is: "Because GitLab is open source the
enhancements can become part of the codebase instead of being external. This
ensures the automated tests for all functionality are continually run,
ensuring that plugins always work. It also ensures GitLab can continue to
evolve with it’s plugins instead of being bound to a plugin API that is hard
to change and that resists refactoring. This ensures we have many years of
quality code and great monthly releases ahead of us."
[https://about.gitlab.com/direction/#vision](https://about.gitlab.com/direction/#vision)

~~~
yamitty11
Almost all of what you said here is totally wrong.

>What is interesting is that Jenkins has a DSL but that this is not part of
the repository.

Netflix job dsl plugin, Jenkinsfile pipleline, dotci all of these let you put
a .travis.yml like file in your repo.

>But the plugins have access deep into the Jenkins internals, preventing a
rewrite of core functionality.

Plugins don't have deep access to jenkins internals they use clearly defined
"extension points" .

I know you are trying to plug gitlab but 'our stuff is better because we don't
allow plugins' is not exactly a plus. Jenkins is so popular because it can
customized via plugins to wide range of use cases.

~~~
sytse
Thanks for educating me!

1\. Cool that there are different plugins that let you check the configuration
in a dotfile.

2\. I was not aware they used extension points instead of something more
intrusive.

3\. Obviously I like GitLab and try to promote it. But I post this too to see
what others think and to learn. Turns out I had a lot to learn in this case. I
still think that some items in Jenkins should be part of the core, such as the
pipeline. But I agree that plugins are needed for some cases. For deploys
we're thinking about plugins for GitLab CI too in [https://gitlab.com/gitlab-
org/gitlab-ce/issues/3286#note_402...](https://gitlab.com/gitlab-org/gitlab-
ce/issues/3286#note_4027611)

------
Alupis
I've been using Jenkins for years, and have been overwhelmingly happy. 2.0 is
a big step for the project, congrats!

The flexibility Jenkins provides as build system is enormous. The plugin
community is also a huge benefit -- chances are, if you have a need, there's a
plugin for it.

~~~
derFunk
I feel the same. Using Jenkins since 5 years now on a daily basis. Yet I also
feel it may be coming too late. During the past 5 years our projects have
grown to a size were we're expecting to have more than 1m DAU end of this
year, and I don't think Jenkins will survive this journey with us. I'm
speaking about scripted Docker container baking & deployments to e.g. ECS,
ECR, asset uploads and versioning, basically release management. I'm looking
for modern solutions for the latter which is basically as stateless and
reliable as our product Docker infrastructure. I would dismiss Jenkins in case
I find an equally flexible but safer tool.

~~~
Alupis
Well, I don't know your specific circumstances, but I'll say one should strive
to have as little logic in their build system as possible.

It essentially should be running shell scripts and/or language-specific build
scripts/tools. All logic should be self-contained in your project(s) repos, so
that it doesn't really matter what build system you use, and/or one could
build the project without the build server if required.

With that said, I don't see any reason why release management, containerizing,
versioning of assets, etc... could not all be done with Jenkins (or any build
system).

------
code_research
Every random piece of software expects some kind of file in the root directory
of a project - this is not acceptable and leads to messy project layouts.

Could developers of these tools please stop dictating where people have to put
the "Blahfile" dsl config files and finally allow people to configure these
kind of details.

Also I would like to propose a common standard directory "projectroot/config"
for these kind of files, NOT the project root directory itself.

Thanks for your attention!

~~~
kbody
"projectroot/config" it's too common and already used in different ways that
will produce conflicts. If something is established it should be done in a
proper way through a spec like semver was established.

~~~
code_research
you could avoid conflicts by not naming your project files "Dockerfile" or
"Jenkinsfile" or "travis.yml"...

~~~
mkobit
I don't like the Dockerfile or Jenkinsfile either, but I can kind of
understand the .travis.yml. The good thing about the "Dockerfile" is that with
most of the docker commands that interact with it, you can use the '-f' option
to specify the file. For example "docker build -f
dockerfiles/application.dockerfile" -t myapplication .". I don't see any easy
way to do that with the Jenkinsfile.

The amount of top-level files in these projects has started to become quite
large, and it is unclear at first glance to see what it is important when
exploring a new ecosystem.

I'd like to see these products would move in a direction where these were
would do any/some of the following:

1) use a file extension rather than "important" describing filename (.docker
for Dockerfiles and .groovy/.jenkins for Jenkinsfile)

2) moved into a product specific folder (travisci/ci.yml, github/ for the
issue and pull request templates)

3) use yet another top level file for indirection that is agreed upon (my
initial thought is something named integrations.yml)

------
xrstf
Google cache: [https://webcache.googleusercontent.com/search?q=cache:Sm-
mng...](https://webcache.googleusercontent.com/search?q=cache:Sm-
mngGjeo8J:https://jenkins.io/2.0/+&cd=1&hl=de&ct=clnk&gl=de)

------
mschuster91
As the website got HN'd, a question: does Jenkins support now pass-through of
parameters in pre/post build actions to other jobs?

Background: I have "deploy frontend" and "deploy backend" job, both of which
need to invalidate some layers of caching, and a job parameter that specifies
the environment (it's the docroot).

For now I have a metric shitload of shared code between the jobs as there is
no way to move the cache clearing code to its own job and pass the environment
parameter through in pre/post build actions.

~~~
zzzeek
I can't imagine what you couldn't accomplish using the Parameterized Trigger
Plugin [https://wiki.jenkins-
ci.org/display/JENKINS/Parameterized+Tr...](https://wiki.jenkins-
ci.org/display/JENKINS/Parameterized+Trigger+Plugin)

~~~
mschuster91
Cool, thanks for that. I'll mail this one to our infrastructure team, but it's
sad to see that this isn't installed by default.

~~~
zzzeek
how does it being an optional plugin change anything vs. Jenkins having every
possible useful thing pre-installed resulting in a much more complicated setup
screen? Plugin architecture is total win IMHO.

~~~
mschuster91
Parameters without the ability to pass them on is kind of pointless, imho the
right solution is to provide a single "parametrization" plugin.

------
dkarapetyan
That's nice but even the current version of Jenkins is buggy as hell and it's
been around for a while now. Every place I've been at takes the base Jenkins
worker management and then layers routing and job orchestration on top of it
through something else because Jenkins itself just can't deal with anything
that is beyond triggering and canceling a job. Even canceling doesn't work
half the time.

So given that's the foundation they're layering even more complicated stuff on
top of it. Thanks but I'll pass.

------
jquast
Having been using jenkins several times since it was Hudson, as well as
Bamboo, travis-ci, python buildbot, and TeamCity, Jenkins is a loser from my
perspective. Not worth becoming invested in.

Its simply too difficult to use by a laymen developer or sysadmin when
accommodating complex buildchains, which only grow more complex over time in
most software businesses.

If you're going to program your CI with a web administrative interface, and
provide an easy to navigate interface for users, expand with build metrics and
external systems, I suggest TeamCity over any others.

Though jenkins is free, my experience is maintenance teams of 2 or more people
with tribal knowledge develop around the tool that far outweigh the license
cost of TeamCity, which solve many of these homegrown engineered problems as
part of the product cost.

If you have a strong developer team, I would suggest using something like
python buildbot and programming your CI end-end in a single unified language,
much better than shoe-horning plugins and groovy code inside your jenkins
textarea fields!

That said, all CI systems are just software, and any requirement can be
accommodated with sufficient amount of additional software, forks, or changes.
Any will do.

~~~
nod
If we were hypothetically "too big" for TeamCity, with scaling difficulties
all over, do you have (or have heard) any recommendations? I hear the biggest
companies all roll their own, but is there anything between TeamCity and
Google-scale?

~~~
zabilcm
There's [http://go.cd](http://go.cd) if you are looking at something focused
on CD (Pipelines etc)

~~~
fathom108
Yes, we've found gocd does a terrific job of allowing you cleanly model your
full continuous delivery workflow and organize your servers.

------
needusername
Jenkins always struck me as one of the few software projects that don't use
OSGi but should be using it.

------
donatj
I've become a big proponent of Drone CI. It's very configurable and Docker
powered.

------
foolinaround
Is there a potential release date, so that we could plan for the adoption of
the beta for future releases?

------
glasz
ugh. this is still alive? i was done with jenkins the first time i had to use
it.

since everbody's at it, i'll recommend github.com/drone/drone. drone really
gives docker a pretty use case.

------
smegel
If there was ever a webpage that needed a facelift, it would be Jenkins, so
hopefully this is not just a backend rewrite.

~~~
Alupis
Take a look at the Usability Section[1]

Looks like a massive facelift, and lots of work done to make things more
intuitive, faster, and more pleasing to the eye.

[1] [https://jenkins.io/2.0/#ux](https://jenkins.io/2.0/#ux)

~~~
ChristianBundy
Unfortunately, the new design still looks almost a decade old.

~~~
hueving
So does hackernews. If it actually addresses the usability problems, it
doesn't matter how 'old' it looks.

~~~
Alupis
Couldn't agree more. Jenkins is a tool - it doesn't need to look "uber
modern", but it needs to be functional (which it is).

------
t0mk
I came here to write sth like "still in Java... :/" but I can't even check it,
the site seems to be hugged to death.

~~~
Alupis
> I came here to write sth like "still in Java... :/"

And the problem with that is...?

~~~
FooBarWidget
Typical issues surrounding Java projects:

\- High startup time. Anything bigger than a hello world takes seconds, more
typically tens of seconds, to start. My Jenkins setup takes 3 or 4 minutes
before it is fully initialized. Even starting the JVM in client mode doesn't
help much. I know, it's a server, and startup time doesn't matter once it's
running, but it _feels_ annoying.

\- High memory usage. Heaps in the 1-2 GB range are not uncommon. This can
probably be tweaked but as a user who is not a Java expert, figuring out how
to do this (and figuring out what value is safe) is frustrating.

\- XML configuration files. XML was hip in 2004 but these days it is frowned
upon and users generally don't like it. This is technically not Java's fault
(and I'm sure newer projects use something else) but it is something
traditionally heavily associated with Java projects.

\- Non-Unixy feel. Lots of things just behave like they're not really native.

~~~
mark242
"Heaps in the 1-2 GB range are not uncommon."

Is this really still a complaint in 2016? Your phone has that much RAM.

Honestly if you're worried about Java heap size, spin up a t2.medium instead
of a t2.small for your build pipeline. Running Jenkins on a Raspberry Pi is
probably not a great idea if you're writing any serious application.

~~~
ascotan
Actually it's becoming more of a complaint in 2016 with rise AWS instances
that pay by the container size and with the rise of languages like go, etc.
which have smaller runtime sizes and are more compatible with cloud computing
environments like docker.

The JVM adds something like 400MB to every layer of a docker container
resulting in multi gigabyte containers. Additionally the memory requirements
for running a JVM put you immediately into the t2.medium range which will cost
you 4X cost over a t2.nano for simply choosing to run Java instead of
something with a smaller runtime footprint.

~~~
mark242
Don't run production code on a nano.

Don't run production code on a nano.

Don't run production code on a nano.

I don't care what language you're using, if you find yourself saying "well, I
could run on a nano" just stop.

