
CERN migrates to open-source technologies - tdhttt
https://home.cern/news/news/computing/migrating-open-source-technologies
======
whatshisface
Public funding of open source software is one of the most direct ways that
government spending can grow the supply side of the economy. Taxpayers will
directly benefit from the public domain technologies developed to replace the
contracts CERN is moving away from. Software and science share an important
economic trait, that once you produce them you can maximize their total impact
by spreading them around as much as possible. That's an advantageous situation
for public funding without IP restrictions, which is the model of academia.

~~~
Nightshaxx
Counter point: the more the government gets into funding OSS, they will
essentially become a competitor to other software companies. Now the
government both has the power and incentive to hinder private companies with
rules and regulations.

I think that if the government needs OSS software for actual use by the
government, then that's fine. Don't make the government become a competitor to
private industry. Look how that turned out with Uber vs. Taxis.

OSS is just like anything in the free market. If people and companies want it
that much, they will spend time and energy on it.

~~~
usefulcat
> Now the government both has the power and incentive to hinder private
> companies with rules and regulations.

What incentive? In the scenario you describe, I don't see an obvious mechanism
for the government to receive additional tax revenue as a result of funding
OSS, and that's the usual mechanism for such an incentive.

I mean, I guess you could argue that tax revenues will tend to increase as the
economy grows, but that seems like a case where the interests of the
government are aligned with the interests of the nation(s) as a whole, which
is generally considered a good thing.

------
cernguy
Disclaimer: I worked at CERN in IT for 5 years as staff.

CERN has never been a Microsoft organisation excepted in the head of
management that was pushing for Microsoft solutions everywhere without
success.

\- Scientific computation are done at CERN under Linux with the Root
framework.

\- Most (all?) scientists uses OSx or Linux.

\- All computing clusters runs SC Linux or centos.

\- Most internal softwares: indico, EDH, landb, and other are running under
Linux and are web based.

\- All DBs are OSS or oracle

\- All storages system are home made (EOS, Castor) or ceph based and run under
Linux.

\- Data distribution is home made and based on a framework named xrootd under
Linux.

\- Software distribution is also Linux based and run as a fuse module
(cernVMFS)

\- Most systems services are UNIX C++ and Java for the control part.

\- CERN uses Openstack for virtualisation After the management pushed for
Hyper-V and failed miserably.

\- Management pushed for SharePoint for years before the entire website
switched to PHP and Drupal.

There si not real "Microsoft" at CERN excepted AD and phones. It is however a
study case of bad management decisions in IT.

~~~
l0b0
I'm not proud of it, but I worked on SharePoint at CERN IT. Some examples of
how dysfunctional it was:

\- When asking about version control I was told that if I really wanted to use
it I could have Visual SourceSafe 6.0 or something from way back in the
nineties. It was basically CVS 0.1 alpha with a GUI, and I ended up learning
Git instead.

\- I made sure all my changes worked on Firefox, because that's what I knew
the physicists were using. My boss wanted me to support IE only, and after a
heated discussion the quote which always haunted me was "We don't care about
the physicists!"

\- A colleague lamented about having spent about a year developing some uber-
flexible interface in SharePoint which would've been an order of magnitude
simpler in other content management systems.

The rest of CERN is completely different, as indicated by cernguy.

I left a year into a two-year contract, and I'm happy to say I've not worked
on a Microsoft platform ever since!

~~~
cernguy
> The rest of CERN is completely different, as indicated by cernguy

Yes, that is also a thing at CERN.

Many other teams re-develop things internally (Linux based) because they could
not trust or rely on IT management to do the right things.

That had a very perveted effect of internal duplication across teams and
experiments.

------
Polyisoprene
Since people don't read the article:

"The Microsoft Alternatives project (MAlt) started a year ago to mitigate
anticipated software license fee increases. MAlt’s objective is to put us back
in control using open software."

"A prime example is that CERN has enjoyed special conditions for the use of
Microsoft products for the last 20 years, by virtue of its status as an
“academic institution”. However, recently, the company has decided to revoke
CERN’s academic status, a measure that took effect at the end of the previous
contract in March 2019, replaced by a new contract based on user numbers,
increasing the license costs by more than a factor of ten. Although CERN has
negotiated a ramp-up profile over ten years to give the necessary time to
adapt, such costs are not sustainable."

~~~
budu3
Wow. Heads will surely roll once this reaches Satya's desk.

~~~
passer_byer
...or the person gets promoted.

I previously worked for an enterprise analytics software provider whose
success hinged on low-cost license agreements to schools and universities.
Once the company became more successful commercially, it took the same tact--
substantially raising these fees in search of profitability. Today, the firm
is under serious pressure from open source analytical libraries. The point
being, what looks to be optimal in the short run may not be optimal in the
long run, as future consequences could be anticipated but short term revenue
growth was too tempting.

~~~
mkl
> same tact

"Tack", from sailing: changing direction lets you use the wind differently.
Two boats on the same tack are using the same wind strategy.

My workplace currently teaches students several proprietary systems that are
now old and clunky (and of course expensive). We want to switch to open source
competitors, but the how and when are rather complicated to coordinate. It
will happen eventually though.

~~~
slim
he meant tact

------
panpanna
CERN had a major role in making kicad what it is today. they implemented tons
of complex stuff that you normally only find in really expensive CAD tools.

Hope they can repeat that with a few more FOSS projects!

~~~
sigstoat
and you can donate to KiCad development via CERN:
[https://cernandsocietyfoundation.cern/projects/kicad-
develop...](https://cernandsocietyfoundation.cern/projects/kicad-development)

~~~
turbinerneiter
Aisler.net, a German boardhouse, makes a donation to kicad for every kicad
designed pcb you order there and it's my favourite way to pay for open source.

They literally get paid the moment value is created from using their software.
It's beautiful.

~~~
lnsru
You aren’t right! Aisler doesn’t donate. The user can choose to donate.

From [http://www.kicad-pcb.org/about/kicad/](http://www.kicad-
pcb.org/about/kicad/) “That is why AISLER allows its users to easily donate to
the KiCad project during the ordering process.”

~~~
turbinerneiter
True.

The donation is an item on the bill, they automatically set it and the user
can change the amount.

------
duxup
I worked support some hardware CERN used once. Fun guys to talk with / work
with.

They call me up one day to note that one of their devices I supported had an
accident and they were concerned they didn't know why. So they sent me photos.

This device had rows of modular cards installed in it. In the center of the
device with two cards pulled out you could see that something had burred and
even melted some of the surrounding cards. But it didn't look like any given
card had failed as much as there was some sort of really hot fire ... that had
been in the air between the cards or something. Now keep in mind this was just
what a handful of photos looked like, so who really knows. Makes no sense that
there was something floating in the air between the cards hot enough to do
that thing ... but that is what it looked like.

Anyway it was like a good 100k+ in hardware burned up, possibly MUCH more as
the full chassis held a lot more than that. So I promise them a new chassis
and such and tell them to pack it up nicely and we will have a courier come
and get it and send it to our QA team. The CERN guys promised not to expose
the equipment to anymore micro black holes ;)

The process to send stuff to the QA team in strange situations like this was a
painful series of steps. The QA team was BRUTAL about process (even if they
never followed it themselves...). They also were a real pain to even email,
but that was part of the process. Amusingly enough when I sent them the photos
and explained it was CERN even the QA ultra dry guys cracked some good X-files
references ;)

Still wonder what the hell happened to that equipment.

------
sytse
It is awesome to see how CERN is supporting open source. They have been long
time users of our open core GitLab with 12,000 users
[https://about.gitlab.com/customers/cern/](https://about.gitlab.com/customers/cern/)

~~~
jobvandervoort
I visited CERN once in my role at GitLab. The scale of CERN and the level of
people there is really impressive.

Many teams had implemented very complicated pipelines doing all sorts of
things. Including using GitLab to design, validate and eventually produce
hardware that was used in ATLAS.

------
Beltiras
I'm waiting for some of these documents being made public. I am working for a
government institution that is making much of the same moves. Would be nice to
see what an immense IT undertaking like this comes up with.

~~~
Boulth
Are your org's documents public? I'd be excited to read about the practical
effects of such deployment (from CERN too).

~~~
Beltiras
Ours is a more ad-hoc process than that. I work for Statistics Iceland. We are
not changing OS or productivity suite (yet at least). Those that want Linux on
their laptops get Ubuntu. We deploy Libreoffice alongside Office. Users using
other statistical suites are encouraged to use R. On the infrastructure side
we are making more use of Samba (one new DC, next up is migrating fileservers
from old WServers over to Samba on Linux), just replaced Exchange with Kopano.
We are an org of about 120-150 people/workstations.

~~~
ethbro
How's the LibreOffice / MsOffice compatibility in practice?

Was thinking about this yesterday, and was curious how much functional
coverage or issues there are with what people actually use it for.

~~~
psalminen
Not the OP, but I tried to do a lot of work in Libreoffice Calc on XLSX files.
I had a constant issue of Libreoffice not being able to handle >10000 rows,
and actually crashing when trying to do something intense like vlookup.

~~~
a3n
I worked QA at a place that had huge CSV log files for a device. Sometimes the
log files were slow to open in Excel, or just would not open at all.

I started using Jupyter and Pandas for my own work, just so I could get
something done. I really liked it.

But the organization was moving even more in the MS direction, I'm sure any
thought of Pandas died after I left.

Try something like that if you can.

------
dgudkov
>...increasing the license costs by more than a factor of ten

When I hear stories like that I always wonder what was the thought process of
the sales people that managed that account. Probably something like that:

\- Hey, do you remember our old customer CERN, a world-famous scientific non-
profit organization that pushes the boundaries of human knowledge about the
universe?

\- Yeah, what about them?

\- Let's charge them 10 times more for our software licenses

\- Can't see anything wrong about it, go ahead

------
newaccoutnas
I think this was posted at the time. It's worth saying that CERN have been
massive open source users for decades

~~~
pletnes
It’s also worth noting that CERN has been running (not just using!) many open
source projects since way back. Geant, madx, root and cling come to mind.

------
gen3
Also see the comments last time this was posted:
[https://news.ycombinator.com/item?id=20166070](https://news.ycombinator.com/item?id=20166070)

and what I said last time:

I'm honestly pretty happy about this. I'm hoping that by aiming to replace the
commercial products they use with opensource alternatives, the alternatives
leave with a better polish and user experience. I also see CERN as an
institution that is willing to hire the devs needed to maintain / support a
project.

------
crb
While this post is referring to desktop/server software, CERN are also using
open source platforms in some of their experiments: they recently re-performed
some of the 2012 analysis that led to the discovery of the Higgs boson, on top
of Kubernetes.

Adam and I spoke to a computer scientist and two physicists from CERN for our
podcast, and you might like to listen if you like physics or software.

[https://kubernetespodcast.com/episode/062-cern/](https://kubernetespodcast.com/episode/062-cern/)

------
kazinator
That's kind of ironic. I was doing paid contract work in 1994 (on Linux!) that
involved customizing the open source CERN httpd server (before Apache existed,
or loadable modules in a web server).

------
MayeulC
Matrix would be a good fit for most research institutions, allowing
cooperation trough federation. Heck, even Mastodon could be interesting, and
ActivityPub as a transport layer for publications.

However, one issue is that small institutions cannot afford to self-host
everything, or make the necessary adjustments themselves. I wonder if CERN and
other big institutions could perform some heavy lifting (and maybe provide
some shared services, hosting servers, etc.) without necessarily centralizing
everything like it is done nowadays.

~~~
secfirstmd
I'm a big Matrix fan but am curious about what options they have for
integrating and managing it in a typical enterprise environment. For example:

-what happens for stuff like user management with Active Directory etc? -what happens for things like FOIA requirements considering it is e2e encrypted?

~~~
roblabla
Matrix doesn't E2E encrypt everything, only certain rooms. It's not currently
possible, but I suppose there's no technical roadblock to making a server
prevent access to E2E encrypted rooms to meet FOIA requirements.

As for user management, synapse has identity servers and password providers to
form a complete authentication solution. mxisd[0] is a service that uses both
to offer LDAP authentication, although it seems dead with no real replacement.

[0] [https://github.com/kamax-matrix/mxisd](https://github.com/kamax-
matrix/mxisd)

~~~
Arathorn
So Matrix has a bunch of enterprise integration support either in existence or
in the making.

In terms of audit compliance for E2EE, there are three main options:

a) Turn it off (as per the parent post); there isn't a button for this in
synapse but would be possible to add, albeit technically an abuse vector given
it is effectively a downgrade attack.

b) Add an audit user to rooms which need to be 'on the record'. This is our
preferred solution, as it makes it crystal clear to users as to which
conversations are on the record (and whose record!) and which aren't. One
could run such a user via a client like [https://github.com/matrix-
org/pantalaimon](https://github.com/matrix-org/pantalaimon), and have the
server autoinvite them into rooms which need to be recorded.

c) Add an audit (aka ghost) device to users who need to be 'on the record'.
For instance, you could use pantalaimon to log in as a given user and record
their messages. The audit device will appear in the E2E devices for the user,
and once cross-signing lands, could be signed by the user (or their sysadmin)
to be trusted. However, we're not keen on ghost devices in general - we've
built all of Matrix's E2EE trust model to protect users against unexpected
devices being present in their rooms, so we'd recommend audit users instead.

In terms of LDAP integration - there are more and more enterprise integration
options appearing; for instance, ma1sd is a maintained fork of mxisd, and
we're working on better LDAP bridging for Matrix in general. We want Matrix to
work in an enterprise environment, so if people see stuff missing, please
yell.

------
josephagoss
Does this mean that no scientist at CERN can use applications like
Mathematica? Or is this just focusing on essential foundation software like
operating systems (moving away from Windows)

~~~
lokimedes
While some rely on Mathematica most at Cern uses ROOT for data science. The
impact on the science program is more likely to come from ditching Oracle
databases and LabView... That said, this is an IT department initiative, users
also have access to software from their home institutes.

~~~
deven88
Is there any reliable and good opensource alternative of LabView ?

~~~
cycomanic
I would argue any dynamic programming language is a good alternative to
labview. I'm a proponent on using python for labautomation (we run workshops
at some major optics conferences, see
[http://python4photonics.org](http://python4photonics.org) for an
unfortunately somewhat out of date summary about some of what we do). In my
experience labview is a mess, it's only momentum (in particular drivers
provided by vendors, but that is changing) that keeps it going. I find labview
code developed by amateurs is an unmaintainable mess. Grad students pass along
code from on generation to the next without anyone daring to fix some of the
glaring bugs. Really use anything but labview, even if it's the buggy matlab
instrument toolbox.

~~~
mhneu
Exactly. Labview is a mess anytime more than one person has to maintain a
codebase. Source control tools are absolutely KEY to using any language for
automation. Which mostly rules out Labview.

Do you have any more workshops/hackathons scheduled?

~~~
lavezza
I know I'm late to the discussion, but I've worked on a very large LabVIEW
program with ~10 active developers and source control was extremely important.
But I'm talking about SVN with the option to lock files. If by source control
you mean "distributed, file merge-based" source control like git, than I would
agree there aren't good options for LabVIEW. But a central repo with file
locking, small changes and LabVIEW graphical diff worked fine.

------
sh-run
I'd like to see more done in the open networking space. Cumulus Linux was
talked about a ton just a few years ago, but it seems like I haven't heard
about it as much lately. Probably because Juniper and Cisco have (mostly)
managed to catch up on the automation front.

I couldn't find anything concrete on the CERN website, but in their virtual DC
tour I noticed HP Procurves as top-of-rack switches. I was really hoping to
see whitebox switches.

------
nullify88
I came across a white paper years ago (2012/2013 ish) on how they used Icinga
(An early but rapidly growing Nagios fork) and mod-gearman to monitor their
ATLAS computing farm. At the time, as a newly appointed sysadmin, I was
looking in to possible solutions that could scale well myself and their usage
was very inspiring. Eventually deploying something similar where I'm still
working now.

[https://cds.cern.ch/record/1455464/files/ATL-DAQ-
PROC-2012-0...](https://cds.cern.ch/record/1455464/files/ATL-DAQ-
PROC-2012-014.pdf)

------
ridaj
Good idea but

> Needless to say, isolated initiatives will waste effort and resources.

If the centralized procurement approach is what put everyone in the current
mess, what indication is there that a centralized approach to open source
won't produce the same issues?

I personally wonder why commit to open source in a centralized manner vs
commit to interoperable standards (but not the specific tools used to speak to
the standard).

~~~
cat199
jury probably still out, and no direct insight, but the overall
incentives/philosphy driving acquisition for each could theoretically be
drastically different (e.g. optimizing for openness, community, xyz instead of
vendor size, ability to negotiate discounts, etc)

------
Apofis
E-mail and ip telephone migration to start... doesn't seem like a big deal but
they have been with Microsoft for 20 years.

~~~
lokimedes
When I worked there, they had 10k+ active user accounts and a wealth of
systems integrated into Microsoft active directory. The nature of CERN also
means that multiple institutes around the world will have to adapt their
systems when CERN changes.

------
linker3000
Is there a public-facing list of project activities?

------
raxxorrax
I suspect this is specifically targeting Windows. I think this would be a good
idea for anyone really. I don't know whose or which decision brought MS into
this mess, but this isn't a sustainable OS for the future.

Our company still relies on it, but any new piece of software has to be
platform independent. Licensing income probably skyrocketed on MS end, but I
don't think they made any friends with W10.

Maybe MS is correct that W10 will be the last Windows. But maybe not because
of rolling releases.

Anyway, good news and the correct strategy in my opinion.

~~~
zucker42
As long as schools use Windows, it won't die.

~~~
cwyers
Businesses aren't using Windows because schools use it (I think Chromebooks
and iPads are more popular among schools now than Windows machines because
children can trash a PC easily). They use it because of a long tail of
software applications that only run on Windows, they use it because of stuff
like Group Policy and Active Directory, they use it because of a lot of things
that Linux doesn't do as well.

~~~
nicoburns
Don't forget MS Office. That's microsoft's real crown jewel, and it only runs
on windows, or much more expensive macs.

------
shmerl
Are they using Linux instead of Windows already?

~~~
dukwon
There's a mix of linux, windows and mac OS machines, depending on the use-case
and/or user-preference. This won't change.

------
user5994461
CERN is not a university and has no student, it's entirely justifiable for
Microsoft to not give them a university discount.

Universities have thousands of students that do not generate any revenue. The
standard licensing model per person or per computer totally breaks down in
that case, asking for a huge bill based only on the sheer amount of users.
Education discounts are adjusting for that, large user bases with no money to
pay.

That being said. The CERN is a government entity. They should argue to be
given government discount.

~~~
dukwon
> Universities have thousands of students that do not generate any revenue.
> The standard licensing model per person or per computer totally breaks down
> in that case, asking for a huge bill based only on the sheer amount of
> users.

There are thousands of students, postdocs and academics with CERN computing
accounts (way outnumbering employees). It's not like they're generating
revenue either.

> They should argue to be given government discount.

They did, IIRC, but their status as an IGO didn't cut it with Microsoft.

~~~
user5994461
IGO:
[https://en.wikipedia.org/wiki/Intergovernmental_organization](https://en.wikipedia.org/wiki/Intergovernmental_organization)

Honestly, it looks like the terms and licensing are being dictated by some
dude(s) from Microsoft in the US, based on US norms. The way
academy/university/research is organized and operated in Europe is fairly
different. Bet Microsoft doesn't mind the added money either.

If the CERN were playing by US rules, they would be suing Microsoft to be
recognized as a government or academic entity.

------
williamxd3
That moment when left-pad creates a black hole on earth

------
lone_haxx0r
Wait. Why the fuck is there a cern TLD? Are ICANN naming TLDs after specific
organizations now?

~~~
dukwon
Basically yes:
[https://icannwiki.org/Brand_TLD](https://icannwiki.org/Brand_TLD)

I agree with the move away from .ch, since it's an international organisation,
but .int would have been the correct choice IMO

~~~
lone_haxx0r
I expect nothing, and I'm still let down.

------
lowlevel
My gut says this is related to forced updates and unwanted interface changes
in current Microsoft products. It's been super annoying where I work, and
staff are pretty upset.

~~~
cwyers
I mean, they list a pretty compelling reason from their point of view:
Microsoft changed their licensing terms from academic to business licenses,
which are more expensive.

I suspect they will discover over time that the fully loaded costs of
switching are higher than the costs of the licenses, unless this is just a way
of trying to leverage MS into giving them academic licenses again.

~~~
nullspace
They say that the licensing costs increased by "by more than a factor of ten".

Given that figure, do you really think that one-time project implementation
and switching costs amortized over the next X years + ongoing support would
really come to more than 10x? How do you make such a strong claim that easily,
any prior examples?

Not saying you're obviously wrong or that CERN is obviously right, but this
seems to be a much harder call (and intuitively feels right in fact), given
the number.

~~~
cwyers
There was a push in Germany to migrate to Linux, and they eventually gave up
on that, so yeah, there are some prior examples.

[https://www.techrepublic.com/article/end-of-an-open-
source-e...](https://www.techrepublic.com/article/end-of-an-open-source-era-
linux-pioneer-munich-confirms-switch-to-windows-10/)

[https://www.theregister.co.uk/2018/07/27/lower_saxony_to_dum...](https://www.theregister.co.uk/2018/07/27/lower_saxony_to_dump_linux/)

~~~
tasuki
Do you know why they gave up? Did Linux not work for them? Or were they
persuaded by the MS sales team?

~~~
fock
I guess it's a little bit of both. First mistake they made was that they
tasked local contractors (think O(1) people) with forking their very own linux
distro from Ubuntu. Then they put a lot of ressources into the KDE3-fork which
was made to look like Windows NT (because they were migrating from that in
200x and you know the secretaries for life...). In the end they wondered why
all their application software was old and buggy and I suppose from the whole
project there was no significant contract work upstreamed (think LO
performance patches...). Also being 1% of the market apparently does not
entice any competition in the public software (think access-frontends in
VB6@1000€s per place) market.

Given this, some years ago, the mayor changed, Accenture did a 'cost' study
and MS moved its german HQ to the city.

