
Winding down my Debian involvement - secure
https://michael.stapelberg.ch/posts/2019-03-10-debian-winding-down/
======
jcoffland
I've been using Debian for over 10 years and I still love it as a user. But as
a developer, I find it extremely frustrating. I've several times attempted to
figure out how to package my open-source projects [1] [2] for Debian but the
process is a nightmare. As I understand it, I first have to find someone with
appropriate privileges to mentor me. I should be able to just submit a
potential package for review.

Then there is a ton of documentation on creating packages but which 300 page
guide is the right one to use is unclear. And which set of packaging tools
should I use? In the end, the time investment required to get started has kept
me from contributing.

[1] [https://camotics.org/](https://camotics.org/)

[2] [https://foldingathome.org/](https://foldingathome.org/)

~~~
billsix
Agreed, I've tried and failed to make Debian packages, and have no idea how to
proceed.

In contrast, with Gentoo and their documentation, it was straightforward for
me to make my own additional, separate repo, look at other ebuilds to see how
packaging works, and have everything just work.

[https://github.com/billsix/billsix-
portage](https://github.com/billsix/billsix-portage)

I submit ebuilds to Gentoo itself if I think others would benefit.

As far as I can tell, packaging for Arch is straightforward as well.

~~~
bamboozled
NixOS was also easy to get started with, I really like it !

~~~
nextos
NixOS (Nixpkgs actually) is indeed incredibly simple to contribute to.

It's a monorepo, and packages are declarative. So for most usecases, its just
5-10 LOC describing the source location, dependencies and build process.

After that, if it builds on your local copy of Nixpkgs, it will build on the
same Nixpkgs commit as everything is purely functional. A simple PR is all you
need.

For updates, there's super low burden too. Thanks to packages being
declarative, everything is quite explicit. So a bot can check for source
updates upstream, update your package and rebuild it. All it requires is
simple maintainer approval.

------
nh2
In my experience, better tooling always wins in the long run.

I've built Debian packages in the past, and after packaging the same software
with Nix, it's very hard to not feel that Debian tooling and packaging is time
consuming for no good reason. The nixpkgs model, where all you need to build
everything is one `git clone`, all you need to make a change is a pull request
and wait for a range of automated tests to tell you it's good (for both build
and the _uses_ of a package!), and all you need to land a contributed change
is click one button, seems strictly better.

Over the years I've heard many people say "but shell scripts, FTP servers and
arcane helper tools is the way we've done it for decades and that will never
change" in many projects, but eventually, these projects shrink and those with
a good developer experience and clean tooling overtake them.

Similarly, after experiencing automatic, safe refactoring across billion-line
typed-checked code bases, you can't help but wonder why people put up with
spending their time going through heoric community efforts distributing work
across people that a machine could easily do if you used good tooling.

In my opinion, the real strength and legacy of Debian is successfully running
a large, diverse, distributed project over decades with (reasonable) cohesion,
democracy, and (reasonably) good organisation and project management.

But even some non-technical problems go away with good tooling, and more time
gets freed up to solve those hard tasks.

Concrete example: In nixpkgs it is very easy to build overlays for the whole
of NixOS that allow you to switch from dynamic to static linking or add
hardening flags across all packages, avoiding big debates over which is the
"one true way" because providing both is so easy, and both can be merged
upstream.

I think any big and successful project should continuously invest into better
tooling, and simplify and automate things. That keeps contributors motivated
and on board.

(14-years happy Ubuntu [and thus Debian] user, and 10-years i3 user, so thanks
for your efforts, Michael.)

------
Boulth
> I have more ideas that seem really compelling to me, but, based on how my
> previous projects have been going, I don’t think I can make any of these
> ideas happen within the Debian project.

Interesting article. I had a chance of asking two distributions about some
particular feature (making it easier to get GPG keys of developers).

The first one I contacted was Gentoo: they quickly CCed my email to relevant
people, discussed the matter between themselves and deployed the change in a
week.

Then I contacted Debian about the same thing. The email was basically
identical. But the reply was largely negative, complaining about details and
openly avoiding work. The entire interaction reminded me of large corporations
where any change is met with resistance for resistance sake.

(I use Arch btw.)

------
justinsaccount
I used debian for over 10 years and never managed to contribute anything.

10 Minutes after using homebrew for the first time I sent in a PR to update a
package to the latest version and update the built dependencies.

~~~
nothrabannosir
Honestly, as both a Mac and a Debian user, I’m not sure which one I prefer. I
understand your frustration, but random strangers not being allowed to push
updates to my operating system in 5.34 seconds doesn’t sound all bad. To me.

(Obviously not that TFA paints a rosy picture..)

~~~
giovannibajo1
Those changes are reviewed by maintainers. The fact that a random stranger is
able to push a change and get it reviewed, approved, merged, and available in
minutes should be the goal of most open source projects. On the contrary,
older projects tend to be too reactionary when it comes to infrastructure
tools, so in turn they get very slow interactions. This becomes a demotivator
for anybody who is used to more efficient workflows.

It is not a coincidence that the author became demotivated after doing some
professional experience.

~~~
msh
> The fact that a random stranger is able to push a change and get it
> reviewed, approved, merged, and available in minutes should be the goal of
> most open source projects.

That sounds like a security risk to me.

~~~
pawelmurias
How much safer is it if pushing in a malicious change takes two months?

------
robocat
I noticed a mistake in a comment in a default file in the /etc/ directory -
pretty certain the file is part of Debian (although this was on Ubuntu).

I thought I would try to fix it.

Two hours googling later and I couldn't even work out who the maintainer was.
I don't like eating other people's time, but I even tried using IRC.

~~~
bnegreve
For next time:

    
    
        dpkg -S path/to/file
    

gives the the name of the package containing a file, and

    
    
        dpkg -s package-name
    

gives you the name of the maintainer.

~~~
ktjfi
All mails I've sent to maintainers of packages regarding issues with them have
been ignored. I think they prefer you to use the bug tracker. (Which sucks, so
I always end up doing nothing about it.)

In the end I switched all my servers to Ubuntu. It's been good and I love
PPAs.

~~~
iforgotpassword
Ubuntu bug reporting is just as shitty. No reply for 6+ months was common for
me. One time I even tracked down the issue, which was fixed upstream just one
commit after the one they used for their package. Still no reaction, but about
a year later they asked of the problem still persists with the current
release. I didn't bother to reply and stopped reporting to either Ubuntu or
Debian.

~~~
lokedhs
I had the same experience. After an Ubuntu update, my workstation was suddenly
not able to mount an NFS filesystem from FreeBSD when Kerberos was enabled.

The bug was rather quickly marked as confirmed, with absolutely no updates for
several years, even though the root cause was known. As far as I know, it
still hasn't been fixed.

~~~
dman
Exact same experience. The best bug reporting experiences I had were with arch
and Nix.

------
chubot
_Instead, currently, all packages become lint-unclean, all maintainers need to
read up on what the new thing is, how it might break, whether /how it affects
them, manually run some tests, and finally decide to opt in. This causes a lot
of overhead and manually executed mechanical changes across packages._

I always wondered if Debian/Ubuntu could benefit from a "monorepo". It seems
to work for other distributions, e.g. Alpine Linux and Homebrew.

[https://github.com/alpinelinux/aports/tree/master/main](https://github.com/alpinelinux/aports/tree/master/main)

Right now every Debian package lives in a separate repo, or it doesn't even
have to live in a repo at all AFAIK.

I think Debian has the most packages because their process is very loose and
decoupled (as well as it being one of the oldest distros). But having tighter
integration does help move things forward faster.

~~~
JoshTriplett
I don't think it necessarily needs a monorepo, but having every package on
Salsa (Debian's GitLab installation) would help.

~~~
andrewharvey
The move to GitLab was a glimmer of hope that one day I'll be able to help
contribute to the project which I'm a heavy user.

I wish they went all in and used GitLab Issues for bugs and GitLab CI/CD to
auto-build packages for both validation and pushing new packages into the
Debian repositories.

~~~
h1d
I'm not following the situation but what's stopping them from improving their
packaging system, except for being complicated?

~~~
genghizkhan
If the systemd discussion has taught me anything about Debian governance,
getting everyone to agree is going to be a pain. Far easier to let it be.

~~~
debiandev
Loud minorities and trolls on public mailing lists are not representative of
the community of DDs and DMs.

~~~
JoshTriplett
No, but they're representative of the decision process. "Should we do A or B"
results in people popping up supporting A, B, C, D, and E, and the decision
tends to be "let's support them all". And as the original article we're
commenting on discusses, that "support" comes in the form of "hey package
maintainers, here's what you have to deal with".

~~~
debiandev
This is a misunderstanding of how Debian works.

~~~
JoshTriplett
I've participated in the Debian community for 18 years, since Potato (2.2). I
have a fairly good practical idea of how Debian operates. The above
description was based on many, many, many decisions over the years. On the
rare occasions that Debian has to actually _decide_ something rather than
answering "X or Y" with "yes" (and the decision doesn't fall solely to a small
number of developers responsible for the packages in question), it results in
painful institutional friction.

I love using Debian, I care deeply about Debian Policy and Debian's
procedures, I enjoy many aspects of the Debian community, and I'm also well
aware of where Debian has difficulties.

------
glennpratt
The sidebar makes this barely readable on Pixel 2. The markdown is easy to
read:

[https://github.com/stapelberg/hugo/blob/master/content/posts...](https://github.com/stapelberg/hugo/blob/master/content/posts/2019-03-10-debian-
winding-down.markdown)

~~~
secure
Sorry about that! I’m not great with Web Design, as you can tell. If anyone
wants to contribute a CSS fix, or just a pointer to a good article on how to
fix this, I’d be very grateful!

~~~
switz
You could add the following CSS to the container (`.row` here but you should
make the class name a bit more specific) which will stack the sidebar and your
content into a backwards column (so the sidebar shows up after scrolling past
the content, which looks and feels _ok_ for a quick & scrappy mobile fix.

    
    
        display: flex;
        flex-direction: reverse-column;
    

Put that in a media query to only target phones/small devices.

Feel free to message me if you're confused and would prefer a fast PR, but you
strike me as someone who wouldn't mind learning a bit & I'm in Morocco with
limited internet.

edit: I would also move the branding (name & logo) above the content so people
know who they're reading. And adjust the margins a bit. Frontend is Fun.

~~~
secure
Thank you so much! Committed your suggestion in
[https://github.com/stapelberg/hugo/commit/3a7227fef47fc49550...](https://github.com/stapelberg/hugo/commit/3a7227fef47fc49550c83e05d7ee7f626cbdfaee).

Let me know your Paypal, if you have any, and I’ll gladly invite you for a
virtual coffee as a sign of my gratitude :)

~~~
switz
All good! Feel free to shoot a small donation to the Internet Archive or we
can grab a coffee if you ever find yourself in NYC.

Glad it worked for you!

------
liuw
I love Debian as a user. I considered becoming a DM then DD -- I read all the
relevant documents and tested water with maintaining a package I used -- but
ultimately gave up due to the bureaucracy and politics involved.

The last straw was this:
[https://lwn.net/Articles/704608/](https://lwn.net/Articles/704608/)

~~~
emmelaich
(re the lwn link) Holy shit that is nuts.

------
chungy
The Arch User Repository
([https://aur.archlinux.org/](https://aur.archlinux.org/)) seems to solve a
lot of the collaboration issues. It is pretty painless to create PKGBUILD
files to make a package and to upload a new one to the AUR. Most maintainers
read the comments and accept patches on a timely manner, and there's even a
way to forcibly relinquish a package if a maintainer is AWOL for significant
time.

On the other side of the spectrum, I've found that the official binary
repositories for Arch Linux suffer many of the same issues described in the
article for Debian. Patches being ignored and collaboration or involvement
being near impossible. Even worse, the few people in charge of the official
repositories are allowed to basically remove packages from the AUR with no
interaction with the AUR maintainer, for the purpose of "promoting" them to
the official repositories. This has happened to me twice, and it resulted in
what I think is a worse package in one of those cases.

~~~
Foxboron
Yo! Trusted user from Arch Linux.

>On the other side of the spectrum, I've found that the official binary
repositories for Arch Linux suffer many of the same issues described in the
article for Debian. Patches being ignored and collaboration or involvement
being near impossible.

Well, we still relay on svn internally so things are complicated to say the
least. Even if we had things on git (which we are working on), I'm unsure if
opening stuff up for outside collaborations like gentoo, alpine, void and
nixos does is a good way. You need the proper tooling setup to make sure this
isn't a burden on the maintainers.

>Even worse, the few people in charge of the official repositories are allowed
to basically remove packages from the AUR with no interaction with the AUR
maintainer, for the purpose of "promoting" them to the official repositories.

Which is true. Some people email maintainers of complicated packages before
inclusion, and some also gives a headsup in the comments of the AUR. However,
this is all done if the packager want to. There is no rules here. The removal
is on the grounds that AUR packages shouldn't overlap with official ones.

>This has happened to me twice, and it resulted in what I think is a worse
package in one of those cases.

I'm interested taking a look at this if you want :) foxboron@archlinux.org or
just type in the comments.

~~~
chungy
> I'm interested taking a look at this if you want :)

I'd rather not, the thing in question was around three years ago and I don't
think I even have my AUR PKGBUILD for comparison. It wasn't a cry for help,
and I'm not naming packages or individuals for a reason. Just voicing some
frustration at the process.

~~~
Foxboron
Content is never deleted from the AUR, so any PKGBUILD can be retrieved.

------
lifeisstillgood
For me this is the telling part::

"""When I joined Debian, I was still studying, i.e. I had luxurious amounts of
spare time."""

OSS has stopped (if it ever truly was) being a part time endeavour. I know
from bitter personal experience one cannout up with a lot of bureaucracy and
process of it is your day job - you have time to get through the rubbish in
order to find the diamonds.

How we (as a society now utterly dependent on OSS) manage this problem is on a
par with how we manage journalism - they are bigger questions than I have easy
answers to

~~~
JohnFen
> OSS has stopped (if it ever truly was) being a part time endeavour.

I think that depends on what OSS crowd you want to run with. The part-time
hobbyist sector may be, I think, stronger than ever before.

The difference now is that there exists the "commercial" OSS sector. That is
certainly not part-time hobbyist.

There is some overlap between those two worlds, but they are very different
and distinct worlds nonetheless.

------
ktjfi
You're the guy behind i3? Thank you very much. I love it.

~~~
secure
I’m glad you like i3!

~~~
iforgotpassword
Still using i3 too! It just works great and never gets in the way.

Btw. You changed from .name to .ch I noticed... Trying to naturalize? ;)

~~~
secure
Baby steps ;)

------
devnonymous
While this is sad and painful to read, I can't say I'm surprised.

The problems listed are precisely the kind of problems that Redhat
strategically supports fedora with, in terms of investment of resources. For
all the hate Redhat receives it has consistently been a good community member
by being willing to help fedora in areas that it knows are hard and yet not
'cool' enough to attract volunteer contributions.

What has Ubuntu done for the debian community along the same lines?

------
ansible
I've been using Debian / Ubuntu for many years, as much due to inertia and
familiarity as anything else. And I have a lot of respect for the project.

If I wanted to start being a contributor to a distribution, which one would be
the best to dive in to?

~~~
camdenlock
I would be interested to know the generally-accepted answers to this as well.
What are the free/libre OSes which most folks enjoy contributing to?

~~~
doublepg23
It's a bit of an odd ball, but I loved contributing to the GuixSD project. I
contributed multiple packages after learning Guile Scheme in a couple of days.

------
wgjordan
The post shares several pain points with Debian's slow, aging change-process
and integration-infrastructure.

Open question: If Debian contributors feel the need to drop out and move on
due to these pain points, are there less-painful Linux-distribution projects
out there that are getting more of these pain points right they can flock to?
Is this a sign that Debian needs to reform, or that other, newer distributions
are outpacing it?

~~~
nik736
I switched all my servers to Ubuntu years ago when people were telling me
"Ubuntu is no server distribution". I honestly never looked back.

~~~
Twirrim
Ubuntu is... interesting.

Watch out for what packages you rely on, and what repository those packages
come from. If they're in universe, expect your bug reports on Launchpad to go
unanswered or unresolved. The only way to get fixes in them is to go upstream
to Debian. But note that not every bug in a universe package is actually
fixable or broken in a Debian world, because the bug _might_ be down to
interactions between a decision made by Canonical with packages in the main
repository, and those for the universe package.

~~~
bubblethink
I think ubuntu has done a decent job with universe by and large. I find it in
much better shape overall than epel. A good way to judge the health of these
auxiliary repos is to look at the age of chromium. Ubuntu is generally good
with patching at least security issues in universe. epel is still running
behind by a month or two despite the security disclosures. So RH ought to take
a page from Canonical's book when it comes to epel and make it a more official
thing than the entirely community driven thing it is right now.

~~~
h1d
I don't mind RH being slower as I take it that's what makes RH what it is. Old
packages but most stable.

~~~
bubblethink
It's not about RH packages directly. They are fine. As it is with universe on
ubuntu, you quite likely need epel on rhel for doing quite a lot of basic
stuff. If you are able to get by with just the proper supported packages in
each distro, this doesn't affect you. And that's where the two diverge.
universe is in better shape since it is semi-official, v/s epel which is
completely unofficial/community-based.

~~~
Conan_Kudo
Actually, since EPEL is often backports from Fedora, many people elect to keep
them up to date with what's shipped in Fedora, so it tends to do better than
other counterparts.

The closest counterpart is SUSE's PackageHub, which is often seeded by stuff
going into openSUSE Leap that isn't part of SLE itself.

~~~
bubblethink
Just saw your other comment about Fedora. Thanks for Fedora and EPEL. I was
just pointing out that EPEL would benefit from some recognition as a semi-
official but ultimately unsupported repo from RH. Technically RH already has
'optional' channels which are already in that category, but EPEL should really
be that. Some communication channel between RH and EPEL and QA so that stuff
doesn't break on RHEL point releases and maybe some resources for security
updates would go a long way in making RHEL more usable.

~~~
mattdm
Are you a RHEL customer? If so, telling your salespeople this would not hurt.
Just sayin' :)

------
jdub
Improving the distribution developer experience was one of the early
objectives of Ubuntu, and where it had the least amount of success. It's a
pity, because perhaps the only useful thing you could get out of essentially
forking Debian (in itself not a great idea, but say you wanted to...) is
independence from its technical and social processes. Ubuntu's processes are
different, but not wildly improved... primarily because Launchpad was designed
from a database schema, not for users.

Ubuntu exploited a clear gaping hole in the distro market in 2004. There's
still a gap in 2019, but it's a weirder market now than it was back then.

~~~
secure
Thanks for the perspective! Ubuntu’s founding was before my time.

------
rixed
Been using Debian more or less exclusively since Potato. And I'm a bit
surprised by the tone of these comments and even more by the content of the
blog post. To me, Debian had become an irrelevant fossil not because of any
technical factor such as those listed here or there (lack of powerful tools to
go over packages silos, bug reporting tools, communication tools, packaging
tools...) Indeed I tend to believe not following the industry /best/ practices
is actually an asset in many cases, sometime even a Debian landmark that
served them and their users well (for instance, refusing to stick to release
dates and favouring rock-solid new releases instead); Indeed, "they still use
X why everybody else is using Y by now" should sounds very suspicious to many
ears.

To me as an outsider the obvious cause for Debian obsolescence is, and has
been for more than a decade, the growing bureaucracy and consequently the lack
of new blood and innovation. This opinion anchored the day when, participating
to a Debian bug squashing party surrounded by similarly minded hackers, we
were approached by a DD asking if he could check our identity papers in order
to "simplify the process" of accepting our fixes.

Organisations, and companies too to some degree, can sometime be best
described by what they stand against. Since the beginning Debian has been
standing against a hostile environment: I mentioned already the industry bad
practices, but also part of this hostile outer world were the negligent
upstreams, the unaware users, the cheating corporations and the misguided FOSS
enthusiasts. Some bureaucracy was certainly in order to protect against them
all. But I'm afraid one of Debian legacy will be that the DD will personify
the FOSS bureaucrat, with its 300 pages long packaging manual and 30 steps
long contributor approval processes, in the cultural pantheon of the
distributions of the future.

------
shmerl
Personally, I'm annoyed at Debian still not having a Web frontend for bug
filing. Practically all other distros have some kind of issue tracker site
that allows filing bugs through it.

Using reportbug or e-mail feels way less comfortable in comparison.

~~~
xyzzyz
Better have fewer high quality bug reports than more low quality reports. The
latter just teaches you to ignore them.

~~~
rleigh
It's not just about "low quality" bug reports. The entire system is baroque
and messy to use even for experts. I've been using it for 2+ decades now.

As a maintainer, I frequently needed to change bug tickets. Tags, versions,
dependencies, and other details. All need doing through the control@ email
interface. Changes can take over half an hour to be processed and
acknowledged. It's inefficient and tedious, and prone to error if there's a
single typo. While I'm not a fan of web interfaces, bugzilla, JIRA, gitlab
issues etc. are all a world more effective and efficient to use.

The Debian BTS does what it does well. However, the other systems all do a
much better job. The Debian BTS is a couple of decades past its prime.
Nowadays we don't need email interfaces to services like this, we have much
simpler and more immediate ways to talk to services, which are even easier to
script for non-interactive use.

Take a typical interaction. I need to respond to a ticket I didn't originally
file or get CC'd on. With any other system, I'd log in, find the ticket, make
a comment and any metadata changes, and press "submit". With the Debian BTS, I
need to find the ticket, download the mbox mailbox by hand and open that in a
mail client by hand, then find and reply to the relevant message(s) and make
any other needed changes using the control interface. It takes much longer,
and it wastes huge quantities of valuable time. It is not a pleasure to use.

I can not understate what a huge time sink it is to use the Debian BTS to work
with dozens of tickets every day. It's an exercise in frustration.

~~~
j16sdiz
Why download the mbox? Don't bugnumber@bugs.debian.org work?

~~~
rleigh
Because you might need to reply to a specific email with all the same people
CCd in who wouldn't necessarily be copied in by the bugnumber address. It also
includes the appropriate message ID to make threading work. And you can quote
the necessary bits to put the reply in context.

The fact that you need to jump through these hoops to respond so that people
will see your reply is another reason why it's a really painful and
inefficient workflow.

------
apple4ever
Sad to see it takes this to raise such issues.

It’s hard to make a change on the inside when so many people’s lives depend on
not making that change.

Maybe Debian needs to go through a OpenWRT/LEDE split. That was over a lot of
resistence to change and general bureaucracy. In the end, they merged back
after fixing the biggest problems.

~~~
cicloid
Isn’t that Ubuntu?

------
xtat
10+ year emeritus DD here -- Honestly I think "modern" dev culture is quite
different from the core of he Debian community and this is both a strength and
weakness. I can't say that I want Debian to look anything like the node
ecosystem.

~~~
xtat
Not saying there's not enormous room for improvement - just would be careful
to conflate that with "it needs to work more like homebrew" or whatever--
yikes

~~~
int_19h
Look at some of the other examples people are bringing up to compare against,
though. Gentoo definitely predates the modern dev culture, for example.

------
alexandernst
Another issue, IMHO, is the fact that every time somebody proposes some
change, for example Gitlab + Gitflow instead of sending patches in some
awkward format, via mail, there are a few people that already have their own
workflow based on (probably) mutt combined with 10s of scripts, and they don't
really want to change their workflow for the greater good; instead, they'd
just find reasons why the new proposal sucks and keep using what they already
have.

------
infinity0
This developer is largely responsible for the Go ecosystem in Debian where
they develop against HEAD. That's pretty hard and results in a lot of
breakages. Go developers don't give a shit and push a lot of the externalities
onto distro packages like Debian packagers (volunteers) or else Google
developers (who get paid a huge amount to do bullshit engineering that nobody
else cares about).

We're making a lot of progress with Debian Rust packages and have automated
away 90% of the ordinary Debian crap - which is needed in the general case but
not for Rust where the constraints are very well-defined by Cargo - you only
have to maintain two files (d/changelog and d/copyright) for the vast majority
of Debian Rust crates.

------
WhatIsDukkha
I wonder how well
[https://salsa.debian.org/public](https://salsa.debian.org/public) adoption is
going?

It seems designed to address a lot of these issues?

------
z3t4
I haven't made any debian packages, I heard it's a lot of work, but these
looks good, not bad:

* Granting personal freedom to individual maintainers

* All maintainers need to read up on what the new thing is, how it might break, whether/how it affects them, manually run some tests, and finally decide to opt in.

~~~
JohnFen
I routinely make Debian packages. It's not actually that hard -- unless you're
intending to submit them for inclusion in the Debian repository. I don't do
that, so I can ignore a lot of the most painful stuff.

------
Conan_Kudo
So I've been involved in Fedora for as long as Michael has been involved in
Debian, and I have attempted branching out into other distribution communities
over the years.

To this day, the Debian community is the only community where I have not been
able to get past the initial stages to get involved. And you don't have to
look too hard to see that I'm in quite a few communities...

There's a lot of parallels to Debian and Fedora when I started in the project
over a decade ago.

The clear divider in how the two projects evolved was that Fedora elected to
implement a lazy consensus model for decision-making, and developed a culture
with a bias for action and improvement. Debian requires full consensus
(generally) and has a culture that favors inaction. This difference is what
has kept me in the Fedora Project for over a decade, and I still enjoy working
in that community and doing my part to improve the greater Linux community and
ecosystem.

Over the years, Fedora shed a lot of its more complex processes and developed
simpler tools and supporting infrastructure to make it easier to use and
contribute to the development of the distribution and outlying projects. Over
the years, I've seen us replace our buildsystem infrastructure[1][2][3],
develop APIs and protocols for weaving tools together[4], migrate SCMs and
create the first ever binary store system for Git[5][6][7], develop tools to
simplify complex tasks[8][9][10], and build replacements to proprietary or
overly-complex systems and support open standards and interoperable
systems[11][12], all to benefit our users, our contributors, and our
ecosystem. We've taken a similar hammer to our processes and structures so
that we enable a wider range of people to be involved, representing their
concerns and making our community healthier than before.

We're still continuing down this path of making it easier for people to
leverage the Fedora Project resources for the benefit of the community with
things like COPR[13], CI on packages with Koschei[14] and CentOS CI
integration for projects and packages[15], etc.

That's not to say Fedora is perfect, mind you. It still has some technical and
process warts. But I'm proud of the fact that our community is still actively
trying to improve our processes, our tools, and our distribution. We're not
afraid to make things better, and our community generally wants to make the
Linux world a better place.

[1]:
[https://fedoraproject.org/wiki/FedoraSummit/NewBuildSystem](https://fedoraproject.org/wiki/FedoraSummit/NewBuildSystem)

[2]:
[https://fedoraproject.org/wiki/Infrastructure/CoreExtrasMerg...](https://fedoraproject.org/wiki/Infrastructure/CoreExtrasMerge)

[3]: [http://koji.build/](http://koji.build/)

[4]:
[https://fedmsg.readthedocs.io/en/stable/](https://fedmsg.readthedocs.io/en/stable/)

[5]:
[https://fedoraproject.org/wiki/Dist_Git_Proposal](https://fedoraproject.org/wiki/Dist_Git_Proposal)

[6]:
[https://fedoraproject.org/wiki/Dist_Git_Project](https://fedoraproject.org/wiki/Dist_Git_Project)

[7]: [https://github.com/release-engineering/dist-
git](https://github.com/release-engineering/dist-git)

[8]: [https://www.mankier.com/1/fedpkg](https://www.mankier.com/1/fedpkg)

[9]:
[https://bodhi.fedoraproject.org/docs/](https://bodhi.fedoraproject.org/docs/)

[10]:
[https://mirrormanager.readthedocs.io/en/latest/](https://mirrormanager.readthedocs.io/en/latest/)

[11]: [https://pagure.io/pagure](https://pagure.io/pagure)

[12]: [https://ipsilon-project.org/](https://ipsilon-project.org/)

[13]: [https://copr.fedorainfracloud.org/](https://copr.fedorainfracloud.org/)

[14]:
[https://fedoraproject.org/wiki/Koschei](https://fedoraproject.org/wiki/Koschei)

[15]:
[https://fedoraproject.org/wiki/CI/Pipeline](https://fedoraproject.org/wiki/CI/Pipeline)

~~~
vbernat
As a Debian Developer, I often point people to Fedora to show you can be a
community distribution very similar to Debian and still be modern in your
practice. Your post is insightful.

~~~
Conan_Kudo
Heh, and I still forgot stuff. I forgot that we created HyperKitty[1], the
Mailman 3 frontend that offers web forum style interaction workflows while
still remaining a mailing list for people used to that model. This was
precisely because mailing lists have historically been horrible for new people
to interact with.

[1]:
[https://hyperkitty.readthedocs.io/en/latest/](https://hyperkitty.readthedocs.io/en/latest/)

------
bhaak
> Gmane used to paper over this issue, but Gmane’s availability over the last
> few years has been spotty, to say the least (it is down as I write this).

The Gmane web interface is not just down but shut down for good:
[https://lars.ingebrigtsen.no/2016/07/28/the-end-of-
gmane/com...](https://lars.ingebrigtsen.no/2016/07/28/the-end-of-
gmane/comment-page-1/#comment-13502)

So this issue won't get better without Debian doing something themselves.

~~~
vthriller
It's a shame that the only thing Gmane v2 effort [0] yielded is just a couple
of blog posts. Why did they stop so abruptly?..

[0] [http://home.gmane.org](http://home.gmane.org)

------
ausjke
debian user for 20 years here, love it.

debain is the base for so many other projects, it needs to keep going strong.

in the meantime debian in many aspects is a bit "old" now, its infrastructure
and the way to do things need evolve. I'm not a developer per se, it is not
easy to become one either.

There are so many ways to make packages and it's hard to pick the "best" one
for example.

I hope Debian can reform its model to make it even better, there are just so
many archaic baggage from the past for new comers.

------
jgoerzen
Hi Michael,

Thanks for writing this. I wrote a response here:
[https://changelog.complete.org/archives/9971-a-partial-
defen...](https://changelog.complete.org/archives/9971-a-partial-defense-of-
debian)

The tl;dr version is I agree with you about some of the things you mention,
but also feel like there's an element of personal preference for web-based
tools showing through.

------
forty
I have a naive question (I'm a long time Debian user, but never really tried
to contribute): why isn't all the Debian in a single git repository? I'm not
suggesting to vendor all the third party code but only all the packaging
information, like, say, libreelec does (and I assume many others)

~~~
rleigh
It's all down to history. When Debian was started, individual maintainers
owned their packages completely, and they were often not kept under version
control. This was a practical necessity for distributed development in the mid
'90s. They uploaded the source and binaries when they uploaded a new Debian
package version, but the actual source packages weren't necessarily under any
sort of version control other than the Debian archive storing the uploaded
versions.

Today, keeping packages under version control is considered good practice, but
even today it's not mandatory as far as I'm aware.

While other systems, like the BSD ports, are in a single repository, this
complete separation has had its advantages. The system is almost completely
modular, with all the interdependencies explicitly documented. It's easy to
add third-party packages. Look how easy it is to add extra BSD ports packages.
You have to fork the entire thing because the repo doesn't just include the
packages, it includes all the build infrastructure. This makes building third-
party ports a bit more of a pain, because it wasn't considered important,
while for Debian it was a fundamental design goal. On the flip side, making a
patch for the BSD ports is the same for every single package and it takes just
a few minutes to attach it to a bugzilla ticket.

------
stretchwithme
Thank you for all of your hard work.

------
bfrog
Compared to nixos everything else seems arcane, and compared to arch's
pacman/makepkg combo everything else seems overly complicated.

------
coleifer
So some guy realizes that the vast resources and top-down direction available
at large corporations don't translate to a huge open-source project (which
itself is compromised of tons of open-source projects)? And it took him 10
years to realize this?

Seriously though, I'm amazed at how well Debian just works. And I know it's
because enough people are willing to put up with these types of frustrations.
Thanks for the hard work.

~~~
secure
I’m fully aware of how spoilt one can be as a developer at a large
corporation, but I believe I have toned down my expectations reasonably, and
still felt compelled to write this article and step down.

------
Annatar
I'm curious as to what the author would think of this in contrast to Debian:

[https://illumos.org/books/dev/](https://illumos.org/books/dev/)

------
fxfan
Thank you, Mr Stapelberg. Some people in the OSS community can be overbearing
and dealing with them on not one but two popular projects is something I, as a
user (of both), want to thank you for.

------
paulcarroty
According to my 10 years of Linux experience I can say: community Linux
distributions sucks. 'Cause it's funny when you can upload a couple of
packages and _not funny_ when you need to debug installer or understand why it
doesn't work under upgrade. When you do this professionally, your salary is
the great motivator.

------
kanox
Not really a fan of "I'm leaving X because of free time and also this 20-point
list of why you suck."

~~~
secure
I’m sorry you got that impression from the article. I’m not leaving over less
free time, I’m leaving because of the issues I describe. Reduced free time
amplified some issues, though.

------
ams6110
Comes off to me as a guy who's just tired of the project after 10 years.
Understandable that he finds annoyance and frustration in so many things. It's
like a marriage that has reached the point where one or both partners have
decided they don't want to be together anymore.

As an outsider, to me the list of complaints sounds kind of petty and whining.
Better to just say "I'm moving on, I wish everyone the best" and be done with
it.

~~~
glennpratt
You should try patching Debian packages. I too am an outsider, but I have had
to modify Debian and it's brutal; every package is different and the number of
entry points into the tooling is mind boggling. I have bugs I've never filed
over years because I don't understand the particular packages process.

~~~
ams6110
I'm sure it's true, all I'm saying is that if you reach the point of walking
away, just walk away. A list of grievances at that point is just going to be
seen as a parting shot rather than having any real chance of motivating
change.

~~~
ratsmack
I think the tone of the article was pretty stoic and so I didn't take it as
"whining". I believe he just sees no future in an ever growing Debian
bureaucracy and decided to voice the reason he is moving on. I've been with
Debian since it's inception and can relate to what he is seeing.

