
Debian project has plenty of money but not enough developers - mr_toad
https://www.theregister.com/2020/09/10/debian_project_address/
======
tluyben2
Bit of a weird headline as the article says Debian org has $900k in the bank
but that is not spent on the devs; they are volunteers. If they would offer,
let’s say, 2k/mo (the article says they need good people who like though
problems; 2k is nothing for those types however, many people would like do it
because of love for Debian; they do not do it now as they need at least a
little food on the table) per full time contributor, there would be plenty of
devs joining. But for their goals to be met, they would be also immediately
broke. So in fact they do not have plenty of money to achieve what they want,
unless people work for free.

~~~
elipsey
I am currently unemployed/independently-wealthy/thrifter/bumming around the
bay area. I love debian, and I sometimes joke that if there was such a thing
as a minimum wage dev job, I would gladly take it to get out of having to pass
another technical interview :/

It would be neat if they could split the difference between free and real job,
but I wonder if part of the reason they don't is that they want people who
care unreservedly about values and code quality rather than people who need
$500 a week?

~~~
neilwilson
It is of course one of the problems a Federal Job Guarantee would solve.

Clearly working full time on Debian is of value to society, a good use of time
and clearly verifiable by committed contributions.

Definitely worth the living wage.

~~~
dmos62
Agreed that Debian provides (a lot of) value to society. The big question is
why don't they have money to fund development? Disregarding the 900k
mentioned, because it's not enough to pay everyone and it would be spent soon.
In my opinion public infrastructure would ideally be maintained by states, not
individuals or private companies.

~~~
indigochill
I may be jumping to conclusions here, but my first impression was you were
suggesting that Debian is a public infrastructure, and that therefore in your
opinion it would ideally be maintained by the state.

I have a rather strong emotional reaction to this because to me a major reason
for FOSS is to break the problem of perverse incentives that arise in
corporate -and- government work. Governments want to control their populace.
The NSA, for example, has made algorithm suggestions that would give them an
easier time listening in on the world. And the US has prominent figures
arguing for encryption backdoors that supposedly only the "good guys" would
have the keys to. Which is to say nothing of governments that outright ban
encryption. So I would be very skeptical of FOSS officially maintained by a
government (though I suppose its open source nature would still be a point in
its favor for the transparency).

~~~
dmos62
I understand the strong negative reaction; it's common. Thanks for taking the
time to formulate a response.

I thinks it's two different problems: maintenance and cleptocracy. For
example, should the government maintain roads and bridges? It falls within the
common good (public infrastructure) so using taxes for that makes sense.

But, if the government has authority over the bridges, how to stop them from
performing obligatory cavity searches (or some other bad thing) on people who
use them? That's not a question of what the responsibilities or objectives of
a government should be. The question is how to reform a hostile, cleptocratic
government.

I won't presume what context you're coming from, but I've noticed that in some
cultures it's normal to think of the government as "them". For me it's "us".
While someone from the former group might say "government does that", I'm
inclined to say "we do that". My outlook seems to be more common in Europe
than in US, I think.

For me, using the government is the most direct and efficient way of
maintaining FOSS, or doing anything that should be done by "us", as opposed to
by "somebody". I see that as the purpose of governments.

These FOSS projects spanning many borders makes things slightly harder, but
that's just because we're not used to that kind of cooperation. In principle
there's no reason multinational programmes for financing these things can't
exist.

Alternatively, make it difficult for states to use non-FOSS software. That way
you'd redirect all that money being wasted on closed source.

------
teruakohatu
Is $900k in the bank a lot of money for as large and important an organization
as Debian? That is how much the Mozilla Corporation earns every 18 hours. Even
if the staff are volunteers, having rainy day spending seems prudent.

~~~
bluedino
Debian doesn’t spend millions of dollars on executives

~~~
dsr_
Or on any salaries.

Debian's expenses are hosting, conferences and travel reimbursement -- and not
a lot of that.

It's still not really a lot of money, though. As a thought experiment, if
Google decided to donate $1 per installed Debian or derivative OS, I think
that might double Debian's bank account.

That's a plausible funding slogan -- "A Dollar for Debian".

~~~
pabs3
Debian doesn't pay for hosting for most services, all the ones run by the
Debian sysadmin team use donated hosting environments at universities and
companies. For a while we also had a partnership with HPE that gave us gratis
x86 hardware, but these days we do have to spend money on x86 hardware
unfortunately. The non-x86 ports usually run on donated hardware but we only
run build servers on those machines so far. Google already donates quite a bit
to Debian through conference sponsorship and also a one-time 300K donation
last year or the year before.

~~~
sangnoir
I see Debian is one of the projects that benefit from Google's Summer of Code
student sponsorships, although that is a much smaller contribution compared to
those you mentioned.

Do you know if any of the students who particpate in Debian's summer of code
end up being regular contributors after the sponsorship period ends?

~~~
pabs3
Definitely both Outreachy and GSoC students participate after their
internships, although a lot of them of course don't participate for various
reasons.

In particular I know that for years now former GSoC interns are mentoring
current GSoC interns on the project to package parts of the Android SDK for
Debian. This year it is last year's intern and also two others from prior
years.

[https://wiki.debian.org/SummerOfCode2020/ApprovedProjects/An...](https://wiki.debian.org/SummerOfCode2020/ApprovedProjects/Android%20SDK%20Tools%20in%20Debian)

------
insertnickname
This 2019 blog post from a Debian developer describes some of the problems
they are facing from a developer perspective: "Winding down my Debian
involvement" [https://michael.stapelberg.ch/posts/2019-03-10-debian-
windin...](https://michael.stapelberg.ch/posts/2019-03-10-debian-winding-
down/)

~~~
bnmaker
Not liking the bug tracker and mailing lists as well as several other points
come across as typical things-need-to-be-done-differently now, which has
destroyed the culture and meaningful participation in other projects, like
Python.

I'm sure there are many valid criticisms of Debian, the main one for me is
overpatching upstream packages.

~~~
rwmj
The bug tracker is objectively awful. It takes 15+ minutes to subscribe to a
bug, which is ridiculous! Absolutely no one would write a bug tracker today
where everything is processed by email[1].

[1] Which is not to say that _messaging_ is bad - a web interface with a
proper message queue behind it would be fine. In fact that's how Fedora's
systems work. It's that email is not a reasonable messaging system for this
purpose.

~~~
rleigh
Tell me about it. Doing bulk updates to dozens of bugs as a routine daily
activity was an exercise in masochism. As for making comments on existing
bugs, you had to manually download the mbox archive just to make sure the
right recipients were copied in.

It was pretty neat in 1998. Today, it's long eclipsed by pretty much
everything else.

If I were Debian, I'd migrate the whole lot over to GitLab issues seeing as
they have the infrastructure for it set up (salsa). The problem is there is a
very vocal minority who still think it's not only good, but the best choice
for Debian. I doubt the silent majority who suffer it agree. Like anything,
it's an entrenched part of Debian culture that's hard to change. But it was
time to retire it over a decade back.

------
Tepix
I like Debian and have entertained the thought of becoming a Debian developer
multiple times. The learning curve for a new developer is steep. Just
packaging a small piece of software requires a large amount of documents to
consider last time i looked.

~~~
danieldk
It is a tiresome process. I attempted to submit a particular package (and
others before and after me). It was either met with a lot of red tape (IIRC
document licenses for each file individually) and months of no visible
progress when comments were addressed.

This was in stark contrast to e.g. contributing to nixpkgs. Do a PR on GitHub.
The package gets built automatically by CI. Get feedback. Make changes to
address the feedback. And it gets merged. Of course, PRs sometimes go stale
(most projects are dealing with a lack of manpower), but the process is
generally fast.

But the outcomes are radically different: I never attempted to contribute to
Debian again, but I have become an active contributor to nixpkgs.

I think to attract developers the process needs to be modernized. Put all the
package definitions in a Gitlab monorepo. Build each PR in a sandbox on CI.
Describe the submission process on a single page.

~~~
ylyn
Debian cares a lot more about licensing than Nix. Debian also has far more
developers than Nix, I imagine, and processes that have probably existed for
longer than GitHub itself.

~~~
pantalaimon
> processes that have probably existed for longer than GitHub itself.

There is a saying in German that goes

> Wer nicht mit der Zeit geht, geht mit der Zeit.

Now Debian has lot inertia and Ubuntu is a huge factor too, so there are
enough incentives for people to put up with that process.

Also, maybe this is by design. By keeping the bar for every high, maintainers
are not bothered by low effort PRs.

------
qznc
I guess the call to action is:
[https://www.debian.org/intro/help](https://www.debian.org/intro/help) or
[https://www.debian.org/devel/join/](https://www.debian.org/devel/join/)

Maybe also [https://wiki.debian.org/how-can-i-
help](https://wiki.debian.org/how-can-i-help)

------
kasabali
No wonder, they have the largest red tape on maintainer application process.

Until very recently they were expecting GPG keys signed by 2 Debian
developers, and that could be only done by being developers face to face, or
by attending a key signing party on DebConf. Tough luck if you're in a remote
location. Not sure how it's being done after pandemic.

~~~
octoberfranklin
I think it's awesome that they have a fully autonomous cryptographic web of
trust. Far too few organizations do.

------
diminish
I wish Mozilla mimics Debian or Linux development organization. It earns
hundreds of millions and can't keep pace with chrome.

~~~
tus88
Actually firefox is faster at javascrupt that chrome.

~~~
octoberfranklin
That's like saying that emacs's LISP interpreter is faster than vim's
vimscript interpreter.... Yeah, so what?

The performance-critical parts of vim are written in C. You can't say that
about emacs. Or firefox.

~~~
Naga
The performance critical parts of emacs actually are written in C, such as the
low-level primitive functions.

------
protontypes
We have developed a tool for projects like Debian to distribute donations over
larger projects. By distributing the donations in the dependencies, Debian
could even attract new developers in its own dependencies:
[https://github.com/protontypes/libreselery](https://github.com/protontypes/libreselery)

------
liveoneggs
NetBSD (with a lot less money in the bank and similarly bad bug trackers) has
hired people using a few models in the past -- hourly, CoD, fixed-rate, etc
but it's really the hiring itself that is so much trouble, just like in the
real world! Not to mention doing "management" is also a total pain in the butt
that someone now has to do _for free_.

The irony is that the people being hired are mostly already there doing work
for free.

There is no way debian (or NetBSD) could pay market rate to compete for the
time of anyone working on the project for very long so it's usually a
motivational push or justification for sabbatical or "I have three months
between contracts".

It's much much easier to buy hardware and things for developers who ask for
it.

I suppose these projects are seeking developers with passion, personal wealth,
and free time.

~~~
bluGill
Hiring people who would do it for free allowed them to work faster. It becomes
there day job. I know of a few projects in FreeBSD where the author said I'm
between contracts in a few weeks, you can pay me to do this work and it is
done in a month, or I can find a new contract and the feature "that everyone
wants" will be worked on in my spare time and done in 2 years.

------
ausjke
Unless Debian has systematic discrimination for race or gender or whatever, I
don't think there is a need to focus on "diversity" for its developers, NBA is
a good example there and nobody complained. Improving certain area is fine,
put it a priority for problems non-exist will do more harm than good. You do
it because you like it or need it, not because of where you are from or who
you are.

And yes $900K should be used to improve the situation, there might be some
great developers actually need the money, or the money can get more young
developer to start in debian.

------
ilaksh
With the amount of government and corporate reliance on Debian derivates, the
budget should literally be 100 times larger. Like $90 million instead of
$900,000. Or maybe just $9 million. Then you have no trouble finding
developers. You use money to get them. It works very well.

Part of the issue though I suspect may be the way packages work. I am just
guessing but it may be the case that these volunteers are creating packages
for lots of third party software and constantly fighting with incompatibility
to upgrade them. Which seems like the third parties should be more involved
with updating the packages.

But personally I think that even though snaps are so hated, the relative
impossibility of keeping all packages up to date and similar issues is making
things like Snaps or Flatpak make more and more sense as a general direction.

I don't know if Flatpak has a way to do compression or reuse layers like
Docker but I wonder if it might eventually make sense to go in that direction.

Or another crazy idea: maybe it's possible to train an AI to resolve
dependency or build issues and create packages. Maybe pretty far-fetched but
when there are so many tens of thousands of packages I feel like you should
start going for radical ideas.

~~~
rstuart4133
At the 1000ft view RedHat and Debian perform much the same function.

Debian has $900k in the bank. IBM purchased RedHat for $34B. The number's
aren't directly comparable, but methinks a factor of 100 understates it.

------
credit_guy
Plenty of money is always a relative term. If you think you have "plenty of
money" but you don't have X, in reality you don't have plenty of money. There
are exceptions like X=love, admiration, etc, but for most X, if you have
plenty of money and a shortage of X, you just exchange some of your loads of
money for X and you are good.

------
raverbashing
Debian from the exterior sounds like the anti-corporate organization: flat
hierarchy, top-notch democracy, self-directed work with almost no top-down
guidance/"orders". And not so surprisingly, that also cause problems. Let's
not forget the high barrier to entry for contributors.

It is complicated.

------
mlatu
[https://nm.debian.org/faq/](https://nm.debian.org/faq/)

"The site gives me 403 Forbidden errors if I disable referers"

yah cool faq, many questions answered.

------
moonbug
if only there was some way to exchange money for labour.

------
BiteCode_dev
I had to create a deb for a web app, it had to setup and populate a DB (from
web sources), and manage updates for this db when you upgraded the deb. It had
a build process of static files that couldn't be vendored. It has a lot of
configuration to be entered prior those would happen, using debconf. It had to
vendor a virtualenv. I had dependancies on 2 other web apps that had the same
story, and they shared some config files.

I didn't chose this architecture. But I was tasked to package it so that
people could just "apt get install stuff" and start using it.

It was HARD.

The tech is very old school, it doesn't give a lot of feedback, it's hard to
debug, documentation is sparse and examples in the wild are not up to date,
not to mention the state of tooling. And you do all that with shell scripting,
the worst language for the job. Sure you can use anything else, if you like
zero doc and example.

Then we had to distribute it.

Well, turns out creating an APT repo is not so easy, either.

Now, we did all that by cutting a lot of corners, zero red tape, and only us
to please. Also, the consequences of not using best practices, or worst,
breaking something, were few.

On a team of 6, only 2 were really understanding what was going on. The rest
were lost, awaiting instructions from us because they just couldn't picture
were their work would fit in this blob of bash workflow. Now that I'm gone, I
have no idea how the team is going to maintain the thing.

I can't imagine doing the same thing with the level of road block working for
something as sensitive as the official debian repo would bring.

If debian wants more contributors, they have to make this stuff way, way more
accessible. And be friendlier with people telling them it's hard.

Every time I raised the topic, somebody came along to say I shouldn't be doing
x or y. Or just that I shouldn't be packaging something and just publish it on
the debian repo, letting the debian maintainers do the packaging. It's not
going to bring me to work with the community if the first taste of it is the
demonstration they completely ignore their shortcomings.

And I say that while I've been typing apt commands for a decade now. I'm also
french. Debian is, with VLC, Libre Office and Firefox, one of the 4 pillars of
FOSS for me.

~~~
sitkack
Making deb _is_ painful, the politics of how packages are folded, bent,
welded, drilled and smashed is not a simple landscape to navigate.

A lot of Docker is a response to how difficult it is to build packages and how
invasive package installation is to the base system (Nix gets this right).

If in the future you need to build packages again, I'd take a look at fpm.

The goal of fpm is to make it easy and quick to build packages such as rpms,
debs, OSX packages, etc.

[https://fpm.readthedocs.io/en/latest/](https://fpm.readthedocs.io/en/latest/)

[https://github.com/jordansissel/fpm](https://github.com/jordansissel/fpm)

------
mikece
If LMDE picks up steam I’m sure that could change. Is the Ubuntu distro so
different from Debian that those devs can’t simply work more on the upstream
distro (or are Canonical devs not really doing much Linux work but
userland/app work instead)?

------
dxxvi
$900k is not much. It's only enough for 4 developers for 1 year. If you don't
spend the money the sponsors give you, the sponsors won't give you anymore.

------
nottorp
Why don't they hire Poettering to further make Linux unusable for hobbyists?

~~~
octoberfranklin
Because getting him do do that does not require paying him!

Badump-ching! I'm here all week, folks. Don't forget to tip your waitresses...

~~~
nix23
He works at RedHat, and i strongly believe they paying him.

