
Dstat project ended due to RedHat replacing it with its own dstat tool - tanelpoder
https://github.com/dagwieers/dstat/issues/170
======
tedivm
Having read through this post and the previous comments[1] I have to say that
RedHat did the right thing in the completely worst possible way.

Stealing the project name was a mistake. The authors are right to be upset by
the fact that people used to get their project when they did "yum install
dstat" but now are getting a completely different (and not completely
compatible) project. I think a lot of the friction could have gone away if
they had just named it "pcp-dstat" or something similar, but instead they
essentially took this person's namespace without discussion, and really for no
good reason that I can tell.

The other issue is that forking a project without even talking with the people
running it is a very touchy subject. Ultimately it does look like the project
was dead- even to the point where people were asking to contribute but not
getting responded to, and the fact that python2 is EOL and python3 support
wasn't added until after this drama started shows that the project wasn't
really "active". So while I do think they should have reached out, I totally
understand why that slipped their mind.

So at this point I think RedHat should apologize and fix the package name.

[1]
[https://github.com/dagwieers/dstat/issues/156](https://github.com/dagwieers/dstat/issues/156)

~~~
dagwieers
It didn't just slip their mind, they did not really care:
[https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c7](https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c7)

The decision to do this was made before June 2018, 18 months after the last
activity. One of the reasons cited was lack of activity, but that is no thanks
to them, I guess.

That is why I am convinced the goal was to replace it from the onset, there
was no interest in helping out the project. In fact "no activity" was the
right excuse to make their action seem legitimate. Attempting to contact the
project could have jeopardized that plan.

They could have just removed "dstat" from the distribution, and added a note
that users can now use pcp-dstat. But now they made it impossible for users to
add the original dstat. Let alone the support nightmare of having a different
tool with the same name. There's no winning this one.

~~~
dralley
I totally agree that they should have left a note on the issue tracker - even
if they thought it would go unread. However I think you're being a little
unfair on this one point.

>there was no interest in helping out the project.

They saw that people were filing issues and making pull requests and that
those people were getting total radio silence. I can understand how someone
might think "well, people are already trying to 'help the project' and the
maintainer isn't even acknowledging it, so it's a waste of time".

And while again I totally understand and agree that they should have tried to
contact you, you've known about the removal for like 9 months and you
apparently never reached out to them either? Like, a couple words in reply to
a bugzilla or an IRC message does not amount to picking a fight with a
company.

~~~
dagwieers
Huh? Jumping to conclusions while not knowing all the facts. Please read up on
the discussions that we had _after_ Red Hat made this decision. Their decision
was made, end of discussion.

I am sure they made a sound business decision, and I think as a result of that
I made the right personal decision. And here we are now having this meta-
discussion with people not having a clue. Welcome !

~~~
dralley
You knew that it had been removed from Fedora months before they announced it
was going to be replaced in RHEL. You seem totally confident that they
wouldn't have just added the package back once you showed back up and
addressed the issues that lead to it's removal. Maybe that's true, but I'm not
sure it is.

~~~
dagwieers
There was a discussion in the Github issues. From that it was pretty clear
this was a done deal. If the project is no longer maintained, why would they.
And they had the PCP reimplementation ready.

You seem to imply there was no communication, and I stopped the project out of
the blue. That is a misrepresentation.

Nobody stepped up to take over maintainership, and I don't see anyone doing
that now. But if someone wants to try, I can unarchive the project and restore
the PRs and issues.

If not, the king is dead, long live the king!

------
certik
I submitted this PR long time ago:
[https://github.com/dagwieers/dstat/pull/50](https://github.com/dagwieers/dstat/pull/50)
that fixed an important installation issue, and it took the author 5 months to
look at it and simply close it without providing any hints how to fix the
problem in some other way. So I was discouraged to contribute further to the
project.

~~~
rurban
You added a security problem. The fix is obvious for everybody working in
security, but it's questionable if it should be discussed there.

~~~
lstamour
I can see how the original CVE patch removed using the current working
directory for loading plugins, but it seems like less of a security issue to
simply look up one directory relative to the current binary install, assuming
the binary is stored in ./bin after being built and plugins would be stored in
./shared ... Now, it might be inconsiderate to look up a folder when one could
be preconfigured based on a common root or home folder, but it doesn’t
necessarily seem as bad to trust the parent folder of a binary as trusting the
current working directory’s contents?

~~~
mehrdadn
This isn't relative to the binary install, it's relative to argv[0]. It's the
same security issue -- that argument is under user control. You can make a
symlink to your target wherever you want and then argv[0] will be the symlink
location. I don't think there is any secure way to get the current script path
in Python.

~~~
contras1970
so you install dstat in /usr/bin/dstat + /usr/share/dstat (because you are
root), and an attacker creates /home/eve/bin/dstat with
/home/eve/share/dstat/evil.py. why would you run /home/eve/bin/dstat? if eve
can get you to run dstat from here ~/bin, why wouldn't she just have
~/bin/dstat with completely different contents?

i'm still convinced this is cargocult security.

~~~
mehrdadn
You're right that this wouldn't matter if the program is run with the same
privileges as the caller. I was imagining if that wasn't the case (e.g. the
user gives dstat setuid permissions), then letting it load arbitrary code
could produce a security hole. Admittedly I didn't think too hard about
whether/why something like this might be done -- if it wouldn't be, then never
mind. As a matter of general safe practice I err on the side of caution, i.e.
not depending on argv[0] to have any particular value for the correctness of
the program, because otherwise you have to think through all the possible
attack/error scenarios and likely document them for the user, etc... and I
thought in any case it was worth pointing out that argv[0] did not necessarily
correspond to the program location regardless.

------
clhodapp
Devil's advocate: Isn't this kind of behavior how we ended up with what became
the POSIX command-line utilities? A bunch of people re-implementing and
extending each other's stuff without changing the name?

I know that the word has changed a bit since then and that it's a bit
different to appropriate a soul-less corporation's command names vs an
individual volunteer.. It just happened to strike me as odd at the moment I
came across this thread that we're so up-in-arms about this when we're all
happily using utilities that were produced by the very same process every day.

~~~
zonidjan
I don't know that massively popular utilities were ever "reimplemented" with
the same name. Ported, or forked, or extended, but not reimplemented. It's not
clear to me if pcp-dstat is a fork or a total rewrite.

Also, most of those utilities were named things which solely described what
they did. It's a bit different to make a new "cut" vs a new "clhodapp".

~~~
JdeBP
Then there is _a lot_ of history that you do not know, starting with BSD and
progressing through the late 1980s and early 1990s when there was a torrent of
"clones" of things.

The "Vixie cron" that you might even still be using is a "PD" clone of the
Unix cron program, reimplemented from scratch with the same name. The van
Smoorenburg "init" that you might have heard of was a clone of (an at the time
sadly _already_ out of date) Unix init program. All of the stuff in GNU
coreutils and other similar GNU toolsets was a deliberate reimplementation
from scratch.

And then there are the Thompson, Mashey, Bourne, Almquist, Bourne Again, Korn,
Z, and Watanabe shells. And BusyBox and ToyBox. And /usr/lib/sendmail . And
the "mailx" utility. And "awk". And "vncviewer". And "getty". And so on.

* [https://unix.stackexchange.com/a/508142/5132](https://unix.stackexchange.com/a/508142/5132)

* [http://jdebp.uk./FGA/inittab-getty-is-history.html](http://jdebp.uk./FGA/inittab-getty-is-history.html)

* [https://news.ycombinator.com/item?id=19988382](https://news.ycombinator.com/item?id=19988382)

* [https://news.ycombinator.com/item?id=17005677](https://news.ycombinator.com/item?id=17005677)

* [https://unix.stackexchange.com/a/316279/5132](https://unix.stackexchange.com/a/316279/5132)

~~~
breakingcups
I'd like to add the common aliasing of vi->vim

~~~
JdeBP
You're too late. It's the fourth item in the first Stack Exchange answer. (-:

------
proctor
this strongly reminds me of the hn post from five days back in which the main
thread talked at some length about a very similar scenario

> "He missed a big one: you have no way to stop Linux distributions from
> hacking up your software, and you'll suffer the consequences of whatever
> they do."

[https://news.ycombinator.com/item?id=19935648](https://news.ycombinator.com/item?id=19935648)

~~~
chubot
Yeah I thought of the same comment. That is unfortunate.

Maybe Debian is more respectful since it's open source not backed by a company
with enterprise customers?

~~~
raverbashing
"respectful" as in let's wait years for a maintainer to come up with a
solution and when we try to propose our own the maintainer trows a fit and
turns it down?

~~~
dagwieers
Is that what happened? I remember it differently...

------
mattdm
So, from a Fedora Project point of view, I'm not really sure how we could have
done this better. This change was properly announced and publicized as
[https://fedoraproject.org/wiki/Changes/MergeDstatAndPerforma...](https://fedoraproject.org/wiki/Changes/MergeDstatAndPerformanceCoPilot)
and approved by FESCo (the Fedora Engineering Steering Committee) in August
2018
([https://pagure.io/fesco/issue/1956](https://pagure.io/fesco/issue/1956)).

The change page says "The original dstat utility has reached end of life - it
does not support python3 and there are no plans to update it." This may not
have been right after all, but I don't think it's in _bad faith_. Perhaps
FESCo should have dug a little deeper, but there aren't really any particular
red flags indicating that that's warranted.

I'm open to suggestions, though -- we _do_ want to continually improve our
processes, and we _do_ want to work well with upstreams. And, y'know, be
better to long-time Fedora ecosystem friends.

~~~
dagwieers
Well, I had to find out this was planned and acted on after the decision was
made and the alternative was written without any regards of the original
project.

And that is fine if it would stay pcp-dstat as-is.

The most upsetting to me is that Dstat is no longer a python tool you can drop
on a JeOS, Synology NAS or a WRT router to get it to work, you now have to
install PCP and all its dependencies to make it work, which goes against the
original design goals. (i.e. we used to support Python 1.5 for a long time to
accommodate RHEL2.1 during its life-cycle)

Also, by taking the Dstat code, removing the plugin mechanism and replacing it
with a PCP backend, writing a python plugin for Dstat is no longer possible.
This is promoted as being a feature as "your plugin is now a config file"
which is a bit disingenuous as you have to write a PCP backend which is a lot
harder.

And it is not even a drop-in replacement, it only implements the built-in
counters, not the full set of plugins (i.e. --top plugins are missing). So the
argument that it needs to work as-is for existing customers does not hold true
either.

By taking that name (with the Red Hat clout) there is no chance of anyone
taking over maintenance without having to deal with 2 products using the same
name, which I guess is forcing your wish on other distributions too.

Wrt. the change was properly announced. I bet the checklist was properly
checked. Or to quote Douglas Adams:

“Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked
filing cabinet stuck in a disused lavatory with a sign on the door saying
‘Beware of the Leopard.”

~~~
mattdm
Well, we do try to do better than the Cottington planning department. Like all
changes, it was posted to devel-announce
([https://lists.fedoraproject.org/archives/list/devel-
announce...](https://lists.fedoraproject.org/archives/list/devel-
announce@lists.fedoraproject.org/message/S2TZ2VS4DMK2PW466U3GKZUYPGAVWOGA/))
to get more visibility than just a devel-list post (or a random wiki page).

In retrospect, it would have been nice for the devs working on this to contact
you directly and explicitly about the proposed change. I'm sorry that didn't
happen.

But here we are now -- at this point, what outcome would you like?

~~~
dagwieers
It has played out as it did, that's why I closed the project.

I don't have any wishes. Just a bad aftertaste but that will go. I don't like
to dwell in the past, regardless of the fact I have to defend myself in this
forum to total strangers who seem to know better :-)

And that will pass as well. We'll see...

~~~
mattdm
Okay. Again, I'm sorry it came out like this. I would like us to do better.

------
detaro
redhat announcing their "new dstat":
[https://www.redhat.com/en/blog/implementing-dstat-
performanc...](https://www.redhat.com/en/blog/implementing-dstat-performance-
co-pilot)

~~~
astine
The claim is that the original is no longer being maintained. Is that true? It
changes a lot about what I would think of this move.

~~~
codewiz
From the git changelog, it doesn't appear to be completely unmaintained, but
not very vital either:
[https://github.com/dagwieers/dstat/commits/master](https://github.com/dagwieers/dstat/commits/master)

Red Hat's announcement (posted in February 2019), also cites lack of Python 3
support as another reason for replacing the original dstat, but Python 3
compatibility was added last January.

~~~
dagwieers
Yeah, I added python 3 support after I learned that was the main reason for
replacing it. It took me less than an hour to make it work on Python 3,
including all (but 2) of the plugins.

And given that PCP is using most of the original code, they must have made the
same changes to get Python 3 support.

~~~
appstateguy
maybe you shouldn't have waited until the absolute last minute to do this or
it wouldn't have been replaced.

~~~
dagwieers
I am not sure what you are implying here. I have zero interest in maintaining
the project. This was the last blow to a dying project.

The king is dead, long live the king.

------
throw7
I only know enough to get me in trouble, but pcp is a general "framework" for
performance monitoring... they have pcp versions/rewrites of all types of
traditional monitoring utilities like vmstat, iostat, top.

Note: they don't overwrite/replace those utilities on the command line. You
can still use the traditional utilities same as it ever was. The pcp versions
are called like so: "pcp dstat" or "pcp atopsar".

------
rurban
Reminds me on moreutils parallel (C), hijacking GNU parallel (perl), and
changing the command line options. [https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=597050](https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=597050)

~~~
JdeBP
This sort of thing happens regularly. For another example of many, witness the
effects on users of the change of the "mailx" command in Debian.

* [https://unix.stackexchange.com/q/469780/5132](https://unix.stackexchange.com/q/469780/5132)

* [https://unix.stackexchange.com/q/489477/5132](https://unix.stackexchange.com/q/489477/5132)

------
peterwwillis
Honest question: aside from "they provide Enterprise support", does anyone
have a reason they would actually use redhat or centos for new servers? Why
not use a Debian-derived one, which seems to both be more common, and have
less constant creepy corporate influences?

~~~
cpitman
Disclaimer, I work for Red Hat. That being said...

The big things for our customers isn't just support, it is stability. When you
build your systems on RHEL 8 (just released), you will get 10+ years of
support and patches for that system. Those changes are all guarenteed to be
binary compatible, so the risk of patching ever impacting systems is low. If
there is a critical security vulnerability (say Heartbleed in openssl), Red
Hat will backport the fix to every supported version of RHEL. That means you
get the security fix without (1) having to upgrade the entire openssl
dependency and risk breaking something else and (2) without pulling in newer
changes that might have their own vulnerabilities.

Going along with the stability, vendors can QA their products against RHEL as
a known target.

Last reason for now, if you are in an industry with any kind of compliance
requirements, like FIPS, Red Hat jumps through the hoops to get the
certification. That makes RHEL the only option in many environments.

~~~
user5994461
I'd say both stability and update.

Debian never upgrades anything (unless security issue). All packages are
frozen at the time of the initial major release. Have fun using
curl/htop/nginx/libreoffice from 5-10 years ago, plenty of obscure bugs and
missing command line flags.

RHEL actually maintain their system packages.

------
snorrah
Curious, i know dagwieers from being involved in the Ansible project (RedHat
owned). Seems friction is occurring, given the language in his post.

~~~
dagwieers
What language ? It is being quite factual really.

~~~
sdiepend
Time to go to Puppet Dag? :)

~~~
fwdIT
:D

------
icedchai
I literally use dstat all the time! This is very sad.

~~~
op00to
Why? What has changed? Do you use the plugin functionality of dstat, or do you
just use the dstat command? If it's the latter, then you can do everything you
could with dead-dstat and more with pcp-dstat.

~~~
icedchai
It saddens me when a useful, good open source project closes. I realize I can
still use it, and I will... That's not the point.

------
throw2016
This is undoubtedly shady behavior. dstat is a widely regarded and mature
tool. Looking at its repo for 'activity' without context - maybe nothing needs
to be changed and its working as designed - is completely disingenious.

Why should repo activity without context be used as an indicator of anything
in discussion instead of focusing on what Redhat has done?

This is openly hijacking an open source project by a billion dollar company
because it can and makes a mockery of not only open source colloboration
culture but basic professional behavior. Has Redhat reached out to the author,
made any requests, tried to work out some way forward, offered to pay for the
brand name? Cmon this is simply indefensible.

~~~
dralley
>Looking at its repo for 'activity' without context - maybe nothing needs to
be changed and its working as designed - is completely disingenious.

>Why should repo activity without context be used as an indicator of anything
in discussion instead of focusing on what Redhat has done?

"Activity" doesn't just refer to commit traffic. People were filing issues and
making pull requests and not getting any responses for more than a year. And
furthermore it was a Python 2 tool and the Python 2 EOL date is fast
approaching.

I agree that the packagers should have at least left a note on GitHub (even if
they thought it would go unread) - but clearly it was not a "maybe it's just
mature and nothing needed to be changed" situation.

~~~
throw2016
This is muddying the waters. Surely if you have gone through the trouble of
looking at the repo and finding these pull requests you would also know the
context and should then share it to give others here a context. Without that
you are just here making 'vague' charges given dstat is a well known, highly
regareded and mature tool. The Python3 port was already done by the author so
the python2 eol seems a complete non sequitor. Again a rush to condemn without
good reason.

Are you saying anyone running a Python2 project can expect to have Redhat
reimplement it in Python3 secretly and have people defend it on HN? This kind
of behavior is simply indefensible.

It's obvious there is a culture clash of open source developers sponsored or
working for corporates conflating heavy activity on Github that they are paid
to perform as their day jobs as the only open source model. But open source
was traditionally about people having other jobs and using their spare time to
develop open source without profit motives because they believed in the
movement. They certainly did not face corporates and paid developers analysing
their frequency of contributions to dismiss their efforts and justify an
unethical takeover.

------
thisgoodlife
I have been using dstat for years. It's a great tool. But why kill the project
just because some company has a project with the same name? There is more than
one distro, you know

~~~
pas
The project was already dead. But now the maintainer[s] found time to complain
about how evil RH simply opted to provide a py3 compatible replacement and
provide it as dstat.

~~~
dagwieers
That's not exactly what happened, but great to see you are making improvements
in your fiction-writing skills :-) Keep it up !

------
rain1
dagwieers, sorry this happened man that really fucking sucks. They should have
done this differently and reached out to you for communication and stuff.
Pretty disappointed with them for this. Hope you keep on and find other
projects that excite you to work on.

------
gigatexal
Why not come up with a new name for the project instead of rage quitting?

~~~
anthony_doan
> Why not come up with a new name for the project instead of rage quitting?

Why should he change the project name when he originally have it first?

Also why did you have to frame it as rage quitting?

~~~
gigatexal
It’s the way of the world that larger fish get to do as they please.

A simple name change and he can continue to do his work. It’s the idea of
turning the other cheek. That or trademark the name I guess.

~~~
ben0x539
Would redhat ship dstat under the new name?

~~~
gigatexal
They should have shipped the original devs version or forked it and worked
with him to get it supported and submitted their RH changes upstream etc.

------
marktangotango
Performance monitoring tools for enterprises can be very lucrative. Surely
they’ll move to an “open core”, closed source commercial license for the
useful bits, if they haven’t already.

~~~
loeg
That isn't Redhat's model. They open the code (it's mostly 3rd party OSS
anyway), including packages (hence, CentOS) and sell access to the updates
system and support contracts.

~~~
mindcrime
_That isn 't Redhat's model._

Well... it hasn't been _so far_. But this isn't the old Red Hat anymore. This
is IBM Hat. And for all the talk about keeping Red Hat mostly independent or
whatever, I think we all know that's bullshit. Acquiring companies _always_
give that little dog and pony show, and it never holds up in the long-term.
There's no way that IBM culture won't wind up infecting Red Hat. Now whether
that's a good thing or a bad thing is a question I'll leave to the
philosophers. But I'd be very leery of assuming anything regarding what RH
will or won't do going forward.

~~~
klohto
Every single thread someone has to bring it up. The acquisition isn’t even
closed yet and suddenly every decision done is mysteriously pushed by IBM.

There is more than enough people who have their own minds. That is the RH
culture, if this freedom is touched they will just quit.

IBM won’t affect RH in any way, Jim is not an idiot because he knows this
would destroy RH and IBM are also not idiots because it would destroy they
purchase.

~~~
mindcrime
_The acquisition isn’t even closed yet and suddenly every decision done is
mysteriously pushed by IBM._

I am not saying that "every decision is mysteriously pushed by IBM." I'm
saying that IBM culture _will_ \- over the long run - infect Red Hat. To
pretend otherwise is, IMO, pretty naive. Over the decades, company after
company after company has been acquired, and nearly every time the acquiring
company makes the same promises about maintaining autonomy for the acquired
company. And sometimes it holds up for a few months, or even a few years. But
every single time, at least that I can recall, in the long run the acquired
company eventually gets totally absorbed by the parent and starts to take on
their characteristics. I haven't heard any cogent argument yet to justify
believing that this time will be different.

 _IBM are also not idiots because it would destroy they purchase._

Like the way IBM managed to avoid destroying Lotus, Informix, Tivoli,
Rational, Sequent, Truven, Explorys, Phytel, etc.?

