
Htop explained - p8donald
https://peteris.rocks/blog/htop/
======
guessmyname
I switched to macOS from Linux two weeks ago and I am starting to regret that
decision. Most — if not all — the commands that I thought I had mastered after
years working on Linux environments stopped working because their BSD
counterparts have slightly differences that completely break everything I try
to do. It is even more frustrating when I realize that I have to deactivate
security features built-in the new Apple computers in order to make things
like "dtrace" (the strace alternative) work.

~~~
nrclark
You should give Macports a try if you haven't already! I run it on my macbook,
and it's made a world of difference. 99% of everything I use in Linux can be
found in Macports, and it "just works" for the most part.

You can't get kernel-specific things like strace, but they do have just about
everything else. The big sell for me was being able to install the GNU
versions of everything.

Macports keeps everything in /opt, so you also never have to worry about it
clobbering anything that MacOS expects to work in a certain way.

edit: some background on Macports vs Homebrew - it's basically a vi/emacs kind
of a thing. Homebrew utilizes the MacOS system stuff a fair amount, and
installs everything to /usr/local. Macports tries to keep everything fully
separate, and installs into /opt.

My personal experiences with Homebrew weren't great, but others will probably
have the opposite opinion.

~~~
gregmac
One of the things that I really dislike about homebrew is its refusal to run
via sudo plus the requirement that you own /usr/local.

This means:

1) It is a security hole; any application can now modify system-wide
applications at any time.

2) You can't have multiple users able to use homebrew.

I was expecting homebrew to be something similar to apt/yum/etc, but this
makes it pretty fundamentally different.

How does Macports handle this? Does $USER still need write access to /opt?

~~~
packetslave
There's no requirement that Homebrew be in /usr/local. Some individual
(poorly-written) packages may assume that, but I've been running Homebrew in
$HOME/.brew for multiple years on multiple machines with no issues at all.

~~~
ghthor
That's actually fucking brilliant, and aligns much better with what homebrew
is. I'm setting up my new macbook from work tomorrow and this is absolutely
the way I'm going to do it, and how I'll suggest any of my colleagues do as
well.

------
camtarn
"[explanation of acpid] But I'm on a virtual server that I don't intend to
suspend/resume. I am going to remove it for fun and see what happens. ... I
was able to successfully reboot the droplet but after halt Digital Ocean
thought it was still on so I had to Power Off using the web interface. So I
should probably keep this."

Science! :D

I love the approach of systematically going through and prodding everything to
see what things do and whether they're actually required. I remember doing
this on my Windows 95 machine, stripping down running tasks using Task
Manager. Of course, in Windows 95, if you killed the wrong task the system
would just reboot... :P

~~~
dguaraglia
I used to do a lot of that when I was first learning Linux (back in the day
when each Linux distro came with a full "HOWTO" collection.) I have become
much lazier since, usually going for defaults and not touching anything unless
I _need_ the behavior to change.

------
baldfat
htop is one of the most missed tools when I am on a non-Linux OS.

On servers I find atop to actually be a better tool for finding issues that
happen incrementally and not an obvious issue. I think that atop and htop both
are very complimentary.

[http://www.atoptool.nl/](http://www.atoptool.nl/)

"Atop is an ASCII full-screen performance monitor for Linux that is capable of
reporting the activity of all processes (even if processes have finished
during the interval), daily logging of system and process activity for long-
term analysis, highlighting overloaded system resources by using colors, etc.
At regular intervals, it shows system-level activity related to the CPU,
memory, swap, disks (including LVM) and network layers, and for every process
(and thread) it shows e.g. the CPU utilization, memory growth, disk
utilization, priority, username, state, and exit code. In combination with the
optional kernel module netatop, it even shows network activity per
process/thread."

~~~
digi_owl
unless by non-Linux you meant non-unix, htop recently got updated to better
support BSDs and OSX/MacOS.

~~~
baldfat
That's excellent news.

~~~
digi_owl
[https://archive.fosdem.org/2016/schedule/event/htop/](https://archive.fosdem.org/2016/schedule/event/htop/)

Found a presentation from Hisham about the effort.

------
makkes
> The second number is how much of that time the machine has spent idle, in
> seconds

Actually, the second number is the sum of the number of seconds that all cores
on the machine have spent idle, so that number may be higher than the first
one. Source: [https://access.redhat.com/documentation/en-
US/Red_Hat_Enterp...](https://access.redhat.com/documentation/en-
US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2-proc-uptime.html)

~~~
p8donald
Thanks for catching that. You are right. Here is the source code:
[https://github.com/torvalds/linux/blob/master/fs/proc/uptime...](https://github.com/torvalds/linux/blob/master/fs/proc/uptime.c#L19-L21)

My source (first result in Google) was this link
[https://www.centos.org/docs/5/html/5.1/Deployment_Guide/s2-p...](https://www.centos.org/docs/5/html/5.1/Deployment_Guide/s2-proc-
uptime.html) which looks like an older version of the documentation that does
not have that last sentence.

------
j_s
Slightly related utility I discovered to my great delight today: ncdu - "htop
for disk space usage"

[https://dev.yorhel.nl/ncdu](https://dev.yorhel.nl/ncdu)

~~~
y4mi
oh yes, i mostly don't bother with 'du -sch' anymore since ive come to know
ncdu

~~~
justinsaccount
Have you tried

    
    
      du -macx ~ | egrep '^[0-9]{2}' | sort -n
    
    

egrep filter for files at least 10m ends up being a nice speed up, otherwise
you end up spending most of the time sorting all the tiny files to the top
while up writing hundreds of megabytes of /tmp/sortXXXX files in the process.

------
Waterluvian
I LOVE this style of documentation/teaching.

What is it? How does it know that? How did I figure that out?

The second and third question are far more useful but usually not included in
documentation or commentary.

------
TeMPOraL
Thank you! Great writeup!

I applaud the way you approached it - figuring out every single thing about a
tool.

Also, I didn't know about 'od' command. Thanks again!

~~~
Morpheus_ro
please do explain in 2 words the od command. I was looking on the page and
there is only one od command and it is not explained. thank you

~~~
Morpheus_ro
I really didn't know about this command ... is it that bad to give me -1? ...
I will never ask again here because of this and in the same time thank you you
to the guys that actually lost some time to explain.

~~~
i336_
I'd like to strongly encourage you in the opposite direction, to always ask
questions. Always do this, regardless of how others react.

Many younger people have security issues explaining their knowledge with
others, and sometimes exhibit this as criticism and acidity. It's possible
that's what happened here.

I'm reminded of [http://www.huffingtonpost.com/catherine-clennan/being-a-
dumb...](http://www.huffingtonpost.com/catherine-clennan/being-a-dumb-girl-in-
computer-science_b_9636248.html) \- it highlights the importance of always
asking questions regardless of the context.

It's also possible that someone disliked your use of the phrase "two words",
maybe because they interpreted it literally. Some things resist explanations
that are both short and accurate.

Regarding od, it's a flexible format conversion tool that lets you convert
arbitrary data to hexadecimal, octal and C escaped printable text. I use it a
lot!

~~~
Morpheus_ro
The nicest article in a wile now .. :) thank you for it and for the extra
explanation on od :)

~~~
i336_
No problem :)

(PS. gia-web looks interesting)

~~~
Morpheus_ro
:) no it doesn't. I've send you an email

------
idsout
You swapped the child and parent in the printf statements here
[https://peteris.rocks/blog/htop/#z-defunct-zombie-process-
te...](https://peteris.rocks/blog/htop/#z-defunct-zombie-process-terminated-
but-not-reaped-by-its-parent).

    
    
      printf("The parent process is exiting now\n");

This should read child is exiting.

    
    
      printf("The child process is sleeping now\n");

This should read parent is sleeping.

Overall, I really enjoyed the article.

~~~
p8donald
Thanks for catching that.

------
camtarn
"You'll probably want to keep cron. But if you don't, then you should stop and
disable the service. Because otherwise when trying to remove it with apt
remove cron it will try to install postfix!"

...wat?

Is that actually a thing?

~~~
pjc50
Dependencies in debian systems which rely on "virtual" functionality can often
have surprising results. At a guess removing cron removes things which depend
on it - perhaps one of those is Exim. Something else "requires" an MTA, so it
installs postfix instead to fill the gap.

~~~
ThatGeoGuy
This would be apt. Apt requires an MTA because maintainer messages / change-
lists are sent as mail to the root user.

~~~
stevekemp
Close. It is cron that requires a package which provides `mail-transport-
agent`, because output from cronjobs is traditionally emailed to users.

You can see this via:

    
    
          apt-cache show cron|grep -i mail
    

Note that the `apt` package itself does not require a MTA.

------
nodesocket
Wow so much unix and shell goodness in this post.

Useful bits/tricks to me:

    
    
          # see what files a given program opens
          strace PROGRAM_HERE 2>&1 | grep open
    
          # get last pid
          echo $!
          
          # all sorts of useful commands and information on a given PID
          ls -alh /proc/PID_HERE
    
          # get a tree listing of all processes
          pstree -a
    
          # echo into a file using sudo
          echo "some text here" | sudo tee -a FILE_PATH_HERE

------
CoolGuySteve
Is there a resource out there on how to write these beautiful terminal HUDs?

I'd really like to step up my game for GUI monitoring apps without having to
choose between either terminal log spam, PyQT, or a web server. Ideally I'd
have something like htop/wavemon/power top that I could stream compactly via
ssh.

~~~
lallysingh
I think you're just looking for a book on (n)curses.

~~~
blacksmith_tb
For another fun ncurses tool (on a similar wavelength) I have become fond of
glances[1] - it breaks out docker containers nicely, along with the good top-
like overview of all procs.

1:
[https://github.com/nicolargo/glances](https://github.com/nicolargo/glances)

~~~
Gruselbauer
For general use I'd like to add multitail (logging done wonderfully right),
canto-rss (RSS reader/server), ncTelegram (Telegram client), turses (Twitter
client), centerim (multi protocol IM client), weechat (IRC) and iftop (traffic
monitoring for me), alpine (mail client) and mc (file manager). At least
they're the tools termux starts with on my VPS and I pretty much live on
there.

------
azinman2
Can every technical article be written this well and using this way? It's a
far more interesting read than the title suggests...

------
arca_vorago
I love htop and use it daily on desktops and servers, but I admit I didn't
know half these things. Great practical writeup!

------
piotrjurkiewicz
Well, his 'Extreme edition' is actually not so extreme. This is how Debian
base installation looks like.

A server can perfectly run without dbus, accountsservice, logind, policykit
and acpid. And, unlike he wrote, timesyncd NTP synchronization works well
without dbus.

(Debian base has also cron and rsyslog running by default, for legacy
reasons.)

------
AceJohnny2
> _It turns out that id gets this information from the /etc/passwd and
> /etc/group files._
    
    
        $ strace -e open id 1000   
        open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
        open("/etc/group", O_RDONLY|O_CLOEXEC)  = 3
    

Well, by default yes. But if your system is configured to use NIS/YP or LDAP
(through NSS/Name Service Switch), then these files won't have all the
information (though they'll likely have a few, it's important to have local
fallbacks in case of network issues!)

A more generic tool is 'getent', which will query all the underlying databases
for you. For example: "getent passwd" or "getent group".

Nevertheless, using strace to get a first approximation is an excellent
exercise :)

~~~
LukeShu
The problem isn't really that they used strace+id, it's that they abbreviated
the output a bit too much. A better clipping would have been:

    
    
        $ strace -e open id 1000   
        ...
        open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
        ...
        open("/usr/lib/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3
        ...
        open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
        open("/etc/group", O_RDONLY|O_CLOEXEC)  = 3
        ...
    

Where nsswitch.conf informs it how to look up the information; the default on
most systems being

    
    
        passwd: files
        group: files
    

which tells it to the functions in /usr/lib/libnss_files.so to access the
"passwd" (user) and "group" databases. libnss_files.so uses /etc/passwd and
/etc/group respectively for these.

------
Debonnys
> If I had two cores, my CPU usage would be 50% since my computer can run two
> processes at the same time. The load average of a computer with 2 cores that
> has a 100% CPU usage would be 2.00.

If I recall correctly the CPU usage would still be 100% but could go up to
200% if both cores are fully used.

~~~
p8donald
I am getting feedback that I should not be using the terms CPU usage and
utilization when talking about load averages. I will update the post later.

~~~
willsher
My understanding from my Solaris days was that CPU load is the number of
processes queued for CPU time over the given period. It doesn't related to
usage directly at all. (Other than higher cpu speed and io through out leads
to a reduction in load because tasks finish quicker)

~~~
_TylerDurden_
That's correct.

------
amelius
Nice explanation, but too much to remember in practice.

If only the UI had something like hover-tips to show what each field actually
means...

------
syrrim
kill as a shell built-in also allows you to pass it job numbers rather than
PIDs. For example, kill -9 %1 would kill the first process you sent into the
background, either with & or ^Z

------
creeble
I like the investigation into various cruft that Ubuntu installs (and runs)
that can safely be turned off.

Goodbye polkit!

~~~
creeble
Uh, wait. Can't really get rid of policykit-1 on Ubuntu desktop, I think.
Maybe server.

Pretty OT, but does anyone know if that's true (or worthwhile?)?

~~~
digi_owl
To be a bit snarky, polkit is freedesktop dbus centric (and thus over-
engineered) reimplementation of sudo for the GUI.

It uses a session tracker (used to be Consolekit, these days it is systemd-
logind) to tell if something is running under an active user session and, and
checks that against a bunch of rules (stored in XML, natch) to see if what is
being attempted is allows (and if not, spawn a password prompt so the user can
elevate their privileges).

If you want to do anything "complicated" like powering down your PC by
clicking a button in your DE of choice you need polkit (and entourage) to be
present these days.

Its one of those CADT things that seems to lead to "year of the (linux)
desktop" never happening.

------
mxuribe
This is fantastic write up! I've bookmarked this one for future reference.
Kudos to the author!

------
dorfsmay
I tend to use "atop" because it gives me a better overall picture of my
system. But, there is something really cool about "htoo" that many users don't
realise and that I hadn't realised until somebody pointed it to me recently:

You can use your mouse/TrackPoint in htop to click on menu options, processes
etc...

~~~
y04nn
atop is way more superior than htop, specially with the log history. I just
use htop when I need to check cpu usage or a particular process. But atop
gives a better system status overview.

------
eyeinsky
Beware -- not only about htop :)

------
eltoozero
Love this post, first function explanation for 'uptime' module gives a concise
'strace' demo showing how to determine the files the standard 'uptime' binary
opens.

Really excellent example of showing your work!

------
234dd57d2c8db
If you have python available, I find `glances` to be a great tool (`pip
install glances`)

Has disk, network, proc, mem, cpu, etc. all wrapped up in a a curses
interface. very slick and useful.

------
zython
love it, good overview over all functionalities of htop and easy to follow

Good job.

------
brokenmachine
Wow, that was very interesting and much more information about how Linux works
in a general sense than I would have expected a discussion on htop to contain!

------
vram22
lsof (LiSt Open Files), which used to be a third-party tool and needed to be
install on Unixes, may be built-in by now in some, and fuser (built-in,
usually) are also useful tools for investigating processes and files that they
open (including sockets), and so on. lsof was written by someone at HP, IIRC.
Has a big list of command-line options. Possibly some of the other tools
mentioned in this thread call lsof internally.

------
sabujp
this has nothing to do with htop, it might as well have been top explained,
man ps for crying out loud.

