
“I have [bash] history back to ~2003” - rcthompson
https://twitter.com/michaelhoffman/status/639178145673932800
======
fps

      $ history | wc -l
      56201
    

I use a simple prompt_command hook to push my history (from all my machines)
to a central MySQL database([https://github.com/fredsmith/history-
db](https://github.com/fredsmith/history-db)). Entries are de-duped and a
tally is kept so I can do simple ad-hoc reporting:

    
    
      mysql> select * from history order by count DESC limit 10;
      +-------+---------------------+-----------+----------+-------+
      | id    | timestamp           | command   | hostname | count |
      +-------+---------------------+-----------+----------+-------+
      |     1 | 2015-09-02 19:41:45 | ls        | xicada   | 34458 |
      |    89 | 2015-09-02 14:58:33 | cd ..     | xicada   |  4332 |
      |   519 | 2015-08-24 18:38:55 | ll        | xicada   |  3229 |
      |     9 | 2015-09-02 12:37:05 | cd        | xicada   |  2250 |
      |    82 | 2015-08-28 18:27:07 | add       | xicada   |  1887 |
      |   329 | 2015-08-31 18:22:49 | gs        | xicada   |  1817 |
      |    48 | 2015-09-02 19:17:26 | git pull  | xicada   |  1696 |
      |   275 | 2015-08-24 18:23:44 | df -h     | xicada   |  1529 |
      | 15708 | 2013-10-08 01:33:33 | grid unn1 | Europa   |  1210 |
      |    44 | 2015-08-31 13:10:46 | git push  | xicada   |  1139 |
      +-------+---------------------+-----------+----------+-------+
      10 rows in set (0.02 sec)
    

It's pretty useful (I search through my history constantly with ctrl-R or
using bash's history-search-backward hotkeys), and having a 56K line history
file doesn't seem to impact history searching or shell startup any noticeable
amount.

~~~
pixelbeat
You owe the GNU coreutils project $392

[https://github.com/diafygi/gnu-pricing](https://github.com/diafygi/gnu-
pricing)

~~~
gajjanag
I agree with the general idea that contributing to these projects is good.
However, the link is slightly misleading: "GNU programs are used billions of
time every day all over the world. However, the intellectual property owners,
the Free Software Foundation, don't make a cent from it. What a raw deal! This
project aims to fix that by monetizing many popular command line tools that
GNU maintains."

Maybe true, but keep in mind that the main coreutils maintainers (including
the parent) are funded by Red Hat. There are plenty of FOSS projects that do
suffer from a lack of funding; coreutils, GCC, etc are certainly not one of
them. As such, if your goal is to help the FOSS community at large, you might
want to consider channeling the money to other important projects.

Maybe the donation link already does such channeling; I am curious about this.

~~~
pixelbeat
It's a joke

~~~
geofft
It's a joke that's very much not obviously a joke. This is social commentary
about the state of free software and/or Hacker News, I'm just not sure what
the social commentary actually is.

~~~
brazzledazzle
Seemed pretty obvious to me.

------
nmjohn
I use zsh per directory history [0] as well as this cron job I have set to run
hourly:

    
    
        #!/bin/bash
    
        cd /Users/nmjohn/.directory_history \
          && git add . \
          && git commit -m "Automatic Backup: `date`" \
          && echo "Files Modified:" \
          && git log head -1 --name-only --pretty=format:"" | grep "[^\s]" | awk '{print    "\t\t"$0}' \
          && echo ""
    

It's not bullet-proof, but since I never manually touch the directory it's
worked without hiccup for the last ~6 months. When trying to remember the
syntax to a rarely used function (or even the name itself) or coming up with a
gnarly awk/sed/grep command, it's been incredibly helpful to have fairly
granular search capability to go back and find them weeks later.

Also ctrl+g in a terminal toggles between per directory history and global
history which makes context switching easier when switching to a project not
touched for a while.

[0]: [https://github.com/jimhester/per-directory-
history](https://github.com/jimhester/per-directory-history)

~~~
thebelal
Great to hear you find per-directory-history useful! (I wrote it)

On the topic of the article I have per directory history going back to ~2013
when I wrote the script.

------
leni536
I recently set up my bash history to collect into an sqlite database. Besides
the exact command it records the working directory, return value, starting
time, ending time, a shell and a login session id. It doesn't handle
background jobs well right now though.

~~~
nyir
Okay I'm curious as to why? I mean it's nice to have, but what are you going
to do with the additional data later on?

~~~
leni536
It was mainly for an exercise to get familiar with sqlite. Maybe later very
rarely I can use the session data (hunting down sessions where you only
remember some irrelevant command). Getting the duration of some commands is
useful like long running simulations, compilations. It's especially handy
where you don't expect it to be too long and forget to time it beforehand.

Having said that I didn't use it too much yet, I have my regular history in
place too.

~~~
caipre
zsh has an option to automatically print the time of commands exceeding a
threshold. See here for a brief example:
[http://nuclearsquid.com/writings/reporttime-in-
zsh/](http://nuclearsquid.com/writings/reporttime-in-zsh/)

------
dale-cooper
I guess the downside of this is that ctrl+r would only search the current
day's history? Personally I at least increased the history kept with
HISTFILESIZE=500000 and HISTSIZE=5000

~~~
snogglethorpe
Another issue is that it looks like would start a new history file only when a
new shell is started. Some people log in fresh every day, but I've got
terminal windows on my home machine that have been sitting around for many
months...

I guess you'd have to set the history-size etc large enough to hopefully last
for the maximum expected shell run-time.

EDIT: hmm, the man page says you can just unset HISTFILESIZE or set it to a
non-numeric/negative value to avoid truncating the history file at all.

~~~
michaelhoffman
Yeah, I have this as well:

    
    
       export HISTSIZE=65535 # why not?
       export HISTFILESIZE=65535
       export HISTCONTROL=ignoredups
    

but now I'm wondering if I should unset `HISTFILESIZE` insteead.

~~~
myhf
It can also be nice to use HISTCONTROL=ignorespace

If you're planning on keeping a lot of history, ignorespace makes it easy to
avoid inserting commands with sensitive info. A well-placed $(cat) can work
too.

    
    
        $  PASSWORD=hunter2 # sensitive info doesn't go into history
        $ curl -O http://name:$PASSWORD@server/path # command with correct syntax does go into history

------
DEinspanjer
I keep a pretty long bash history, but I don't have it unified among machines
or anything.

Others have mentioned several handy tweaks for doing this, but I didn't see
all of the ones I use so I thought I'd share them here. One of the important
bits is the HISTIGNORE. I am interested in commands that I might want to
search for and run again one day, so I filter out commands I consider to be
clutter.

    
    
      # Configure my history preferences
        # Load history substitute into readline rather than immediately executing
        shopt -s histverify histreedit
        set histappend
    
        # don't put duplicate lines or lines with leading spaces in the history. See bash(1) for more options
        export HISTCONTROL=ignoreboth
    
        export HISTIGNORE='&:bg:fg:cd*:clear:ls:pwd:history:exit:make*:* --help:'
        export HISTTIMEFORMAT="%m/%d/%y - %H:%M:%S "

~~~
knodi123
who actually types pwd? I've had it in my prompt since I learned about PS1.

~~~
michaelhoffman
I use `pwd -P` sometimes.

------
aaren
If you're going to store a lot of history, you might want something nicer than
the standard ctrl-r to search it with.

[fzf] is ctrl-r on steroids.

[fzf]: [https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)

It's a fuzzy pattern matcher for your shell history (and lots more).

~~~
dmix
I love all of these new Go tools for managing your *nix systems.

I started using Homemaker recently
[https://github.com/FooSoft/homemaker](https://github.com/FooSoft/homemaker)
which is a very clean/lightweight replacement for GNU Stow, Homesick, and
other dotfile managers. The Go versions of these utilities are always simple
and single-purpose.

Modern developers are returning to building Unix style systems programs. Time
is being spent on that instead of the usual bulky python scripts that have
been popular recently or the million personal-productivity web apps people
were creating.

------
giis
Here's something interesting - We collected bash history of around 15000 users
and listed frequented used commands for fun :)
[http://www.webminal.org/fulc/](http://www.webminal.org/fulc/)

~~~
joelthelion
Doing this on my own bash_history is what prompted me to create autojump.

~~~
rkrzr
Thanks so much for creating autojump! autojump + Crl+R are saving me so much
time that would be lost searching around.

There is just this impedance mismatch between our brains hazy recollection of
things and the exactness that is required by computers. I find that fuzzy
search combined with stats like most-commonly-used bridge this quite well!

------
andrewstuart
That's like saving your toenail clippings.

~~~
meric
I saved my milk teeth in small clear plastic bottles when I was a lad and they
disappeared after a year or two from tooth decay.

------
enibundo
This guy is the inverse of me, because I have linked my .bash_history to
/dev/null

~~~
tempestn
Semi-serious question: do you delete all your emails after you've read them
too?

~~~
Touche
No, emails are important.

I also don't run a keylogger.

------
ashurov
Great resource for hackers if they someday get access to your account.. but I
guess you will have bigger problems than.

~~~
judofyr
Yeah, I think they can do more harm with .ssh/id_rsa than with .history.

~~~
Canada
In practice I've had better luck with shell history than SSH keys when
exploiting arbitrary file reads. The history tends to reveal interesting
locations.

------
dingaling
bash history had never seemed to work for me between tmux windows. Commands
are 'trapped' in their respective window and the last one to close 'wins' and
writes its history over the others.

So I end-up with never-closing task-based window sessions and occasionally
copy-paste commands to a safe file for future reference.

I really should investigate the root cause some day...

~~~
tokenizerrr
Your shell is likely configured to write to the file, instead of appending to
it.

    
    
        shopt -s histappend
    

In your .bashrc should do. To have it write after every command take a look at
[http://unix.stackexchange.com/a/1292](http://unix.stackexchange.com/a/1292)

~~~
gknoy
Some people dislike it, but I LOVE it. I like being able to run a command,
open a new terminal, and be able to control-r to find that command again. It
ensures that I don't have race conditions over which terminal writes its
history to disk first, as well, so I lost far fewer (zero?) history elements.

~~~
JdeBP
You've conflated terminal and shell there. It's an important distinction
because, in fact, there's still one race condition with history files, even if
you only have /one/ shell running. It's down to the Bourne Again shell not
doing atomic updates when it rewrites the history file. It's one of two bugs,
the other being systemd sending multiple signals in _very_ quick succession to
kill a terminal login session, that in combination cause the effect discussed
at
[https://news.ycombinator.com/item?id=10151861](https://news.ycombinator.com/item?id=10151861)
. Far from losing zero or a few history elements, it can in fact result in
your losing _all_ of your history.

------
pdkl95
Unfortunately, I only started saving my history somewhat recently.

    
    
        $ cat ~/var/history/bash.d/* | wc -l
        464291
    
        $ for year in 201{2..5} ; do
              echo -n "${year}: "
              cat ~/var/history/bash.d/${year}* | wc -l
          done
        2012: 116285
        2013: 175525
        2014: 117928
        2015: 54553
    

It's not a complete history, as I filter several commands with HISTIGNORE.

------
mcculley
I was impressed that Stephen Wolfram has been recording his own keystrokes
since 2002: [http://blog.stephenwolfram.com/2012/03/the-personal-
analytic...](http://blog.stephenwolfram.com/2012/03/the-personal-analytics-of-
my-life/)

~~~
olejorgenb
You can start too:
[https://github.com/gurgeh/selfspy](https://github.com/gurgeh/selfspy) :)

------
Sealy
yeah i wonder how many passwords are stored in there in free text format

~~~
leni536

       echo "Password: "; read -s pass; stuff --pass "$pass"

Where stuff takes password as an argument. Alternatively you can start your
line with a space character and it isn't stored in your history.

~~~
bratch
The leading space behaviour is true only if HISTCONTROL="ignorespace" is set.

~~~
leni536
Did not know that, for me HISTCONTROL=ignoreboth is set in my .bashrc. I think
Debian's adduser creates a default .bashrc file where this line is present,
I'm not sure though.

------
Nursie
I wish I had bash history that long. Never seems to have the command I want
when I look for it.

~~~
otis_inf
I think it's better to simply learn the commands than to have a tremendous
amount of history logged. If you have to type a lot of command parameters a
lot you can always automate that with aliases anyway.

~~~
fluidcruft
It's not the stuff you do often that is difficult to remember.

------
lottin
I have <M-p> and <M-n> bound to history-search-backward and history-search-
forward respectively, so

ls <M-p>

inserts the most recent command in history that begins with 'ls ' in the
command prompt. I don't know why this isn't the default, as it's extremely
handy. Also it works with any CLI program that uses readline.

If you want to try it out, add this to your ~/.inputrc file.

"\ep": history-search-backward

"\en": history-search-forward

------
uxcn
I considered doing this with bash, but it was really only a marginal benefit
for me. I do keep archives of all my rss/atom feeds though.

[https://github.com/uxcn/scripts/blob/master/misc/bash/newsbe...](https://github.com/uxcn/scripts/blob/master/misc/bash/newsbeuter/newsbeuter)

------
steeef
I use fzf [https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)
combined with bash and zsh for doing some fuzzy completion of history when
pressing CTRL-R. Also opens the list in a tmux pane if it's installed. Plus,
it's very useful in Vim for opening files in the current buffer.

------
nateabele
Anybody have the equivalent for fish shell? Or better yet, a config that will
consolidate history across terminal tabs?

~~~
TimWolla
fish 2.2 ships with `history --merge`:

    
    
           · --merge immediately incorporates history changes from other sessions.
             Ordinarily fish ignores history changes from sessions started after
             the current one. This command applies those changes immediately.

------
b0b0b0b
I've found this tool to be super-helpful:
[https://github.com/dvorka/hstr](https://github.com/dvorka/hstr)

It's like ctrl-r, but it shows you many candidates at once.

~~~
aaren
Also have a look at fzf:
[https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)

------
maw
Is hostname,pid,date really enough to avoid collisions?

If avoiding them is really important, I don't think you have any choice other
than to go the whole hog and basically implement half of Maildir.

~~~
KnightHawk3
Well if it includes seconds, the likely hood of me typing something on a
machine, switching to another and running something with the same pid within a
second, is pretty low.

Mostly because I can't switch to another machine within 1s

~~~
michaelhoffman
It includes hostname too. You can only get a collision if you do all of the
following within one second:

1\. Start an interactive shell. 2\. Enter a command you want saved. 3\. Exit
this shell. 4\. Start another shell. 5\. Be unlucky such that the new shell
has the same PID as the previous one.

This is not something to worry about.

------
tectec
I do something like this in PowerShell too. Forgetting handy one-liners by not
realizing they were important, or losing them when Windows restarted were the
impetuses for me to start.

------
kazinator
> “I have [bash] history back to ~2003”

And due to the way Bash is developed, all you need is the same Bash executable
you had in 2003 to ensure all the commands actually _work_.

------
enibundo
Why even store your bash history? It's like you store everything you ever
said... Would you lose your precious time to go through the old stuff? I
wouldn't.

~~~
michaelhoffman
I'm a computational biologist. I often need to know exactly how some piece of
analysis was done months or years ago. In an ideal world all the details would
be in scripts with carefully written READMEs cross-linked from my lab
notebook, but it's not an ideal world and sometimes we're in a hurry. In those
cases the history saves my bacon.

~~~
uxcn
I'm not a computational biologist, but I've run into this once or twice. I
only use the shell for quick summary statistics now though. I started doing
most of my analysis in R, which made life easier on a lot of different levels.

