
Better Bash History (2012) - Pete_D
https://sanctum.geek.nz/arabesque/better-bash-history/
======
aaronharnly
I set up my bash config to store history separately for each working directory
(ie separate history files that I store under a $HOME/.bash_history_d
directory) – so as I cd into a project directory, the history is populated
with the commands that I've run _within that directory_.

I find this super-helpful to remind me of the context for projects – cd in,
then up-arrow to whatever that command is I usually run to launch this server,
grep this logfile, etc.

There are some downsides – sometimes I want to run the same command in
directory X and also in subdirectory X/Y, but I love doing it this way.

Anyone else done something similar?

~~~
dasl
I use zsh, and I have a thing in my profile that skips writing any `cd`
command to history. Instead, it will rewrite the command using the absolute
path of the directory I `cd`'d into and write that to history.

Thus, all of the `cd` commands that get written to my history use absolute
paths and are re-usable no matter where I am.

It would be cool if this sort of functionality were generalizable to all
commands where you type a relative path, not just `cd`.

[https://github.com/dasl-/settings/blob/3143bbfe23bd75c3e35c7...](https://github.com/dasl-/settings/blob/3143bbfe23bd75c3e35c76350355fba2fe25fce8/.zshrc#L95-L114)

~~~
smacke
You might enjoy checking out 'fasd', which, among other things, creates an
alias ('zz') with autocomplete functionality for changing to frequently-used
directories.

[https://github.com/clvv/fasd](https://github.com/clvv/fasd)

~~~
isatty
Or just github.com/rupa/z

------
ConSeannery
To echo what everyone else is saying, get FZF. Most of these linux
productivity things are just some dumb pet project that solves a specific
problem that a specific person finds annoying. FZF is not this - it makes
something you do hundreds of times a day (history and file opening on zsh,
bash, fish, whatever) indisputably better. If you find yourself sitting there
tapping ctrl+r multiple times like an idiot for some thing that may or may not
be there, do your stubborn beardy ass a favour and install fzf.

You can implement some or all of the tips in this HN post, it'll make fzf
results better.

~~~
mceachen
[https://github.com/junegunn/fzf](https://github.com/junegunn/fzf) for the
uninitiated

------
mrzool
I love this blog and learned so much about the Unix toolkit and the philosophy
behind it by reading it. Too bad it doesn’t seem to be much active anymore,
but there’s a lot of great material in the archives that pretty much never
gets old since its focused mostly on standard Unix software. The author (Tom
Ryder) is an very talented writer with a clear and concise style.

A couple of plugs for your consideration:

\- I integrated all of these history options in my project Sensible Bash [1],
an attempt at saner Bash defaults

\- I made an ebook [2] out of a series which appeared on this blog, titled
“Unix as IDE”.

[1]: [https://github.com/mrzool/bash-sensible](https://github.com/mrzool/bash-
sensible)

[2]: [https://github.com/mrzool/unix-as-ide](https://github.com/mrzool/unix-
as-ide)

------
jefftk
I do:

    
    
        promptFunc() {
          # right before prompting for the
          # next command, save the previous
          # command in a file.
          echo "$(date +%Y-%m-%d--%H-%M-%S) \
            $(hostname) $PWD $(history 1)" \
            >> ~/.full_history
        }
        PROMPT_COMMAND=promptFunc
    

This is in addition to standard bash history, and is much more reliable. Being
able to see not only what command I ran but what directory I was in and when I
ran it is very useful.

(I wrote about this here: [https://www.jefftk.com/p/you-should-be-logging-
shell-history](https://www.jefftk.com/p/you-should-be-logging-shell-history))

~~~
deeg
I do something very similar (just saved in files by month) and it has saved my
bacon a few times. I was recently asked to re-run a process that I ran years
ago. It would have taken me hours to re-discover how to run the process but
instead took a few minutes after grepping the commands.

Every developer should do this.

------
kbd
Bash's broken history is what finally got me to switch to Zsh. The history
behavior you want is: 1. each line of history is appended incrementally, but
2. each shell session only has access to its own history unless you
specifically reload the history into your session.

Every combination of history commands in bash didn't do the right thing[1],
but Zsh's works as desired.

[1] eg. putting 'history -a' in PROMPT_COMMAND as the article recommends
appends your session's entire history each time, IIRC.

~~~
cortesoft
See, what I want from my history is every command I have ever run, so that
ctrl r can find anything I have done.

~~~
kbd
Yes what I wrote doesn't exclude that. The only distinction is that you
generally don't want all your sessions' histories interleaved. I.e. when you
up-arrow, you only want your current session's history, without random things
from other active sessions in there too. If you do want "everything you've
ever done" _including_ things from other sessions that started after your
current session, it's easy to reload your history into your session. I have a
'rlh' (reload history) alias for that.

~~~
jackewiehose
You always only have your current session history in each bash (unless you
reload the history manually) so there is no interleaving.

~~~
kbd
Yeah I know, I was just making a distinction from cortesoft's "I want
everything I've ever done" in my history. I was saying "so do I, except I want
active sessions separate" (which as you point out, they are by default).

------
naggie
I have a global bash history, having set up bash/zsh to [immediately share
without any annoying race conditions.][2] I use FZF mapped to CTRL+R to search
through my history.

The unique thing is that I use a [python script (in my dotfiles)][1] to clean
the history in a more advanced way than is possible with HISTIGNORE.

The script removes duplicates, leaving the most recent copy and removes lines
matching several regexes.

[1]:
[https://github.com/naggie/dotfiles/blob/master/scripts/clean...](https://github.com/naggie/dotfiles/blob/master/scripts/cleanup-
history) [2]:
[https://github.com/naggie/dotfiles/blob/master/home/.bashrc#...](https://github.com/naggie/dotfiles/blob/master/home/.bashrc#L26)

~~~
IronBacon
If one is only interested to remove duplicates (your script does more) I've
found this one liner in a StackExchange post: _awk '!x[$0]++' ~/.zsh_history_

~~~
pwagland
Note that this only keeps the first copy, not the most recent copy. To make it
so that it only keeps the most recent, you would need to run this through
`tac` twice.

------
tapoxi
highly recommended for fellow ctrl-r addicts:
[https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)

~~~
graton
I also recommend FZF. Really very handy for doing CTRL-R searches.

I also use it for doing interactive git rebases. I have an alias for doing a
fuzzy rebase interactive (frbi):

    
    
       frbi = !git rebase -i $(git log --pretty=oneline --color=always | fzf --ansi | cut -d ' ' -f1)^

~~~
kbd
This is a great idea, thanks for it! I've done similar things with using fzf
to git add, git diff, and so on, but I hadn't thought of using it to pick a
commit to rebase from.

I made a few changes so I figured I'd post them back here in case you're
interested:

    
    
        pick-commit = # equivalent to your git log... cut...
        rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f"
    

"rif" = "rebase interactive fzf/fuzzyfind". The function and test just makes
it exit cleanly if you escape out of the filter instead of giving "fatal:
invalid upstream '~'" (and I changed it from ^ to ~ because I think ~ is
usually what one should use over ^, but I'm willing to be corrected). And
obviously I already have 'ri' aliased to 'rebase -i'.

While I'm here, here are my similar git aliases for adding, diffing, etc.
files using fzf:

    
    
        pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f"
        af = !git pick-status-files add
        # and so on for diff, difftool, checkout, etc.
    

Oh, and 'fzi' (fuzzy-inline) is a small shell script that just calls:

    
    
        fzf --height 30% --reverse --multi "$@"

~~~
girishso
That looks great. Do you have your dotfiles shared by any chance?

~~~
kbd
Sure: [https://github.com/kbd/setup](https://github.com/kbd/setup)

Here's my git config with a bajillion aliases:
[https://github.com/kbd/setup/blob/master/HOME/.config/git/co...](https://github.com/kbd/setup/blob/master/HOME/.config/git/config)

Since git commands are one of the most common things to type, I alias all
short git aliases like so:
[https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c58...](https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c585e5154863d87cd/HOME/bin/shell_sources/aliases.sh#L126)

(link to particular commit so line numbers stay correct). That way my workflow
looks like:

    
    
        $ ga (fuzzy find files to add)
        $ gcm "commit message"
    

etc.

------
O_H_E
>By default, Bash only records a session to the `.bash_history` file on disk
when the session terminates. This means that if you crash or your session
terminates improperly, you lose the history up to that point.

I kept loosing my history consistently on my new installation because of this.
I usually put my laptop to sleep, and pretty much never _intentionally_
terminate my x-session except on kernel updates. But because its a cheapo
laptop with a crappy battery, I would occasionally run out of battery when
asleep or it just wont wake up.

~~~
barrkel
I have a fairly expensive ThinkPad and it also runs out if battery if asleep
for more than about 4 days, and frequently freezes on suspend or resume when
it's been both suspended and resumed docked and undocked. (Linux, of course.)

------
thenewwazoo
Because this comes up every so often, I'll post this link again: I wrote a
little widget that stores your bash history in a sqlite db. I haven't touched
it in a while because I now rely on fzf and this doesn't integrate with it,
though maybe I could make it do...

[https://github.com/thenewwazoo/bash-history-
sqlite](https://github.com/thenewwazoo/bash-history-sqlite)

~~~
chubot
This is cool, somebody recently suggested that I add history to sqlite in Oil
[1]. I looked at your code but I didn't see how it integrates with bash?

Most people seem to be using the PROMPT_COMMAND hook for stuff like this --
are you using something else, or did I miss it?

[https://github.com/oilshell/oil/issues/320](https://github.com/oilshell/oil/issues/320)

I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I
should add to support end-user customization.

One thing that might be interesting is to support a hook for the ! syntax, so
the sqlite DB could be searched. I think that would solve some of the problems
you mention in the code's comments, and maybe the fzf issue. (Although that
literally just occurred to me, I don't promise to do this :) )

I haven't looked at how fzf works exactly. But if you want to chat about it
please chime in on Github :)

[1] Latest status: _Success With the Interactive Shell_
[http://www.oilshell.org/blog/2019/02/05.html](http://www.oilshell.org/blog/2019/02/05.html)
. Oil runs a bunch of interactive programs like bash-completion, virtualenv,
and git-prompt!

~~~
enriquto
> somebody recently suggested that I add history to sqlite in Oil

what a disgusting, perturbed mind! Shell history is probably the kind of data
that fits better as a text file. It is naturally textual and it can never grow
too long, even if you are typing commands continuously for your entire life.
Please, keep the dirty hands of sql far, far away from the shell!

~~~
stevekemp
I can almost see the appeal, when you think of decorating it.

Consider a table with five columns:

* uid

* pwd

* cmd

* exit-code

* datetime

Now you can `SELECT pwd,cmd WHERE exit-code != 0` to see all failed-commands.

~~~
enriquto
I see your "decorated" sql table, and I raise you my text file with five
columns, that you can grep and sed and awk as if there was no tomorrow.

    
    
         cat history.txt | awk '$4 != 0'

~~~
zepolen
"As if there was no tomorrow" \- exactly, because once you add another column
suddenly $4 points to something completely different and you have to rewrite
everything.

Not to mention $4 is completely arbitrary and it's impossible to tell without
prior knowledge which field it's referencing.

The original query `SELECT pwd, cmd WHERE exit-code != 0` is both descriptive
and won't break with future updates.

------
stilist
Since everybody has their own way of handling it, mine is

* Generate a new history file for every session ([https://github.com/stilist/dotfiles/blob/master/dot_sh/sessi...](https://github.com/stilist/dotfiles/blob/master/dot_sh/session/history.sh))

* A 'histgrep' script that searches across all the files and highlights matches ([https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/e...](https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/executable_histgrep))

------
MarcScott
I've had this in my .bashrc for years. I thought it came from this post
judging from my commments, but looks different, so I wonder whether it came
from HN comments when this was previously posted.

    
    
      ## Better bash history
      # Avoid duplicates
      export HISTCONTROL=ignoredups:erasedups
      # When the shell exits, append to the history file instead of overwriting it
      shopt -s histappend
      # After each command, append to the history file and reread it
      export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}history -a; history -c; history -r"

------
romka2
I remember reading this article back when I was still using Bash. I've finally
got the history I like only after switching to ZSH.

You can get decent history in ZSH just by setting the right options. If you
want great history, you'll need to write a bit of code. Right now my history
works as follows.

History is written to disk after every command. Up and Down keys go through
the local history (from the same session). Ctrl+Up and Ctrl+Down go through
the shared history (from all sessions). Ctrl+R also uses shared history (it's
easy to add another binding for local history but I don't have it). Pressing
Up/Ctrl+Up after typing something will go over history entries that have the
matching prefix. For example, `git<Up>` will show the last command from the
current session that starts with `git`. Here's my config:
[https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_...](https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_better_history/eon23cl).

In addition, my history is stored in Git. History from the local machine comes
first, and from other machines second. This is fairly easy to do it Bash, too.

------
bhaak
I just recently stumbled over [https://spin.atomicobject.com/2016/05/28/log-
bash-history/](https://spin.atomicobject.com/2016/05/28/log-bash-history/)

Using PROMPT_COMMAND to log every command in a separate timestamped log file.
This allows you to use a much better format than bash history has to be in.

Easily readable to see what you've been doing and extremly efficient to grep
for.

Has been discussed on HN before:
[https://news.ycombinator.com/item?id=11806553](https://news.ycombinator.com/item?id=11806553)

------
pkaye
We have some servers where the user home directory is on nfs. To keep separate
history for each server, I add the following to .profile.

export HISTFILE="${HOME}/.bash_history.$(hostname)"

------
ausjke
that's exactly what my setting is for history, nice post. The only difference
is that I unset HISTCONTROL, don't recall why I did that.

    
    
        shopt -s histappend is the default already I believe.
        shopt -s cmdhist is also the default now.
        HISTSIZE=-1 will be unlimited, which is fine for today's computers.

------
tenken
Meh. I prefer to leverage SQL.

I have yet to find a better way to store and search bash history than:

[https://github.com/tkf/rash](https://github.com/tkf/rash)

More recent bash sqlite history attempts all seem to fall short than this
implementation, they store less metrics or have other caveats.

------
commandersaki
Probably the most useful thing I've done regarding bash history:
[https://eli.thegreenplace.net/2016/persistent-history-in-
bas...](https://eli.thegreenplace.net/2016/persistent-history-in-bash-redux/)

------
omeid2
I use fzf for bash history. Fuzzy history search and much better TUI.

[https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)

------
zonidjan
PROMPT_COMMAND is probably already used for something else by your distro.

HISTFILESIZE and HISTSIZE can be set to -1 for unlimited.

Read man bash.

~~~
bhaak
HISTFILESIZE and HISTSIZE don't solve the abysmal bash history file handling
when you have more than one bash open.

I would also like to know which distro clobbers user settable env variables.

------
mistrial9
can anyone please explain how BASH history on current Ubuntu, is different
when logging in from different desktop machines, to the same login ? the dot-
local directory or dot-gconf directory is part of this ? consternation..

