
Ask HN: What are your best shell scripts that you use? - JadoJodo
I saw a note[0] about using a script to activate Redshift in Linux and wondered what other cool, useful, or otherwise interesting scripts you might be using.<p>[0] https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21916949
======
jdblair
I've been using my "dots" script for over 10 years to give me feedback that a
long task (like un-tarring a big source tree) is still running, without
flooding the terminal with text. I pipe the "verbose" output of a command into
the script: tar xvfz kernel.tar.gz | dots

#!/usr/bin/perl

$| = 1; $i = 0;

if ($#ARGV < 0) { $number = 1000; } else { $number = shift(@ARGV); }

while (<STDIN>) { print '.' unless ($i % $number); $i++; }

print "\n";

~~~
wruza
See also pv.

[https://linux.die.net/man/1/pv](https://linux.die.net/man/1/pv)

~~~
isp
tl;dr - The flag for pv in this case is --line-mode (-l).

So, if you have a command that produces lots of lines of output (e.g., for
demo purposes, "yes" \- which literally floods the terminal with lines):

    
    
      yes | pv -l >/dev/null
    

Example progress output, showing lines processed (total & rate):

    
    
      405M 0:00:03 [ 140M/s] [   <=>     ]

------
captn3m0
I use snippy as my snippet manager. Keep them version controlled in ~/.snippy
and activate it with dmenu:
[https://gist.github.com/coderofsalvation/46549e3788ade2f3a93...](https://gist.github.com/coderofsalvation/46549e3788ade2f3a938)

I have another one that splits AAX files from Audible by chapters:
[https://github.com/captn3m0/Scripts/blob/master/split-
audio-...](https://github.com/captn3m0/Scripts/blob/master/split-audio-by-
chapters) (After stripping the DRM)

ansi2html converts terminal output to HTML:
[https://github.com/captn3m0/Scripts/blob/master/ansi2html.sh](https://github.com/captn3m0/Scripts/blob/master/ansi2html.sh)

dydns.sh sets up a dynamic DNS entry on CloudFlare against my current _local_
IP address.
[https://github.com/captn3m0/Scripts/blob/master/dydns.sh](https://github.com/captn3m0/Scripts/blob/master/dydns.sh)

emojify converts emoji shortcodes to emojies on the command line:
[https://github.com/captn3m0/Scripts/blob/master/emojify](https://github.com/captn3m0/Scripts/blob/master/emojify)

vtt2srt: convert vtt subtitle files to SRT
[https://github.com/captn3m0/Scripts/blob/master/vtt2srt](https://github.com/captn3m0/Scripts/blob/master/vtt2srt)

------
dmoreno
Mine watches for changes in a directory and runs a command on any save change.
I use it a lot for Test Driven Development. Modify source code, save and
automatically on a terminal it runs `make test` or similar.

[https://gist.github.com/davidmoreno/c049e922e41aaa94e18955b9...](https://gist.github.com/davidmoreno/c049e922e41aaa94e18955b9fac5549c)

~~~
0xdeadb00f
Have you heard of wendy[1]?

Might be useful, I'm not entirely sure if it suits your case because my
experience with inotify is limited to playing around with it and nothing
substantial coming from it.

1: [http://z3bra.org/wendy/](http://z3bra.org/wendy/) (Author's blog post:
[http://blog.z3bra.org/2015/03/under-wendys-
dress.html](http://blog.z3bra.org/2015/03/under-wendys-dress.html))

~~~
jumpm4n
Another alternative is the entr command -
[https://www.systutorials.com/docs/linux/man/1-entr/](https://www.systutorials.com/docs/linux/man/1-entr/)

------
nl5887
I'm using a script that will request an AWS spot instance for 6 hours,
reporting the address when fulfilled:

[https://gist.github.com/nl5887/56912b70b782baa4bd580ae22bde6...](https://gist.github.com/nl5887/56912b70b782baa4bd580ae22bde6c9e)

~~~
hoffs
What do you typically use these instances for?

------
blackrock
I use this in Ubuntu to take screenshots. Ubuntu comes built in with gnome-
screenshot, but the file saving mechanism is lacking.

So, this will automatically file away your screenshots to your
$HOME/Documents/screenshot/ folder, organized by year/year_month/file.png.

Where file.png is in the format yyyy_mmdd_hhmmss.png.

I use it to take an area screenshot of all my research notes, useful comments,
gold nuggets, etc. The automatic folder organization files it away nicely, and
keeps it organized as the years go by.

Create it and set the execute bit:

    
    
      sudo vi /usr/bin/area_screenshot
      chmod ugo+x /usr/bin/area_screenshot
    

Then copy the contents below:

    
    
      #!/bin/bash
      
      screenshot_dir="$HOME/Documents/screenshot"
      current_year_dir="$screenshot_dir/$(date +%Y)"
      current_month_dir="$current_year_dir/$(date +%Y_%m)"
      fileout="$current_month_dir/$(date +%Y_%m%d_%H%M%S).png"
      
      # Step 1: Check for screenshot directory 
      [ -d "$screenshot_dir" ] || mkdir "$screenshot_dir"
      
      # Step 2: Check year and month directory
      [ -d "$current_year_dir" ] || mkdir "$current_year_dir"
      [ -d "$current_month_dir" ] || mkdir "$current_month_dir"
      
      # Step 3: Take area screenshot to the current month 
      [ -d "$current_month_dir" ] && /usr/bin/gnome-screenshot -a -f "$fileout" $@
    
    

Then map it to the printscreen key.

~~~
Daviey
Uhm, by default the 'Print Screen' key in Ubuntu Gnome saves the screenshot in
~/Pictures/Screenshot from $(date "+%Y-%m-%d %H-%M-%S").png

~~~
blackrock
True, but it doesn’t organize it further. And over time, the screenshots
accumulate. I needed a way for it to self organize. And I found this to be the
perfect compromise.

------
lcall
Basically, I write a script (or use an alias) for everything that I do
repeatedly which would otherwise require typing more than 2-3 unique
characters before using tab completion. The habit just makes life easier (and
is fun and relaxing sometimes). I try to use names that are quick to type
(like, not the same finger repeatedly) and which are memorable, or grouping
things by first characters (if there is a theme).

(This goes with the idea, which I also try to encourage, that any repeated
process should be first documented in some rough form at least (like a
personal note-base or team wiki), then improved over time, via improving the
doc, scripting it, and moving toward full automation based on balancing
considerations of cost/benefit over time, YAGNI, and avoiding debt, and ideas
from the "checklist manifesto", such as the realization that even smart people
can forget important things, drop the ball sometimes, or leave.)

Edit: This also lets me script away differences between platforms, so I can
just remember my usual little 1-3 letter command and it takes care of the
rest, while the script records the details for reference.

~~~
iudqnolq
I've been holding off making scripts for a lot of cases so that I memorize the
commands. Do you think that's pointless? I'm a college student now and I've
been thinking that I'd be better off not customizing my shell too much so that
I can use other computers I end up in front of. (I have nothing against
customization in general, as I think my emacs would show)

~~~
kixiQu
The most wizardly of the wizards I work with (FAANG, though that correlates
less with shell wizardry than people might think) does a lot of his ~scripting
in a Vim buffer in tmux and just enters text over one chunk at a time. This is
a nice approach because a lot of the time he's starting from bare bones, and
always has his basic building blocks very evident so he knows them well. But,
when he needs to build up a more complicated script for review, he can clean
up whatever he's got. If you ask me, I'd say don't worry too much about
memorizing stuff you _don 't_ use often (viva la man page), but it's great to
have your common commands memorized rather than aliased for when you're
helping someone else out or explaining how something works.

~~~
lcall
Could you clarify what is meant by "enters text over one chunk at a time" and
"basic building blocks very evident"? I use tmux & vim but ... unsure if I
follow.

Edit: Do you mean, personal, local notes on tasks, as the notes evolve into
scripts? I also keep a lot of personal notes which I can export into a web
site (similarly to putting into a wiki, in a very loose comparison, an example
being my site at [http://lukecall.net](http://lukecall.net) or the other at
[http://onemodel.org](http://onemodel.org)), using something like a big,
efficient, outline of all my notes. (I would probably put it in org-vim like
org-mode, or maybe taskwarrior, if I didn't have
[http://onemodel.org](http://onemodel.org), which I hope to make much easier
to install, sometime.)

Thanks.

------
jmstfv
When building a new website, I default to handwriting the HTML, instead of
going with a static site generator. More pages lead to more duplication. I
wrote a script that recursively goes through the directory, and replaces all
occurrences of a string:

    
    
        find . -name '*.html' -print0 | xargs -0 sed -i "" "s/replaceme/withthis/g"
    

The same but with fd:

    
    
        fd -e html --print0 | xargs -0 sed -i "" "s/replaceme/withthis/g"
    

I have a tiny Sinatra app that I use in testing some parts of my Rails app. I
either start it in the background or in a separate terminal window (that I
close after some time). I have a script that kills a process given its port
number (4567):

    
    
        kill -9 $(lsof -ti tcp:4567)

------
shakna
I've been writing a lot recently.

There's a bit of scaffolding around things going into a docs folder, some JS
and CSS tweaks for the various output formats (ePub, HTML & PDF), but there is
a very simple script I use to reduce some friction whilst writing.

It's nice to have a lot of small files. However, you may often want to:

1\. Rearrange those files

2\. Insert a new section between two other files

So, I hacked together this incredibly tiny script that means anytime I want to
add anything, I can immediately get to wherever I'm up to:

    
    
        next="$(ag TODO docs | awk -F':' '{print $1}' | uniq | sort | head -n1)"
    
        if [ -z "$next" ]
          n="$(basename "$(find docs -type f | sort | tail -n1)" .md)"
          n2="$(echo "$n + 10" | bc -l)"
          next=docs/"$(printf "%05d" "$n2")".md
        end
    
        echo "$next"
        nano -s 'aspell -c' "$next"
    

Bonus: It outputs the current filename to the console, so that if I want to
add stuff after where I'm currently at, I have the starting number.

The entire scaffold for my books is three scripts, some standard CSS, JS and
YAML. Which makes setting up a new one to match my sensibilities, quick and
simple.

------
iudqnolq
A hacky python wrapper around bluetoothctl that lets me use aliases as
addresses. So I can write `connect {niss}` to connect my phone instead of
`connect 12:6a:78:c2:74:d5 (ref. to one of childhood favorite books, Startide
Rising).

The script then runs "devices", looks for an alias "niss", and substitutes in
the corresponding address. I use expect in Python to script it all together.

~~~
iudqnolq
Got back to my laptop, here it is (extremely hacky):
[https://gist.github.com/danielzfranklin/bb4848ffda1d44b437e2...](https://gist.github.com/danielzfranklin/bb4848ffda1d44b437e2a9d7c8a8f3f5)

All commands you enter pass directly to bluetoothctl except aliases in curly
braces are replaced. You can use it interactively or pipe to it.

Complex piped stuff may or may not actually work. If stdin isn't a tty the
program exits once bluetoothctl reports a success/failure after it gets an EOF
on stdin. This means you can write `btctl <<< "connect {my_device_alias}"` and
it will exit once it's done connecting or couldn't connect.

If you're using it interactively it only waits 0.01 seconds for results before
displaying them and moving on to the next input() (whereas bluetoothctl will
asynchronously display more results even while you're typing in a new input)
so you may need to spam enter to see results.

------
ivanmaeder
I wrote this to make renaming multiple files quicker/easier:

[https://github.com/ivanmaeder/vimv](https://github.com/ivanmaeder/vimv)

Basically it loads up the output of `ls` into an editor, then runs a `mv`
command for each line.

Not something I use daily, but still handy.

~~~
pheug
There's vidir in debian's moreutils package which does pretty much exactly
that.

Check all also all the other tools from that package:
[https://manpages.debian.org/buster/moreutils/index.html](https://manpages.debian.org/buster/moreutils/index.html)

The package's tagline is "collection of the Unix tools that nobody thought to
write long ago, when Unix was young", so pretty relevant to the whole topic.

~~~
ivanmaeder
This is cool! Thanks for this, I had no idea it existed. It looks like the
functionality is almost identical. Even the command names are similar…

------
zzo38computer
Here is one:

    
    
      #!/bin/bash --
      f0() {
        echo 'select moz_bookmarks.title || '"'"' = '"'"' || url from moz_places, moz_bookmarks on moz_places.id = moz_bookmarks.fk where parent = 2;' | sqlite3 /home/user/.mozilla/firefox/twht79zd.default/places.sqlite
      }
      f1() {
        firefox `echo 'select url from moz_places, moz_bookmarks on moz_places.id = moz_bookmarks.fk where moz_bookmarks.title = '"'$1'"';' | sqlite3 /home/user/.mozilla/firefox/twht79zd.default/places.sqlite`
      }
      f$# $1
    

Another one:

    
    
      #!/bin/bash --
      curl 'http://icanhazip.com/'

------
stevewodil
I wrote a script the other day that starts my dev environment. It opens
terminal and executes the commands to start the backend and CDN services, then
opens another tab for the git directory. After that it opens the frontend and
backend codebases in my IDE. I then added the script to my PATH so I can
launch it with a command from any directory.

This is for a side project and being able to launch the dev environment so
quickly has allowed me to start working more easily as opposed to going on
YouTube or Reddit instead. Your brain will crave the easiest source of
dopamine so anything you can do to make the habits you want to build (like
working on a side project) easier will help you immensely!

~~~
zaphodias
it's probably worth checking tmux + tmuxinator, definitely pays off in the
long run :)

I'm using it for a few months now and I'm in love with it

~~~
stevewodil
I will definitely check that out, a quick search and it seems very cool.
Thanks!

------
robgibbons
Back when I still used Xubuntu, I wrote little script called Highlander that
manages the launching and switching between programs in my launcher tray. It
gives your app launcher icons the same functionality as OSX dock icons: launch
an instance of a program, then for subsequent clicks, bring that instance to
focus. That way you don't get redundant copies of a given program running (and
thus the joke of calling it Highlander.)

[https://github.com/robgibbons/highlander/](https://github.com/robgibbons/highlander/)

These days I use Ubuntu MATE which has a Mac-like dock with the same
functionality for free.

------
vkaku
Here's a few useful ones:

\- SSH wrappers/eval-ed Aliases, which do: exec ssh -l user host "$@"

\- AWS wrappers, that allow me to: exec aws --profile blah "$@"

\- DB wrappers, that allow me to hit a target DB

\- cred: Allows me to pick creds from a password manager / keychain
independent of OS; Usually a polyfill.

\- subtract, intersect, union, sample: for handling one liner data, incl. CSV
files

\- csvcut: Python wrapper though that does -f, -d, -c but on CSV files

\- j2y, y2j: Python wrappers for JSON to YAML, YAML to JSON

\- envsubst, watch: polyfills for some environments

\- vdimount, imgmount: Allow loopback mounting for partitions / images for
VirtualBox/Qemu

\- nanny: Nanny for starting processes, passing listen sockets on restart,
change NS properties; Usually a polyfill

------
Carpetsmoker
I got various scripts here:
[https://github.com/arp242/dotfiles/tree/master/local](https://github.com/arp242/dotfiles/tree/master/local)

------
LinuxBender
There are many useful commands on CommandLineFu [1] for what it's worth.

[1] -
[https://www.commandlinefu.com/commands/browse](https://www.commandlinefu.com/commands/browse)

------
duelingjello
Automatically re-ssh if a host key changed and run _monkeysphere s_ (ssh pk’s
stored in gpg keychain) if not already loaded in the gpg agent (which also
supports ssh-agent functions).

 _lw_ which does _ls -l “$(which -a “$1”)”_

 _ew_ which does _”$EDITOR” “$(which “$1”)”_

 _newsh_ which does _touch ~ /bin/“$1” && chmod +x ~/bin/“$1” && “$EDITOR”
~/bin/“$1”_

And habitat (hab) and arch pkgbuild, which use shell scripts as their package
DSL... the former I’ve hacked up to screen-scrape package versions (due to the
dearth of RDF/metalink usage in release artifact publication) and check gpg
keys.

~~~
miccah
I have one similar to your "newsh" called "new" which does `mkdir "$1" && cd
"$1"`

I also have another one called "store" which does `mkdir "${@: -1}" && mv $@`.
I basically wanted an easy way to move everything into a new directory ("store
* backup").

------
sirtoffski
I’ve got a VPN server on a machine which gets a public IPv4 address via DHCP.
Once in a while it changes. Instead of using something like DDNS, I just have
a script running via chron to get my public address and send it to a GitHub
gist.

If VPN stops working, I can always check to see if the address changed. May
not be the best solution, but it works for me.

Script itself: [https://sirtoffski.github.io/docs/my-pub-
ip](https://sirtoffski.github.io/docs/my-pub-ip)

~~~
thrwaway69
curious, what purpose is vpn for assuming you host it for yourself only?

~~~
JeanMarcS
I installed one for myself only too.

But it’s more like a proxy to be geolocalized in France (some services block
foreign IPs) when I’m not.

Pretty handy.

------
clauderoux
In my lab, we use more and more a programming language such as tamgu, which
has been implemented to benefit from pipes with an integrated editor.
([https://github.com/naver/tamgu/blob/master/docs/English/tamg...](https://github.com/naver/tamgu/blob/master/docs/English/tamgu%20as%20a%20shell%202.pdf))

------
ppurka
quietemerge - pretty output wraparound for Gentoo emerge (
[https://github.com/ppurka/quietemerge](https://github.com/ppurka/quietemerge)
)

btex - interactively compile LaTeX to PDF or PS or DVI, highlighting warnings
and errors ( [https://github.com/ppurka/btex](https://github.com/ppurka/btex)
)

searchforfile - search for files interactively with results "as-you-type" (
[https://github.com/ppurka/searchforfile](https://github.com/ppurka/searchforfile)
)

displaymessage - wrapper around a bunch of gui dialogs (
[https://github.com/ppurka/displaymessage](https://github.com/ppurka/displaymessage)
)

A couple more in my GitHub account.

------
digitalsushi
I have a bash alias mapped to a single letter, and when I connect my macbook
to the various three desks with multiple monitors, I can cmd-tab to the iterm
window, open a new tab, type the alias, and it will infer which desk I plugged
into and fix my monitors. Left, middle, right rotated. Etc.

~~~
aframe
I need this

~~~
jakehilborn
It's pretty easy to set up, the instructions in the README show how to get the
bash command you need.
[https://github.com/jakehilborn/displayplacer](https://github.com/jakehilborn/displayplacer)

------
ooooak
> cat `find * -type f | grep .php` | wc -l

I have a project that is going on since 2015. So ran the above command to see
how much code we have. I was expecting our models to be really large turns out
we have ~(404 * 3) rest endpoints. the scariest thing was Vendor dir.

LOC per dir

Models: 15053

HTTP Endpoints: 28877

migrations: 49757

Laravel: 81728 (Framework)

Vendor: 1158299 (third party php code)

Total: 1797417

~~~
guessmyname
Tokei [1] provides interesting stats as well.

[1] [https://github.com/XAMPPRocky/tokei](https://github.com/XAMPPRocky/tokei)

------
downerending
This very small pdsh clone:
[https://github.com/michaelkarlcoleman/ssssh](https://github.com/michaelkarlcoleman/ssssh)

------
oxplot
flashserve [1] to quickly share a file with my smart phone over WiFi.

sub [2] is a simple shell based template processor.

[1]:
[https://github.com/oxplot/gists/blob/master/flashserve](https://github.com/oxplot/gists/blob/master/flashserve)

[2]:
[https://github.com/oxplot/gists/blob/master/sub](https://github.com/oxplot/gists/blob/master/sub)

------
tardismechanic
sort | uniq -c | sort -n

Boom - histogram!

~~~
vkaku
`sort -rn` alternately

------
salutonmundo
i use noscript, so gocomics.com is a bit borked --

    
    
      gocomics () { eom $(curl "$@" | grep item-comic-image | grep -Eo 'https://assets.amuniversal.com/\w*') &> /dev/null; }
    

this takes advantage of eye of MATE's ability to fetch remote files. i assume
'eog' would work too.

------
mam2
dkl instead of docker kill all

