
Entr: Rerun your build when files change - zdw
https://jvns.ca/blog/2020/06/28/entr/
======
jez
This post got me thinking that it'd be interesting if instead of being given
an explicit list of files to watch for changes to, the list of files were
inferred with an LD_PRELOAD that listened for `open` / `stat` / etc. system
calls that a process ran.

For example, in the example on the blog post, `git ls-files` will almost
certainly ignore autogenerated build files, but it's possible for one of those
files to change without the output of `git ls-files` changing. Similarly for
things like third party packages that are installed system-wide.

With an LD_PRELOAD, all you'd need to do is

    
    
        my-watcher ruby foo.rb
    

and the watcher would figure out which other Ruby files were opened, be them
git versioned Ruby files in the current folder, or Ruby gems / Ruby VM
dependencies in $HOME/.rbenv/, or system packages in /usr/local, or config
files in /etc, ...

I guess I wouldn't actually be surprised to hear if someone has built this
already.

~~~
MrBuddyCasino
Some build systems work that way, tup is one I think. They use strace to
intercept file I/O and figure out what has been updated, and thus can figure
out an optimal way to re-build.

~~~
aidenn0
I think Tup uses FUSE rather than strace (which is why it doesn't track
external dependencies, and requires relative paths for all internal files),
but I might be wrong.

------
q3k
If you're using Bazel, bazel-watcher [1] takes care of this, and does it in a
much smarter way than just looking at a predetermined list of files.

[1] - [https://github.com/bazelbuild/bazel-
watcher](https://github.com/bazelbuild/bazel-watcher)

------
inshadows
Entr and many other similar alternatives do not satisfy my needs. The main
fault is that they take list of files instead of root directory and possibly
include/exclude patterns. So when I create or rename files, these tools either
don't register new files, or they just fail on missing files.

I'm rather reusing existing stuff but seeing how every scriptable filesystem
watcher misses the point, I'm inclined to write my own
inotifywait/inotifywatch wrapper.

~~~
akx
That's literally in the article; `entr -d`. The manpage also has an entry.
[http://eradman.com/entrproject/entr.1.html#d](http://eradman.com/entrproject/entr.1.html#d)

~~~
inshadows
Terminal 1:

    
    
        mkdir /tmp/project
        cd /tmp/project
        touch file{1,2}
        ls | entr -d echo "a change"
    

Terminal 2:

    
    
        rm file2
    

Terminal 1:

    
    
        entr: cannot open 'file2': No child processes
    

Regarding `-d`:

> Track the directories of regular files provided as input and exit if a new
> file is added. This option also enables directories to be specified
> explicitly. Files with names beginning with ‘.’ are ignored.

So first, it doesn't track NEW directories, and second, it exits if new file
is added. Exactly how is this useful?

EDIT: All I want from filesystem watcher is to track files by pattern and just
re-run the command, however complex that may be to implement via OS
interfaces. When I'm working on a Python project that has packages
(directories), I may also refactor (rename files), and this simplistic
behavior does not catch that.

~~~
Adverblessly
> All I want from filesystem watcher is to track files by pattern and just re-
> run the command, however complex that may be to implement via OS interfaces.

I assume I'm misunderstanding something, so this is probably a naive question
and maybe you can explain further, but if that is all you need why not just
write it yourself? It sounds like an <1 hour project to write something cross-
platform in Qt that monitors the filesystem recursively with
QFileSystemWatcher
[https://doc.qt.io/qt-5/qfilesystemwatcher.html](https://doc.qt.io/qt-5/qfilesystemwatcher.html)
checks for the configured regexen and runs the configured command.

~~~
klodolph
I have implemented stuff like this. It is not even remotely a <1 hour project.
Not if you want it to work reasonably well and not be broken all the time.
It’s a full workday for someone who already understands the problem. There are
a number of cases you often want to handle:

\- Multiple files may change in quick succession, for example, if you hit
“save all” in an editor. You might want to delay triggers to see if more
events arrive.

\- You will sometimes see incomplete / corrupted versions of files, because
you ran a command while another program was writing the file. The other
program probably should atomically rewrite the file, but you’re stuck fixing
the problem. The user doesn’t want to see errors from this and you generally
want to rerun the command.

\- You need to execute commands in the correct order.

\- A command may change state from being queued&ready to having outdated
inputs.

If it’s just _one_ command, sure, you could probably do it in an hour. But if
there’s more than one command (usually the case!) then it gets far more
complicated.

Just looking at the last time I implemented something like this--it was
watchman + about 500 lines of code figuring out which actions to run and when.
And that’s not even with any parallel execution.

------
jez
This seems to fill a similar role as watchman[1] (specifically with watchman-
make[2])

I've heard good things about watchman because it will make a best effort to
let filesystem changes "settle" before running the command specified. If
there's a comparison of these two written up or if someone can give their
testimonial I'd love to hear it.

[1]
[https://facebook.github.io/watchman/](https://facebook.github.io/watchman/)

[2] [https://facebook.github.io/watchman/docs/watchman-
make](https://facebook.github.io/watchman/docs/watchman-make)

~~~
michael-ax
dependency free, solid design, perfect with tmux, great online reference. does
not try to do magic:
[http://eradman.com/entrproject/](http://eradman.com/entrproject/)

~~~
masklinn
You do realise that's the tool TFA is about right?

------
imurray
Commands like this are super-useful, and I use them all the time when editing
LaTeX papers.

I used _atchange_ for a long time:

[http://jeffreycopeland.com/work/PDF/1996-03-atchange.pdf](http://jeffreycopeland.com/work/PDF/1996-03-atchange.pdf)
[http://users.fred.net/tds/lab/atchange.html](http://users.fred.net/tds/lab/atchange.html)

And now, out of habit, my own wrapper around _inotify_. But more polished
solutions like _entr_ do seem the place to start now.

------
stefanchrobot
Similar tool that I currently use:
[https://github.com/watchexec/watchexec](https://github.com/watchexec/watchexec)

~~~
mattgreenrocks
Original author here, glad you like it!

watchexec was borne out of a few frustrations with entr, mostly around how it
handles new files being created. However, from a pure design standpoint, entr
is just better than most anything out there due how closely it hews to UNIX
philosophy, and it gets so far on just that.

The only real improvement that can be made to these tools (currently) would be
to have perfect information about what caused a file to change. Currently,
most tools require you to tell it files/patterns to ignore to avoid triggering
loops where the file watcher changes files and ends up triggering itself over
and over. watchexec did good work here by ingesting your .gitignore and using
that.

Unfortunately OSes don't provide great info when it comes to file
modifications. On Linux, ptrace/LD_PRELOAD would enable us to know the set of
all files changed as a result of running the file watcher (and thus ignoring
them automatically). DYLD_INSERT_LIBRARIES is a thing on macOS, though it is
subject to SIP restrictions with some binaries. I'm unsure what mechanism
exists on Windows. The highly platform-dependent nature of this is one reason
why I haven't really pursued this line of work in watchexec.

~~~
psxuaw
Thank you for watchexec. It makes my life better every day.

------
rkwz
Cargo Watch is something similar in the Rust ecosystem -
[https://github.com/passcod/cargo-watch](https://github.com/passcod/cargo-
watch)

------
wheresvic4
Entr is amazing - I'm using it in my go projects to hot reload when
developing!

[https://smalldata.tech/blog/2019/03/15/hot-reload-go-
applica...](https://smalldata.tech/blog/2019/03/15/hot-reload-go-applications)

------
RMPR
Very good piece of software

Tip: Emulate vscode Live preview of Markdown with entr:

    
    
        ls file.md | entr script.sh
    

script.sh

    
    
        #!/bin/sh
        pandoc file.md -o file.pdf
        mupdf file.pdf

------
gumby
I’m sorry: why not just use make? This is a serious, not sarcastic question.

When developing I use make to _invoke_ my program; that ensures everything is
rebuilt.

~~~
anderspitman
Does make have watch functionality built in? Sometimes I use make and entr
together. entr detects filesystem changes and make efficiently rebuilds. But
nowadays a lot of languages have their own build tools besides make, and entr
works directly with any of them.

~~~
johncoltrane
Here is another way to look at it:

    
    
        $ while sleep 2; do make -s build; done
    

What files are interesting is already encoded in your Makefile and make is
pretty good at figuring out if a given file changed or not so, instead of
duplicating efforts, run make every second or two and let it figure out what
needs to be done.

It won't eat your memory or pool of file pointers and your CPU will barely
feel it.

~~~
anderspitman
That's a solid approach if your dependencies are already managed by make, but
languages like Rust and Go do their own dependency management, and Rust in
particular is pretty slow to compile. I wouldn't want it running every 2
seconds.

------
rsanheim
Guard [1] is a great tool in the ruby world for rerunning tests, reloading
your browser, or running any arbitrary task based on files changing inside
your file system. Its built on top of Listen [2] to do the fs piece, which has
pretty well supported at least across linux and mac os over the years.

[1] - [https://github.com/guard/guard](https://github.com/guard/guard) [2] -
[https://github.com/guard/listen](https://github.com/guard/listen)

------
emmelaich
See also
[https://linux.die.net/man/8/incrond](https://linux.die.net/man/8/incrond)

> _The inotify cron daemon (incrond) is a daemon which monitors filesystem
> events and executes commands defined in system and user tables. It 's use is
> generally similar to cron(8)._

~~~
dmux
Thanks for sharing this. I'm continuously surprised by how many tools exist in
the *nix ecosystem that I've never come across. It makes me question if
there's a discoverability problem here. Is there any website that broadly
categorizes what utilities are available given a desired goal?

~~~
emmelaich
I've been meaning to 1. do a blog for the 'command of the day' and also 2.
categorising commands by use / intent.

But until then, no I don't know any.

------
nickcw
Nice tool - I'll be putting that in my toolkit.

I love the way she straces the tool to find out how it works - a true hacker
at work :-)

------
ivanstojic
Entr requires one watch per file. One tool that I haven't seen mentioned here
yet is fswatch, which uses directory based patterns and works more akin to a
find: it outputs changed file names to stdout.

Interesting thing to look at and see if it maybe suits you better.

------
DelightOne
[https://github.com/cespare/reflex](https://github.com/cespare/reflex) works
well too.

------
mixmastamyk
I use a shell script named "onsave" that wraps the following incantation:

    
    
        inotifywait --event modify --recursive . --quiet
    

If anyone wants the full script I can post it here. One thing many of these
tools miss is that I typically want the build command run once at the start.

------
globular-toast
I use this kind of thing regularly with web stuff like Django, Flask and
frontend/JS stuff using parcel, but I wonder what other things this is
appropriate for? I can't imagine it being very useful for C/C++ code. I don't
always want to rebuild every time I save a file.

------
jchook
One of the issues I've seen with entr, fswatch, etc is they don't respond to
ATTR changes (e.g. via touch).

When developing on VM, sometimes you have to "forward" filesystem events to
the guest. Usually that happens via TCP/UDP and touch, but wont trigger the
reload.

------
microcolonel
I will say, inotify is one of the best designed kernel facilities, it avoid so
many of the pitfalls of other facilities used for the same task, and is
generally easier to use. I wonder if any BSDs plan to implement it, outside of
Linux ABI layers.

------
renewiltord
I used to use `inotifywait`. It met all my requirements so I just stuck to it.
Allowed me to have LaTeX rendered to PDF in Evince on one side and write in
Vim on the other side.

Since Evince reloads file if it changes on disk, that meant I had immediate
feedback.

------
psxuaw
Another great tool that I use everyday is watchexec -
[https://github.com/watchexec/watchexec](https://github.com/watchexec/watchexec).

------
amelius
There are better tools which use the ptrace api to check if files changed
which the build actually depends on.

One downside is that ptrace is not recursive/reentrant so some tools won't
work.

------
dang
If curious see also

2017
[https://news.ycombinator.com/item?id=13856623](https://news.ycombinator.com/item?id=13856623)

------
seemslegit
This should go nicely with that new js bundler and that new package manager.

------
0mp
entr is a great too, I use it all the time to automate various little things
when I'm developing. The coolest part that it works in every environment I
care about, which is FreeBSD, Linux, and macOS.

------
je42
Mmh. interesting. I have been using watch + redo to do this.

------
iqandjoke
any equivalent for Windows?

~~~
zaphar
I created [https://crates.io/crates/runwhen](https://crates.io/crates/runwhen)
a while back and it's cross platform. Unfortunately you have to build it
yourself :-p I never got around to creating an automated build pipeline for
it.

------
esperent
This tool seems like a slight improvement on inotify.

Otherwise, it doesn't seem especially unusual or unique to me, and yet the
author is presenting it as a revolution.

Am I missing something?

~~~
cespare
You're missing the fact that another person's knowledge or experiences could
be different from your own.

~~~
mnw21cam
Yeah, it's [https://xkcd.com/1053/](https://xkcd.com/1053/) happening yet
again.

