
Shell Completions in Rust - JoshMcguigan
https://www.joshmcguigan.com/blog/shell-completions-pure-rust/
======
JoshMcguigan
Author here. This blog post is the result of my puzzling over how auto-
complete works in the shell, and subsequently whether shell completion scripts
could be written in Rust. It also introduces a new Rust crate,
shell_completion[0], which is a library that exposes some low level primitives
for writing shell completion scripts in Rust. shell_completion is certainly in
its infancy, so I'd be happy to have contributors of any kind (both code and
opinions are welcome).

[0]:
[https://github.com/joshmcguigan/shell_completion](https://github.com/joshmcguigan/shell_completion)

~~~
sbr464
I’m curious about your approach, did you have any resources or projects that
you found yourself going back to?

(Regarding shell completions in general)

~~~
JoshMcguigan
These [0] two [1] sections of the GNU Bash man pages were very useful. I also
found that this tutorial [2] provided a useful perspective. I spent a small
amount of time looking through the bash-completion [3] project as well as the
git bash completions [4].

Generally, my process for writing is google around until I have some base
level of knowledge, then building one or more demo applications to ensure my
understanding is correct. Then as I'm writing the blog post I'll be refining
the demo applications into examples for the blog post and/or library code that
I want to open source.

[0]:
[https://www.gnu.org/software/bash/manual/html_node/Programma...](https://www.gnu.org/software/bash/manual/html_node/Programmable-
Completion.html#Programmable-Completion) [1]:
[https://www.gnu.org/software/bash/manual/html_node/Programma...](https://www.gnu.org/software/bash/manual/html_node/Programmable-
Completion-Builtins.html#Programmable-Completion-Builtins) [2]:
[https://hackaday.com/2018/01/19/linux-fu-custom-bash-
command...](https://hackaday.com/2018/01/19/linux-fu-custom-bash-command-
completion/) [3]: [https://github.com/scop/bash-
completion](https://github.com/scop/bash-completion) [4]:
[https://github.com/git/git/blob/master/contrib/completion/gi...](https://github.com/git/git/blob/master/contrib/completion/git-
completion.bash)

~~~
sbr464
Thanks for sharing (and @snowe2010)

------
dmix
That sounds useful. I used to write tons of ZSH scripts when I used to
obsessively make my Linux desktop experience nice and automated. Writing
custom completions always felt awkward for me and unintuitive (like most ZSH
programming).

Does your shell_completion library work with ZSH? I'm not familiar with how
sh/bash differs here from ZSH but I'd image it'd be easy to make compatible.

I'm waiting patiently for Evlish to mature so I can use their Go-like (and Go-
powered) scripting language and burn my old ZSH scripts in a fire.
[https://github.com/elves/elvish](https://github.com/elves/elvish)

~~~
chubot
If you're using multiple shells like zsh and Elvish, then you might be
interested in my proposal below for shell-agnostic autocompletion:

[https://news.ycombinator.com/item?id=19950892](https://news.ycombinator.com/item?id=19950892)

I talked with the author of Elvish about it, and we both want it to happen,
but are both working on other things now AFAIK.

If you managed to write a working bash/zsh completion script, then kudos :)
It's not easy. The basic idea is that shell completions could be both (1)
shared among shells and (2) moved into the same language that the binary is
written in (optionally).

------
codetrotter
Seeing this got me curious to know if any work has been done on generating
completions for applications that make use of the clap command-line argument
parser.

[https://github.com/clap-rs/clap](https://github.com/clap-rs/clap)

[https://crates.io/crates/clap](https://crates.io/crates/clap)

Among other things I found this currently open issue that has a lot of
discussion and information in it:

[https://github.com/clap-rs/clap/issues/568](https://github.com/clap-
rs/clap/issues/568)

It might be of interest to you as well, OP?

~~~
JoshMcguigan
I did come across this work by clap while doing research for this post. The
major difference in strategy is that clap is generating completion scripts in
Bash, while shell_completion is a library which helps writing completion
scripts in Rust. I think one exciting bit of future potential for
shell_completion is it could potentially allow building something like what
clap has done using the standard Rust macro system.

In any event, thanks for bringing this up.

------
chubot
If I'm understanding correctly, this is a shim that:

1) serializes the fields the bash completion API gives you (when you hit TAB)

2) exports them to Rust, then

3) passes the Rust program's results back to bash?

I've been working on similar problems, and I understand why that's appealing.
Bash is a pretty bad language to write completion scripts in! It's really bad
at string and array processing [1].

\----

Though on a quick glance, if you're exporting $COMP_LINE and $COMP_POINT, then
it's making the job of the completion script very hard, since you're passing
off the responsibility of parsing shell to it:

In other words, if you have something like:

    
    
        mycommand --flag arg\ with\ spaces --flag2=<TAB>
    

Then it's not desirable to pass the entire string. It would be better to parse
it and pass:

    
    
        ["mycommand", "--flag", "arg with spaces", "--flag2="]  # and indicate index 3 being completed
    

Another common case is:

    
    
        echo $(dirname <TAB>    # I don't need echo $( to complete
    

I noticed this deficiency of bash while essentially cloning the bash API for
Oil: [https://www.oilshell.org/](https://www.oilshell.org/)

One consequence of this is that the bash-completion project (maintained
separately from bash) actually has an ad hoc bash parser in bash! This is
really bad -- unsurprisingly it has obvious bugs, like not being able to
complete inside command subs. An ad hoc bash parser in Rust would also be bad.

So instead Oil parses shell for you -- since it has a shell parser! -- and
then it gives you the partial argv array. So then the plugin is only
responsible for what it can reasonably do.

\-----

Additional, I proposed that this could be made into a protocol that other
shells consume. So instead of writing bash completions in Rust, you could
write zsh/fish/bash/Oil completions in Rust.

[https://github.com/oilshell/oil/wiki/Shellac-Protocol-
Propos...](https://github.com/oilshell/oil/wiki/Shellac-Protocol-Proposal)

A lot of people are already "scraping" bash completions this way, i.e. to use
in zsh or their own shells, so it should not be hard to come up with something
"standard", given those working examples.

[https://github.com/oilshell/oil/wiki/Projects-Already-
Doing-...](https://github.com/oilshell/oil/wiki/Projects-Already-Doing-
Something-Like-Shellac)

If you're interested in chatting about it please join
[https://oilshell.zulipchat.com/](https://oilshell.zulipchat.com/) .

I wouldn't say it's the #1 priority now because Oil already emulates bash
quite well, but I think a bunch of people are motivated because they 1) don't
want to write completions N times, for every shell and 2) Don't want to write
completions in the bash language.

home page: main page: [https://github.com/oilshell/oil/wiki/Shell-
Autocompletion](https://github.com/oilshell/oil/wiki/Shell-Autocompletion)

IMO it makes sense to write completions in whatever language the command line
tool / flag parser is written in.

[1]
[https://www.oilshell.org/blog/2016/11/06.html](https://www.oilshell.org/blog/2016/11/06.html)

~~~
MayeulC
I would have gone with an out-of-band way to signal that the application
supports a completion mechanism;

One way would be to add a new section to the binary (though that would only
work with binaries), that could contain standardized data, such as a json tree
of the possible completions, or the proper syntax to use the executable as a
"completion server" as in your proposal.

In shell scripts, it could be one of the header lines starting with a magic
comment such as "#!completion: shellac_v0.1

IMO having the shell detect that the executable supports a standardized
completion format gets you halfway there ;)

~~~
chubot
Yes, the detection is definitely an important part of it that I've idly
thought about, but haven't written anything about. The proposal is still very
early, and not implemented anywhere.

The Github wiki is open to edit if you want to write up a proposal (on a new
page) and discuss it on
[https://oilshell.zulipchat.com/](https://oilshell.zulipchat.com/) . A
prototype is even better.

Although I think that perhaps running the binary with `-h` or `--help` and an
env variable like SHELLAC_FEATURE_DETECT=1 might be a more backward compatible
detection mechanism. If the binary doesn't support shellac, then it will
probably exit quickly with help, or an error message. (And shells could cache
this detection info.)

If it does support Shellac, then it will print some kind of "spec" to stdout
describing where the completion is. Is it in the binary itself written in
Rust/Python/etc., or is it in a separate completion script ?

The conversation has been dormant lately, but like I said I think there are a
few groups of people motivated to do this (e.g. alternative shell users who
don't want to rewrite completions themselves).

~~~
MayeulC
> running the binary with `-h` or `--help`

These shouldn't be taken for granted, I think the feature should be opt-in
(and quite easy to contribute).

I would have executed the binary if there is a segment saying that it supports
Shellac. The binary then detects the environment variable that's in the
current spec, and prints completion information to stdout.

------
xliiv
I did this [https://github.com/xliiv/fui](https://github.com/xliiv/fui) This
is something different, but maybe someone could say it's sort of
"completions"? Fui also has a feature which allow to copy/dump a whole form
(ctrl-k) string which can be pasted to bash.

------
tormeh
The ion shell is written in what I believe is pure Rust, and has
autocompletion by default.

~~~
JoshMcguigan
It is not possible for a shell to know how to perform completions for every
binary on a users machine. Shell completion scripts define the auto-completion
for each specific binary, so for autocomplete to work (well) it needs to be
supported both by the shell and by the application. This post is about writing
shell completion scripts (the application side, not the shell side).

