Hacker News new | past | comments | ask | show | jobs | submit login
OpenAutoComplete: Shell-agnostic, cross-platform autocomplete specification (github.com)
138 points by GamePad64 12 days ago | hide | past | web | favorite | 26 comments





If we could get some standards defined and adopted, that would be amazing. There are so many CLI tools that could really benefit from auto-complete, but don't yet have it because it is a huge pain to write. Often times the auto-complete is significantly more work than the original tool was, in a couple cases I've written it was an order of magnitude more work to write the shell completion.

I have tried writing auto-complete scripts for various tools several times. I'm quite good with bash (if I may say so myself). I've written some fairly advanced scripts and have been doing it a long time. However, despite my existing experience, writing auto completion is still a major pain in the ass. This is an area that really needs improvement.

I don't know if this project is the correct answer. I think some good discussion should be had. But I do want to say that the problem they are solving is real.


I'have just posted something like "request for comments" thing, so we could talk and make a proper solution. I've tried to use PowerShell in Windows, and it was pure pain: the tools, that are written in PS work fine, everything else (like git, InnoSetup, CMake, everything else...) just didn't have autocompletion at all. Maybe, some tools, like git do have shell plugins (like posh-git), but the problem in general still persists. Also, I am using Poetry and Werf utilities, that have built-in generator for autocomplete (bash, zsh and fish), but it doesn't work very well and the developers had to write it by themselves.

So, I've tried to make the spec as generic as possible, so it could support current compicated cases, like git (with its infinite number of commands and options), scp (with its remote file listing), some old Windows commands (like "format" for windows95 and so on).

Also, this format could be used with a single-pass recursive generator, because each usage pattern is just a JSON array and all its properties can be defined in-place. So, it would be okay to use it with languages, like C, that have no JSON serializer built-in.


I know what you mean. Go's spf13/cobra will soon have working zsh/bash/fish/powershell autocomplete generation!

https://github.com/spf13/cobra/pull/1070


I started down this path last year:

https://github.com/oilshell/oil/wiki/Shellac-Protocol-Propos...

but cut it out of the Oil project (for the forseeable future). Right now Oil takes the approach of running existing bash completion scripts, which solves the problem well enough (but leaves a few things to be desired)

If someone comes up with a spec that works in at least one other shell, and some code, I'll be happy to merge it into Oil.

Though, reading over this, it looks pretty early, and I don't think the "spec first" approach really works. That is, the spec should be derived from working code, and I believe you have to implement it in more than one shell to tell if it's feasible.

Some other links I collected:

https://github.com/oilshell/oil/wiki/Shell-Autocompletion


There is also a Rust POC of a Shellac library/server implementation: https://gitlab.redox-os.org/AdminXVII/shellac-server

Thanks for the reply! I've googled hard, but couldn't find anything like that. The problem I see is that Shellac is dynamic. So, it is pretty intrusive. This is how Click autocompletion works in Python. Also, I don't think, that Shellac is mutually exclusive with OpenAutoComplete. Being a Shell-to-autocompletion server protocol, it could use openautocomplete library as a data source.

> The problem I see is that Shellac is dynamic

So OpenAutoComplete is purely static then?

That makes it much less useful.

Static completion is nice to have for tools you only use from time to time, or when fishing for that argument name, but if you don't know the name of an argument etc you will often have to use `man` anyway.

The real benefit of completions are the dynamic ones, at least for me.

If such a spec does not support dynamic completion, a lot of tools will need to do their own thing anyway.

I realize that this makes it much more complex though.


Nope, it is not purely static. It is static-first (declarative), bot running a command and parsing its output or delegating the completion to some shell code is a planned feature.

You could make it dynamic with "bin -OpenAutoComplete" to have the binary output the Json. This would allow gradual introduction and backwards compatibility. It would also keep the completions up to date with binary versions, and still allow a shell to cache the Json.

More importantly, you should consider other metadata before you go to far; binaries could carry their own documentation, licensing, dependencies... What should be in and out of scope?


Also, the accompanying code: https://github.com/openautocomplete/oactool It currently supports generating completion code for fish (experimental), but zsh and bash are similar to it, but a bit complicated, so it will be implemented later.

I've written quite a number of shell completions for zsh and the amount of variation in command arguments would require something much more flexible than this basic json specification. This might work fine for listing options and arguments but without more information, completions end up being worse than nothing because with dodgy definitions, you end up breaking things like basic file completion in certain contexts which can be very annoying for users.

A few days ago, I came across kingpin [1] (a library for writing CLI applications in Go) which seems to have some kind of --completion-script-bash option which generates a completion script for you. It might be interesting in this context.

[1] https://github.com/alecthomas/kingpin#bashzsh-shell-completi...


This sounds like a great idea, and I'm sure there's people that would be willing to help (me included). Is there somewhere organized already for people to discuss the development, or where to contribute (Gitter, Discord, IRC, etc.)? I couldn't find anything mentioned in the GitHub pages.

Can't you just parse --help in the background so we as developers don't need to write anything at all?

That’s not standardized at all, though.

Not even --help flag itself is standard, many tools only support -h, or not even that. It's truly an unfortunate and unnecessary mess.

I don't think that really matters though. I'd wager that you can write 5 or 6 simple parsers and get 99% coverage of commands used.

Fish does something like this (it scans manpages) but generated completion is mostly useless in more complex cli programs (for example: kubectl, docker). So fish is shipped with a bunch of manually written completions for many commands.

Docker looks like it would be totally doable to me. On every page, all the options are in a standardly structured "Options:" section, all commands are in a standardly structured "Commands" section, and --help and the Options: parser will also work on the subcommands.

It won't have typing, but it'd work. Do the most common argument parsing libraries from the standard libraries of the most popular languages and you're mostly there.

As a maintainer, I'm sick of having to include all sorts of different stupid autocompleters and think that it should be the job of an intelligent shell.


For things like docker simply parsing the help isn't enough. Docker's environment can get pretty complex, pretty quick. There are a thousand different arguments that require different kinds of inputs (think ports, mounts, images id, container I'd and much more). You cannot figure out what the argument type is from just the help or manpages. This is the case for many, many applications.

If by '99% coverage' you mean it can complete the command's options, and know which of them take arguments, and maybe even know about sub-commands, yes. zsh's _arguments, and its _gnu_generic wrapper, can do that — just compdef whatever command to _gnu_generic and it'll pull the options and descriptions out of the --help output for you. fish does something similar, and maybe bash-completion has it too, idk.

But, even with relatively simple commands, that type of completion is very limited compared to the contextual functionality that proper zsh completion functions provide. These functions know...

* Which arguments are meaningful to complete more than once — there's no reason to complete `-a` again after you've already entered `ls -a`, but an option like `-v` might be cumulative

* Which arguments are meaningful to complete in the presence of other arguments — there's no reason to complete `-A` if you've already entered `ls -a`, since those two options are exclusive

* Whether the command supports permutation — GNU tools usually do (`ls mydir/ -al`), BSD ones usually don't (`ls -al mydir/`)

* Whether options can be stacked — some commands require `-a -b -c` instead of `-abc`

* How options can be joined to their optargs — GNU supports both `--foo bar` and `--foo=bar`, curl supports only `--foo bar`, grep supports both `-m 3` and `-m3`, tree supports only `-L 3`

* How the current argument might be affected by a previous one — if you've entered `make -C mydir` it's useful to complete targets from mydir/Makefile

* How to complete multi-part arguments — if you press tab after `ssh foo@` you probably want to complete a host name or IP

* The descriptions of arguments — zsh can explain what the difference between `always` and `auto` is, and even when an argument is arbitrary, it can explain what format it takes, what the default is, what the min/max are, &c.

* Probably most importantly, the values of optargs and operands — just blindly offering file names or presenting a metavar parsed from --help (like WHEN or TYPE) is not very nice; a useful completion function will offer package names, man-page names, compression levels, PIDs, NIC addresses, or whatever else is appropriate

I know it's technically possible to do some of this with bash completion functions, but most don't, probably because the infrastructure isn't there. fish is nicer, but even it doesn't (can't?) usually go that far with it.

This level of richness (along with related UI stuff like menu selection) is why a lot of people switch to zsh even if they don't really care about its scripting features and modules and stuff. It's probably why i started using it originally.

A way to specify completion behaviour declaratively would be really cool, but if it's just going to be dumbed down to bash-like functionality, i'm not sure it's going to gain much more traction with zsh users than _gnu_generic and bashcompinit (zsh's bash-completion compatibility shim) have.


bash-completion already does that, and it works OK, but is limited. It's used on most Linux distros, so you've likely already used that code path.

It actually invokes --help dynamically you hit `<TAB>` and parses the output dynamically. Whereas other systems do it offline in a batch job, e.g. some fish completions.


Why not use language server specs for something like this?

wish there was more autocomplete in powershell

Well, before even writing anything, I've looked up into PS docs, and noticed Register-ArgumentCompleter cmdlet. It could definitely be used for autocompletion purposes, but we need something like a database of autocomplete definitions for common utilities to get started, and a tool to read such a database. So, I propose to create a format for autocomplete definitions and invite everyone to discuss it.

Dreaming about Microsoft implementing proper autocomplete hooks in cmd.exe.



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: