
Bash Completion from JSON Fields - medv
https://github.com/antonmedv/fx-completion
======
accrual
Tangentially related: PowerShell offers something similar for XML and JSON.

    
    
        # XML:
        [Xml]$Data = Get-Content Data.xml
        $Data. <tab> # browse the file with <tab> and <.>
        
        # JSON:
        $Data = (Get-Content Data.json | ConvertFrom-Json)
        $Data. <tab> # browse the file with <tab> and <.>
    

Similar in concept and certainly useful. I'm happy to see this project!

~~~
fxfan
The new rule on HN is that anybody who talks about Powershell has to share
their profile ;)

\-- Happy pwsh user on arch.

------
arketyp
One thing that strikes me as a person who grew up with mouse pointer
interfaces (starting with Workbench on Amiga) is how strange it is that the
terminal allows for a time-lag between a mistake in typing and the user's
becoming aware of it by pressing Return. Intense tab spamming removes this
alienating phenomenon though, and I imagine that with a perfectly smart
terminal (and commands), virtual tab-spamming is automatic so that a user is
immediately informed and even prevented from typing something that could not
possibly render a valid command.

I suppose this is how voice command will evolve too. Cortona will stop you
short when you're not making sense, and even learn your idiosyncrasies in
expressing yourself much like a spouse does.

By the way, I for sure thought this link was about a the complete J.S. Bach
collection.

~~~
chubot
That's an interesting invariant that I've been thinking about: hitting TAB
should never generate something that would be an invalid command (or more
pedantically, generate a command prefix that would have a syntax error once
the user fills in the rest of the command).

You would THINK that shell tab completion already works that way. But it
doesn't in bash, and not even in fish or zsh.

Not only does the completion logic duplicates the tool's own parsing logic, it
doesn't attempt to do it faithfully.

(1) First, there's version skew in all the repos for bash, fish, and zsh.

(2) Leaving that aside, the parsing models they use aren't rich enough to
express some command lines.

"find" is a good example. They will let you do something like

find -type f -dept<TAB> \-- completes -depth, but it's not valid here

This is because find is split up into options and the expression language.
There are probably some simpler and more compelling examples I can come up
with.

Another one is completing flags in this situation:

grep -- --c<TAB>

After --, there are only args, not flags, so it's misleading to complete.

\----

Being able to spam TAB is what I'm going for with Oil. That's not true of
bash. One annoyance the (y/n) prompt that bash gives you when there are
thousands of completions (e.g. hitting TAB on an empty line).

I have some upcoming blog posts along these lines, which I mention here:
[http://www.oilshell.org/blog/2019/02/05.html](http://www.oilshell.org/blog/2019/02/05.html)

~~~
arketyp
Glad to hear someone is thinking about this. Right, hitting tab should never
generate something invalid. That's the shell being sound. Conversely, the
system should also be able to determine whether what's being typed could
possibly become a valid command or not. That's the shell being complete.

~~~
chubot
Yes, I think it basically reduces to the problem of using the tool's ACTUAL
parser. There are enough variations in flag syntax that it's not possible to
be 100% correct without running arbitrary code. And if you're going to do
that, you might as well run the code that's already written (with some
caveats).

That could be solved by my proposal for a "Shellac protocol", akin to
Microsoft's Language Server Protocol (e.g. for VSCode):

[http://www.oilshell.org/blog/2018/12/05.html](http://www.oilshell.org/blog/2018/12/05.html)

Summary:

1\. The protocol was mainly motivated by the problem of bootstrapping a corpus
of shell completions. Every shell has to "boil the ocean" itself, i.e. provide
completions for many popular commands. git's completion is thousands of lines
in each of bash, zsh, and fish. Upstream only maintains bash.

2\. What I noticed in implementing completion for Oil is that bash
fundamentally CONFLATES two problems:

a) The problem of completing the shell language itself, e.g. ${H ->
${HOSTNAME, or ~a -> ~andy

b) The problem of completing command line flag syntax.

This is a huge mistake from the POV of implementation complexity. And
obviously if you want shell-agnostic auto-completions, (b) needs to be
separate from (a), which is shell-specific.

Luckily the authors of bash-completion tried to separate these two problems a
bit, and I was able to salvage most code.

Since I wrote that post, and implemented a compatible completion API in OSH,
it's become a lot clearer to me how the protocol should work.

3) The invariant you mention would be a third benefit of having a more
principled completion API that can run arbitrary code (in arbitrary
languages).

Now, I'm not saying that EVERY command needs to use this. There are some
engineering problems with this approach too. But I think you should have the
POSSIBILITY of doing something correct if you want. Heuristics may be OK in
the simple cases.

But overall what I discovered is that the system is a huge mess. The bash-
completion project tries to parse bash in bash, and unsurprisingly does a poor
job. My upcoming blog posts will explain this.

------
sonofgod
Nice!

The results seems somewhat muddied by the long lists. Have you considered
omitting all but the first and last entries?

~~~
bewuethr
There is a very useful readline option to help with this:

    
    
      set completion-prefix-display-length 2
    

in ~/.inputrc replaces any common prefix longer than two characters with an
ellipsis.

------
Existenceblinks
General Developer Experience (DX) speaking, IMO, File and Folder are kind of
better at CRUD-ing json. One file per attribute, and foldable attributes are
folders (object and array types). It scales quite well rather than loading a
big file into memory.

Tools around file system are battle tested. Basic operations; read=ls,
create=touch/mkdir, update=mv, delete=rm. Linking data = symbolic link. File
explorer/manager helps managing attributes much nicer than in-file operations.
It can even use extension as types, structural folders as schema.

jq is nice but you've already got: find, grep, awk, sed programs.

The only downside I can think of is size might be larger depending on what's
minimum block size (e.g. 4096 bytes) hmm

~~~
medv
Just bunch of nonsense.

~~~
Existenceblinks
Could you elaborate which part is nonsense?

------
natch
After reading the four-word-long README text and viewing the gif, I have no
idea what this is or does.

~~~
hibbelig
Yes. Command line completion for fx is pretty obvious, if you know what fx
does. It's missing a link to fx.

[https://github.com/antonmedv/fx](https://github.com/antonmedv/fx)

~~~
natch
I wasn't asking, but thanks. Was more making a point about the shitty readme.

------
chubot
I was going to suggest that somebody try this in Oil shell, which can now run
thousands of lines of bash completion scripts [1], but I can already see it
won't work because of the VERSION DETECTION anti-pattern.

[https://github.com/antonmedv/fx-
completion/blob/master/compl...](https://github.com/antonmedv/fx-
completion/blob/master/complete.sh)

    
    
        if [[ -n "${BASH_VERSION}" ]]; then
          # ...
        else
         echo "Unsupported shell."
         exit 1
        fi
    

The better thing to do is FEATURE DETECTION, e.g. with 'eval'.

Does anyone know of a concise resource for explaining feature vs. version
detection? I don't find that many great resources on Google.

Here's a page, which is OK but leaves a lot to be desired.

[https://en.wikipedia.org/wiki/Feature_detection_(web_develop...](https://en.wikipedia.org/wiki/Feature_detection_\(web_development\))

It feels like I need to write a blog post to explain this, since this
knowledge seems to be waning. I recall reading about it 10+ years ago, but not
recently.

Richard Stallman understood it very well and IIRC strongly advocated for
feature detection in autoconf. (Despite all the bad things about autoconf, it
does actually work on tons of platforms, and it's solving a hard problem. It
wouldn't "scale" if it were using version detection.)

Here's an outline of a blog post:

1\. Version/browser detection is why the User-Agent string from Internet
Explorer, Chrome, Safari, Edge, etc. all start with "Mozilla". Fun fact: if
everyone used feature detection, we could have saved exabytes of data transfer
over the last 20 years by trimming 10-50 bytes from every single HTTP request
ever made.

2\. jQuery switched from version detection to feature detection early in its
life. I recall some threads where Resig was criticized for this, and then he
saw the light.

3\. Likewise, the LLVM toolchain has to pretend its GNU in some cases because
applications are using version detection:
[https://news.ycombinator.com/item?id=15195370](https://news.ycombinator.com/item?id=15195370)

4\. Autoconf is based on feature detection.

5\. My hand-written configure script also uses feature detection, e.g. to
detect the presence of the readline library. It basically tries to compile a
small program with the C compiler. There are various tricks in this domain.

6\. eval is useful for feature detection in dynamic programming languages. In
fact, this might be the ONLY thing eval is good for! (exaggerating a bit)

[1]
[http://www.oilshell.org/blog/2019/02/05.html](http://www.oilshell.org/blog/2019/02/05.html)

~~~
medv
Feel free to send a PR with feature detection. =)

------
rc_kas
Does it work with json over http?

~~~
medv
See examples.

