
Show HN: A complete Bash parser in JavaScript transpiled from Go - mvdan
https://www.npmjs.com/package/mvdan-sh
======
munificent
This headline reads like a Markov chain trained to generate Hacker News posts.
I love it.

~~~
tlrobinson
Idea: a hackathon where you're required to implement a project with a title
generated by a Markov chain.

Bonus points if the code is written _by_ a Markov chain.

~~~
gargravarr
I think it would still be more readable than half the source I work with on a
daily basis...

------
Zalastax
An important heads up: "Parsing Bash is Undecidable".
[https://www.oilshell.org/blog/2016/10/20.html](https://www.oilshell.org/blog/2016/10/20.html)

~~~
mvdan
Yep, I've exchanged a few emails with Andy before, the oilshell author. I
started the parser in early 2016, so we were both getting started around the
same time.

If you look at my README's caveats section, you'll see a very similar story -
only accepting programs that can be parsed statically in a simple way.

------
ddtaylor
Not that it's useful or desired, but could one transpile it into pure bash as
well? It's Turing complete isn't it?

~~~
mvdan
I'm not sure I understand what you're saying. If there was a Go->Bash
transpiler (or a JS->Bash one), yes, we would theoretically end up with a Bash
parser and interpreter in Bash.

I imagine it would be quite the monster, though.

~~~
solarkraft
Luckily the current creation isn't already one.

~~~
mvdan
I'm going to go on assuming that you're not being sarcastic :)

As I mentioned in another comment, once Go's wasm support is shipped and
stable, I hope to make the JS package smaller and better. Until then, this is
the best I can do without manually rewriting the Go code in JS.

------
monocasa
> transpiled

I still hate this term.

~~~
mvdan
Is there a more appropriate word for source-to-source transformation? I
understand that compilers aren't that different, but generating machine code
is quite different.

~~~
monocasa
> but generating machine code is quite different

It really isn't though. PCC just walked the AST, writing out text to a file
with hardly any optimizations just like most of what people call
'transpilers'. But you wouldn't call a c compiler to asm a transpiler, right?
Pretty much everything that gets attributed to some intrinsic difference
between source-source and source-machine compilers is just a function of the
immature tooling on the web.

~~~
sigjuice
Also, _source-to-source compiler_ can be shortened to _compiler_. The phrase
_source-to-source_ is redundant.

~~~
tlrobinson
Should this Wikipedia page not exist then?
[https://en.wikipedia.org/wiki/Source-to-
source_compiler](https://en.wikipedia.org/wiki/Source-to-source_compiler)

~~~
monocasa
I mean, it should exist according to wiki's rules since there are so many
people using the term. Wiki intentionally doesn't try to independently verify
any research, but just serves as a collation of ideas.

What I would do is ask 'what part there is any different than any other
compiler?'

~~~
tlrobinson
It's a type of compiler. Just like a truck is a type of vehicle, and a car is
another type of vehicle.

~~~
monocasa
Except that there's a true formal (in fact legal) distinction between a car
and a truck (they have different emissions standards). The whole distinction
between a 'transpilers' and other compilers on the other hand is 'what is the
user going to do with the output'.

~~~
tlrobinson
That's not the disinction I've seen. I like Wikipedia's:

> translates between programming languages that operate at approximately the
> same level of abstraction

That's fairly straightforward, though a bit subjective (which doesn't preclude
a word from having meaning)

------
mvdan
I'm not very familiar with JavaScript and npm - looking for suggestions on how
to make the package easier to use.

~~~
crooked-v
Some general thoughts, which may or may not be practical depending on how the
transpiling works (I have no idea about go):

\- Instead of calling .Print(), have the class extend EventEmitter (node
builtin,
[https://github.com/primus/eventemitter3](https://github.com/primus/eventemitter3)
is a good browser shim for the same API), with an event that fires once for
each line that terminates with a newline.

\- Hard mode of the above: Add streams functionality (node builtin,
[https://github.com/nodejs/readable-
stream](https://github.com/nodejs/readable-stream) is an official browser shim
for the same API), with output streams that operate on bytes instead of lines.

\- Combine parser and printer into a single ES6 class (e.g. using `new
Parser()` syntax, rather than the indirect object generation).

\- In optional addition to the above, have .parse() return a Promise for the
output (not the parsed program), with parsed programs stored on the Parser
itself (and examinable via an additional class method).

\- Change all method names to be in camelCase instead of PascalCase. Only
classes (as instantiated using `new`, not just methods that generate class
objects) get PascalCase.

~~~
mvdan
Thanks for the suggestions! I'll need to read up on these js/node features.

One reason that the API is a bit clunky is that it tries to mimic the Go API
closely, so that the documentation and examples are reusable.

However, your points on the string returns are very valid. I replaced all of
Go's readers and writers (byte streams) with strings, simply because I didn't
know of a better way.

It definitely sounds like the features you suggested would be better. If the
transpiler (gopherjs) supports them, I'll definitely give them a try.

~~~
crooked-v
To help, the short version on "why" for EventEmitter, streams, and Promises is
that they're three different ways of doing things asynchronously, with
different use cases and details.

EventEmitter - You expect to intermittently get individually completed values
that you do whatever with and then discard. These values may be grouped into
useful sets by the EventEmitter (each separate event type).

Streams - You expect to get raw partial data that you want to do your own
buffering and handling with. You want to be able to feed this raw data to a
different target (e.g. file writing) without handling the fine details
yourself.

Promise - You expect to get a single completed value when an action is
completely finished, or (for a Promise without a return value) you just want
to have something contingent on an action finishing.

An example of each:

Streams: A TelnetConnection class that handles connecting to a server, then
supplies a stream with the bytes from the connection (with each chunk being
whatever is convenient for buffering purposes), and ends the stream when the
connection drops.

EventEmitter: A TelnetHandler class that uses TelnetConnection and fires an
event for each complete line (ending with \n).

Promise: A TelnetHttp class that has a method get(hostname) that uses
TelnetHandler and returns a Promise for the full text return value of doing a
simple HTTP GET call.

~~~
mvdan
It sounds like I should use streams then. After all, that's what the Go API
uses, Readers and Writers. Those simply pass chunks of bytes around.

Thanks again for the help!

------
ClassAndBurn
And now my head hurts.

~~~
mvdan
This is hacky, but it beats having to write a shell parser from scratch again
:)

Once wasm support is shipped with Go, I hope to be able to accomplish the same
without a transpiler. I'd also hope that the final package would be smaller
and more performant.

~~~
JepZ
I didn't even know they were working on wasm for Go, but it looks like there
was some progress during the last months:

[https://github.com/golang/go/issues/18892](https://github.com/golang/go/issues/18892)

------
hestefisk
Love this. Ironic that it is BSD licensed though, considering it parses and
runs the world’s probably most popular piece of GPL’ed software (probably
second to Linux).

~~~
mvdan
Most Go code out there is under permissive licenses, I assume because static
linking is the norm.

I hope I don't get an angry email from RMS at some point :)

