
Neugram – Scripting in Go - golangnews
https://neugram.io/blog/neugram-briefly
======
kornish
"Go is syntactically a big language. Neugram’s front end has to match it, and
that’s a lot of work, much of which is still to do."

I appreciate the sentiment – that there's a lot of work left to do – but isn't
the assertion about syntax size false? I thought Go's syntax was quite small
compared to other popular languages? It only has 25 reserved words (Java has
50+) and was designed in part to be uniform and easy to read.

~~~
crawshaw
I made this point poorly, because I agree with you that Go is quite easy to
read and very predictable.

With more words: I have written a parser and type checker for an ML-style
language, with parametric types and several other neat tricks in it, and I've
now written a parser and type checker for a good subset of Go. The latter has
been far more work. I am not entirely sure how to explain the work. Go has
lots of syntactic and type conveniences that are easy to use and read, but
quite difficult to implement.

As there are few implementers and many users, I think the original designers
of Go picked well when they put the work on us implementers.

~~~
ainar-g
Can you elaborate on what syntactic conveniences are difficult to implement
and why? Language design is on of my hobbies, so I really would like to know
that.

~~~
crawshaw
One good example is untyped constants. They form a large extension to the type
system only present at compile time. It is a good example because there is an
excellent blog post describing them in detail:
[https://blog.golang.org/constants](https://blog.golang.org/constants)

In particular note the difference between

    
    
        const hello = "Hello, 世界"
    

and

    
    
        const typedHello string = "Hello, 世界"
    

one of these can be assigned to named string types, the other cannot.

As a user of Go I find untyped constants to be extremely useful. They almost
always do what I want without surprise. Implementing them however, is not
trivial.

A tricker example is embedding. It adds a lot of complexity to the type
system. As a user, it can be a bit surprising, but I admit it is very useful.

------
bakul
How does this compare with
[https://github.com/cosmos72/gomacro](https://github.com/cosmos72/gomacro)
which does something very similar? It seems fairly complete, or at least much
further along than neugram + it does lisp style macros! If you ignore the
macros its language is pretty much go except that main(){ ... } is elided just
like in ng.

May be worth checking it out, at least for ideas/inspiration.

------
indescions_2017
100% behind you on this, crawshaw. Although I've never missed having a Python
style REPL for golang ("go run" usually fits the bill). You make a convincing
argument for why it would indeed be a very powerful tool for any Linux admin's
toolbox ;)

It's premature for feature requests I know. But I'd love to be able to view
the Stack Trace of a live running goroutine. Typically something I log to disk
using the "runtime" calls. To be able to probe that info from a command shell
would be neat, don't you think?

------
tedunangst
It's not clear to me where I would go from this page to try it out, or install
it, etc.

~~~
crawshaw
Yes, that sounds like the kind of mistake I would make.

I put a tiny link at the bottom of to the home page:
[https://neugram.io](https://neugram.io). I'll do something more elaborate
when I'm done with this other bug I'm working on. Thanks.

------
dom96
This is really cool, but I can't help but suggest another language which works
incredibly well for scripting: Nim[1].

Of course, I must put a disclaimer: I am incredibly biased as I am of Nim's
core developers. But to help convince you, Nim has:

* Shebang support

* Error handling via exceptions

* A somewhat limited REPL

* Operator overloading

While I applaud the author's efforts, I can't help but wonder if Nim would fit
their needs (or be easier to adjust to their needs).

1 - [https://nim-lang.org](https://nim-lang.org)

~~~
civility
Nim has become so good that no one else should develop languages as pet
projects? If that's what you're saying (which is how I read it), it's a very
obnoxious sentiment.

~~~
veidr
I took it more like "if you clicked on that, you may also be interested in
this" (which I in fact was).

------
dragonfax
I like this. Especially the error handling. Since the focus on a scripting
language can be brevity (Ala perl) I don't want to write all that error
handling boilerplate. But I don't want to ignore issues either. So this is a
compromise. I can check errors, I can capture errors (in _) to ignore them.
But if I don't do anything, they turn into panics.

------
rileytg
this reminds me of the effort to replace a lot of the intel management stuff
with linux and have userland all written in go

[https://schd.ws/hosted_files/osseu17/84/Replace%20UEFI%20wit...](https://schd.ws/hosted_files/osseu17/84/Replace%20UEFI%20with%20Linux.pdf)

------
frou_dh
Glad to see this using
[http://github.com/peterh/liner](http://github.com/peterh/liner) for
interactive line editing.

I used that package in an interpreter of my own and it works wonderfully to
provide `readline` style keyboard interaction in a pure Go program.

~~~
crawshaw
It really is an excellent package. Does exactly what it says on the box.

I fear I may hit a wall with it eventually. For example, several interesting
approaches to
[https://github.com/neugram/ng/issues/49](https://github.com/neugram/ng/issues/49)
would step outside what liner can do. But I'm going to use it as long as I
can.

~~~
frou_dh
Among other things, I actually modified liner to stop using/swallowing all
control codes, and to instead surface some of them to the client program to
decide how to respond. e.g. liner stops doing anything with Ctrl-X (0x18 ?)
and instead ng could respond to Ctrl-X by writing a docstring to stderr for an
identifier under the caret in the input buffer.

[https://github.com/frou/liner/blob/master/common.go#L208](https://github.com/frou/liner/blob/master/common.go#L208)

------
weberc2
It seems like something like this would be useful for a Jupyter kernel. I
don't think I'd use this as an embedded scripting language; it doesn't offer
enough in terms of abstraction and conciseness to justify leaving Go. I'd
probably try for a lisp.

~~~
crawshaw
Jupyter is a good idea that was only recently suggested to me:
[https://github.com/neugram/ng/issues/16](https://github.com/neugram/ng/issues/16)

Definitely something to try.

------
jeffinhat
This is awesome. I've been trying to reach for go instead of bash more and
more and this is exactly what I need for the small stuff.

Any ideas how it could relate to the go toolchain? could I stick my Neugram
files in a repo and go get them?

~~~
crawshaw
The relation to the go toolchain right now is your GOPATH is used to find Go
packages when you import them. (It uses the go tool to build the package as a
plugin for loading.)

You certainly could put your Neugram files in the same git repository as your
Go code, and then "go get" would get them.

I think there is something to be said for something like:

    
    
        import "github.com/crawshaw/foo/mypkg.ng"
    

looking for the .ng file in your GOPATH. I need to think about that a bit
more.

(Note that a Neugram package is limited to a single file, unlike a Go package.
This is for a couple of practical reasons, notably init order, and one
philosophical reason: which is Neugram packages shouldn't get as big as Go
packages.)

------
ericfrederich
No shebang support? What about this? Sure it's not "native" but it
accomplishes the same thing.

[https://rosettacode.org/wiki/Multiline_shebang#Go](https://rosettacode.org/wiki/Multiline_shebang#Go)

------
singularity2001
even cleaner python: with open("hello.txt","w"): f.write("Hello, World!")

------
lox
This will be amazing for plugins!

------
wybiral
Sometimes switching between Go and C is weird (because of the type order or
the semicolons) but I've never had any trouble switching between Go and
Python.

Go has an awesome ecosystem for writing servers and web applications but it's
obvious that the libraries weren't built for scripting purposes and the
language is more explicit (for good reasons).

Python, on the other hand has an ecosystem that's great for scripting and
language features that are more compact and terse.

~~~
crawshaw
If you don't have trouble switching between Go and Python, then the concept of
Neugram has much less to offer you than me.

Possibly you don't because you use both languages more than me. My scripting
needs are approximately once every other week, sometimes even longer. That's
enough time for me to forget what python package has the date/time functions I
need.

I also wouldn't even begin to compare something in its infancy like Neugram
with something as well-established as Python's huge collection of libraries.
There are many years of work between here and there.

~~~
wybiral
> Possibly you don't because you use both languages more than me.

It depends what the client or project demands. I've worked in Python,
Javascript, PHP, VBA, Java, Go, and C (plus a few dialects of SQL). Spending
months writing mostly one or two of those at a time. The only ones I enjoy
enough to use in my hobby projects are Python, Javascript, and Go.

But in my experience the hardest part of toggling between languages like that
is the ecosystem (libraries and tools).

In this case it seems like you're adding a new ecosystem which means I'd still
have to stop and figure out "what's the right package to use for this?"

For the most part, language is easy. It's the idioms, libraries, and tools
that I find myself googling the most.

~~~
crawshaw
My goal is very much to reuse Go packages. It is somewhat working today. The
underlying mechanism is when you type something like:

    
    
        import "github.com/pkg/errors"
    

A .go wrapper file is generated, a Go plugin is built, and then loaded into
the ng executable. You can then use the errors package in the Neugram
evaluator. Under the hood the lifting is done by the reflect package.

Very much my goal is when scripting, to use the os.Open and ioutil.ReadFile I
know, along with filepath.Join, time.Now, and friends.

------
nanodano
You can already "go run program.go" from anywhere, even outside of a project
structure, which is the same result as running an interpreted script.

It's a novel idea but I don't really see it taking off. Is there really that
much benefit to having a shebang at the top of the file versus executing "go
run" and is it really worth having a REPL for a fairly verbose language? I
certainly don't see myself wishing for a Java REPL, and I don't particular see
myself reaching for a Go REPL either.

~~~
crawshaw
The goal is to add other features that are also useful for scripting. The big
one I've worked on so far is syntax for the os/exec package:
[https://github.com/neugram/ng/blob/master/docs/shell.md#a-sh...](https://github.com/neugram/ng/blob/master/docs/shell.md#a-shell-
session)

Another thing I'm interested in is operator overloading, for compact/pleasant
matrix/table types.

~~~
dgacmu
Y'know, if you happened to accidentally create nice bindings for using
TensorFlow in the process... :)

