
Go generate: A proposal for a Go preprocessor - cpeterso
https://docs.google.com/document/d/1V03LUfjSADDooDMhe-_K59EgpTEm3V8uvQRuNMAEnjg/edit
======
cageface

      Motivating examples include:
    
      macros: generating customized implementations given
      generalized packages, such as sort.Ints from ints
    

If this is seriously their proposed solution to the lack of generics then my
estimation of Go just dropped even further.

~~~
MichaelSalib
It is a bit astonishing what lengths the go community is willing to, er, go,
in order to avoid compile-time code execution (the basis of lisp macros) or
generics. I understand that neither of these ideas were invented at Bell Labs
during the 1970s, but at some point, can't we just put aside the not invented
here syndrome?

Could we all just agree to pretend that Bell Labs folks invented those ideas?
Would that work?

~~~
zuzun
So how many programming languages besides LISP are powerful enough to support
turning Protocol Buffers into code during compilation? And is this the
ultimate must have?

~~~
pjmlp
D, Rust, Nim(rod), Template Haskell, OCaml, C++...

~~~
mseepgood
I don't think these can open and read .proto files during compilation.

~~~
pjmlp
D has meta-programming and compile-time evaluation, including reading files.

I bet C++ can do it with some clear use of template metatprogramming, compile
time evaluation and processor macros.

Nim has meta-programming capabilities, including reading files.

Not sure about Template Haskell, though.

~~~
thesz
Template Haskell code can runIO I/O actions, including reading files.

------
pcl
_the author commits the generated files to the source repository, so that they
are available to clients that use go get_

Treating generated files as primary source is a disaster waiting to happen.
Someone will end up just doing a quick edit of the derived file because yacc
isn't set up right, or running a sed command to do a refactoring, or just not
understand the difference. One of the things that I like about go is that it
does a good job of keeping the codebase clean; recommending a solution that
sets up this sort of trap is a big departure.

I wonder what other approaches are available for packages that intend to
distribute in source form, however. I could imagine adding a parallel
directory structure for the generated files, to make it clear that they are
out of bounds, akin to the mvn source / target layout.

~~~
mrweasel
I completely agree. If you're going to add generated files to a source code
repository, they need to be clearly marked in one way or another.

If the author truly want a "go generate" they good sell it much better. I
understand the yacc idea, it can be a pretty good solution for something like
configuration files. OpenBSD use yacc to do the grammar for many of they tools
and that has worked out create.

The big sell though would be WSDLs. I understand the aversion towards SOAP and
WSDL, but it's here and having Go support would be nice. Generating Go code
from a WSDL wouldn't be to hard a sell I think, and it fits nicely into this
idea.

~~~
iand
I think you misunderstand the context. go generate is already in Go tip and
will be released as part of Go 1.4

~~~
pcl
Thanks; good to know. Here's a link to the original discussion thread on the
Go newsgroup from July: [https://groups.google.com/forum/#!topic/golang-
dev/ZTD1qtpru...](https://groups.google.com/forum/#!topic/golang-
dev/ZTD1qtpruA8)

------
deskamess
Given the wait and see approach taken by Go maintainers on generics/templates
I think there may be an opportunity for a template based pre-processor scheme
(not using go generate since that works on go files). This approach would be
the C++ approach - i.e., templates not generics.

The template could be in, say for Vectors, a Vector.got file. In your go code
(also a .got file - say my.got for example) you reference Vector<SomeType>.
Some build system+pre-processor parses .got files and creates a
Vector_SomeType.go file. A corresponding my.go file is also generated from
my.got with all template references flattened to real types.

All *.go files are compiled using the regular go build chain to create the
final application. Any missing types would be identified by the go compiler.

make could be used to drive the process, but the got2go translator application
is the missing piece. Is it better than generics inside the language - no. It
is very loose in terms of type safety as we are just performing text
substitutions but it could save a lot of typing. We could also come to rely on
a semi official/community approved repository of core collection .got files.
The C++ STL implementations may serve as an inspiration for the API of core
template code.

~~~
pjmlp
This is what Borland C++ had in the mid-90s before they added support for
template on their MS-DOS compiler.

Way back when STL had just been announced as being integrated into the C++
standardisation effort and C++ compiler vendors were using preprocessor tricks
to generate type specific generic data structures and algorithms.

------
metafex
This is neither a preprocessor in the means of cpp (c preprocessor) nor a
generics implementation. If you need a tool like cpp for go, use m4. It was
made for things like that.

Now, what this is, is a tool to make it more convenient to convert data into
.go source files. I really like the idea that you can just include your
source-data (string-tables, grammars, etc.) in source format, and have one
tool to convert it. Run "go generate" and everything is ready to build.

------
4ad
The title of this submission is incorrect. Generate is not a preprocessor.

~~~
mc_hammer
it really does need a preprocessor though - with a underscore type lib or
generics the code would be so much faster to write and easier to maintain.

------
Mr_T_
Please correct the title. The original title is correct: "Go generate: A
proposal" the rest is wrong. This tool is not a preprocessor. It's as wrong as
calling 'make' a preprocessor. It can be used to call preprocessors or more
likely generators.

------
robmccoll
To me, this seems like a very unsatisfying solution. Not as easy and simple as
the CPP. Not as immediately powerful and integrated as C++ templating.

~~~
mseepgood
How do you use CPP or templates to generate .c files from yacc grammars or
protobuf definitions, to embed HTML or binary data from files? I think you
misunderstood the main purpose of this tool. You probably only read the last
and least important use case.

~~~
robmccoll
I can do this with external tools and build systems (make, cmake, waf, scons +
protoc and any other binary that I want) that at the end of the day work
nicely with the provided tools. Go on the other hand seems to want to be
monolithic. It is the center of your compile and link universe. To me it is
quite un-Unix. Not unlike how limiting the import system is. Many of the
features of Go make it feel like a walled garden.

------
stewbrew

        with some blank lines interposed so the generator directive does not appear in the doc comment
    

It sound's hackish to me to use specially formatted comments to embed these
directives in the source file.

The authors also don't provide an information if users will be able to write
their own generators, do they?

~~~
mseepgood
> The authors also don't provide an information if users will be able to write
> their own generators, do they?

Yes, generator calls are just calls to executable files.

~~~
stewbrew
Oh sorry, I missed the obvious. So it's really rather a simplified Makefile
mixed into the source in the form of specially formatted comments.

------
zserge
I personally like this solution. I don't know why there's so much negative
opinions on this. Yes, it doesn't solve the problem of generics as a language
construct and it doesn't provide macros as a language construct. But it gives
a lot of power for many other real-life applications like
embeddeding/processing some content or generating type-safe
bindings/wrappers/parsers/collections/...

It's an extension of a build system, not a language. It helps to specify build
process details right in the source files that either depend on it, or provide
necessary details for it. This approach worked for CGo, it worked for +build
flags, so I don't see why it will not fit in this particular case.

One thing I worry is how to get the processor app during the build process.
E.g. if bindata is used to embed a file - how will "github.com/jteeuwen/go-
bindata" be installed?

------
shadowmint
less make?

yes please.

no, this wont fix generics, but its not about that. The dependence on make is
an awkward legacy left over and something that other languages (eg. rust) are
also still sitting on, and the sooner it goes away the better.

