
Convert curl commands to Go code - pythonist
https://mholt.github.io/curl-to-go/
======
mholt
Hey HN, I wrote this to scratch a perpetual itch after reading enough API
documentations and re-writing basically the same code over and over. Right now
it only supports the most common flags that I saw. As far as I'm concerned the
project is finished, but PRs are welcome to cover any glaring omissions or fix
bugs.

Someone also showed me this tool which does the same thing but supports more
languages and options:
[https://shibukawa.github.io/curl_as_dsl/](https://shibukawa.github.io/curl_as_dsl/)

~~~
mlangdon
Very cool. Flags I commonly see/implement (curl to Python or Ruby usually)
include -o (download) and -k (ignore ssl warnings -- you might want an warning
comment if you implement that one).

------
travjones
Great work, mholt!

He also wrote json-to-go[0]--copy/paste JSON and it generates a matching Go
struct. This ends up being extremely useful and saves a lot of time for
extensive JSON responses.

[0]: [https://mholt.github.io/json-to-go/](https://mholt.github.io/json-to-
go/)

~~~
pram
I use this all the time! Thanks for your work op

~~~
mholt
Thank you :) Glad to know it helps more than just me.

------
alpb
This is awesome. Combined with Chrome Developer Tools' "Copy request as cURL"
feature, you can reverse engineer some service and turn it into Go code so
easily.

~~~
infogulch
So how about a variation that takes a curl command and writes Go http server
code that would _accept_ that request?

~~~
jerf
Speaking generally, it's more difficult to do much useful with that, because
it's hard to do much more than route on the URL. You could prototype code that
merely looks at the mentioned headers, but it's getting kinda silly at that
point.

More specific to JSON, you may be looking for something like
[https://mholt.github.io/json-to-go/](https://mholt.github.io/json-to-go/) (by
the same author) or
[https://github.com/ChimeraCoder/gojson](https://github.com/ChimeraCoder/gojson)
(which I've used myself).

(I find that what I pay in using a static language like Go to manipulate JSON
and having to use a tool like that often comes back to me pretty quickly when
I take the tool output as a template and start turning all the structs into
Real Objects (TM). The dynamic languages are pretty good at taking JSON and
yielding a simple pile of dict/map/array/strings/numbers/etc., but if you want
to get real objects back out of them the advantage over Go evaporates. Not
because the dynamic languages make it "hard", but just because with one of
these tools, both Go and the dynamic solutions are roughly as easy.)

------
merlincorey
Given the connection with Chrome Developer Tools' "Copy request as cURL" I can
see the usefulness of this in a general sense; however, I was personally
somewhat sad to see that it doesn't actually convert to code that makes use of
libcurl.

As such, the resulting project is very limited in exactly which cURL
commandlines it is capable of converting, and, particularly, it is only able
to handle HTTP connections (unless I missed something).

A generator that could generate libcurl compatible code would have the full
power of cURL!

------
vessenes
This is really neat. There's, I don't know if it's a niggle or a bug or an
interface issue, but well, let's call it a usability bug.

If your copy to curl request in chrome accepts gzip and/or deflate, then your
tool will add an "Accept-Encoding" Header. So far so good.

In go 1.5.1 at least, from my cursory testing, the runtime will not auto-
decompress a bytestream if you've manually specified an accepted encoding.

On the other hand, if you leave it off, it will offer gzip to a server and
auto uncompress on receipt.

I'm still not sure this is a bug, but I definitely commented out my accept
encoding line in my sample code because I don't want to manually decompress
every response.

At any rate, thank you for making this; I'm using it with a slack integration
right now.

------
joelg236
Another alternative is Postman [0]. Supports C, Java, Go, Ruby, Swift, etc.
Available as a Chrome app. Not really the same (doesn't convert from one to
another AFAIK), but it gives you a common interface.

[0]: [https://www.getpostman.com](https://www.getpostman.com)

~~~
a85
Thanks for the mention. Postman can import curl commands too and then of
course convert to all these languages. We are opening this up so that
converters can be added/coded dynamically.

------
zobzu
Note that by default, curl itself can also output C code version of the CLI
command as well (yay)

------
andersonmvd
It's not that easy to parse this curl command. He needed 400 lines
([https://github.com/mholt/curl-to-go/blob/gh-
pages/resources/...](https://github.com/mholt/curl-to-go/blob/gh-
pages/resources/js/curl-to-go.js)). Most libraries out there do this job, but
are focused on parsing from environment variables instead of a string
argument, because the programs that needs this usually are CLI programs.
That's what I found when doing the parse in ruby which basically has only
OptParse. There is an opportunity here. Projects like these that receives a
command input are very useful.

~~~
jerf
It turns out options parsing is fundamentally hard. It looks simple at first,
but then it turns out that if you want to support everything anybody could
possible want, you end up with some too complicated to use. Nor is it obvious
which "middle ground" is correct. It's a perfect environment to produce
exactly what we get, which is a bajillion options parsers for every halfway
mature language in the world.

Also, that's not 400 lines of parsing code, that intermixes "parsing" and
generation all together.

Mind you Go is not a good language for writing parsers in. (It isn't
_especially_ bad, like, say, C, but it lacks almost every conceivable feature
that could make parsing code anything other than very, very verbose.)

~~~
hyperpallium
Just to expand on this: options can be position-dependent, and treat options
as an algebra (a --and b --or c); or, just executed in order to eg change
directory (-d dir1 a b c -d dir2 x y z).

Basically, by treating the arguments as a stream of tokens, it can be parsed
with the same power used for parsing the source of programming language.

Yet for many uses there are natural complexity-levels for option parser tools
(like position-independent boolean options, and options taking one argument),
hence the many tools for many niches.

And they are easy to write, so everyone writes one.

------
hyperpallium
\tangent Wouldn't it make sense for this to be a library? Though most of the
py looks so simple, it already has the libraries.

Or, to parse the curl command string at run time, so it can easily be human-
read, as a curl command?

But I guess the idea is to generate a start-point template, which can then be
customised for whatever is needed.

It would be nice for a tool to combine both - so new code is attached to the
curl command string... but not very readable... and not very flexible (I bet
the next thing you wanted to add would be something that hadn't been
considered...)

------
lspears
A couple of these tools have sprung up:

JSON: [https://mholt.github.io/json-to-go/](https://mholt.github.io/json-to-
go/)

CURL: [https://mholt.github.io/curl-to-go/](https://mholt.github.io/curl-to-
go/)

SQL: [https://github.com/knq/xo](https://github.com/knq/xo)

The JSON tool especially has saved me a lot of time.

------
misiti3780
great idea - i wish this existed for every language !

------
MasterScrat
Would love the same thing for python!

~~~
sabraham
[http://curl.trillworks.com/](http://curl.trillworks.com/)

~~~
dopeboy
I'm working with the PayPal API right now (send your condolences) and all
their documentation is in the form of CURL requests. Just finished converting
it to python requests commands and now this gets posted.

TIL'd.

------
kilotaras
Looks like backtics aren't sanitized

    
    
        curl -XPOST httpbin.org/post  -d '{"sneaky backtick": "`"}'
    

produces

    
    
        body := strings.NewReader(`{"sneaky backtick": "`"}`)

------
hyperpallium
Super minor point: according to the curl manpage, _-d data_ implies POST, so
you don't need _-X POST_ in the examples

------
nodesocket
Go question. Is:

    
    
        if err != nil {
            // handle err
        }
    

Required, or can you also do:

    
    
        if err {
            // handle err
        }

~~~
mnbbrown
There's no implicit comparison (if that's the CompSci name) in Go. You have to
explicitly compare (!= is required)

------
sylvinus
Nice! Small nitpick: "-A" doesn't seem to be supported.

~~~
infogulch
Yeah I don't see it in the list at the top of the page. But the author says
that he'd be happy see a pull request for common options that aren't supported
yet.

------
dkarapetyan
Go is really verbose but this is really great.

~~~
cm3
It's interesting that all the error handling branches are left empty. What
does production Golang code do in that case as a common idiom?

~~~
kasey_junk
One of the (possibly contentious) principles in Golang is that each error
needs to be addressed individually with an appropriate response to the
particular case in question.

In practice _most_ Golang errors are handled either:

1) Ignored

2) Logged and swallowed

3) Returned at the point it happened.

4) Fatally exits the program.

One of the "features" of the errors as return codes is that you don't get
surprising results that come from the goto style jumps that happen with
something like exceptions (unrolling state, etc).

~~~
tptacek
That's not really true. It's certainly an _idiom_ in Golang to handle errors
individually, but it's not a principle.

The principal is that errors are values like everything else, and you can
write code to handle them however you'd like. See, for instance, the "write"
example in this post:

[https://blog.golang.org/errors-are-values](https://blog.golang.org/errors-
are-values)

There are libraries that people use (sqlx might be an example) that are more
like that example than the if-error-return stuff here.

~~~
kasey_junk
>The language's design and conventions encourage you to explicitly check for
errors where they occur (as distinct from the convention in other languages of
throwing exceptions and sometimes catching them)

[http://blog.golang.org/error-handling-and-go](http://blog.golang.org/error-
handling-and-go)

