

Writing HTTP Middleware in Go - babawere
http://justinas.org/writing-http-middleware-in-go/

======
optymizer

      > a painless and even fun process
    

This claim might be a little too optimistic. I'd like to challenge you
(everyone) to associate a User object with an http.Request. Example code in
Node.js/ExpressJS:

    
    
      function auth(req, res, next) {
        req.User = User("username");
        next();
      }
    

This is such a simple example of a useful piece of middleware, yet it is so
difficult to implement in Go in a 'painless and fun' way.

I would prefer to avoid contention on a global mutex.

I would prefer to keep the compatibility with http.Handler and
http.HandlerFunc.

Good luck!

~~~
elithrar
To be fair, Express is a web framework. Go's net/http package is a little
lower level than that.

Still, this is possible if you make up for the lack of a framework by writing
some code yourself. I'd use (and I _do_ use) gorilla/context[1] to allow me to
pass data in the request context between middleware layers (i.e. CSRF
middleware passes the token along).

You could write a couple of little helpers using gorilla/context
(SetUser/GetUser, or more generically, SetString/GetString) and
SetUser(username, r) in your middleware. The helpers (see the context docs!)
just wrap a type assertion on the Get side, and set a map value on the Set
side.

    
    
       // Compatible with http.HandlerFunc
       func auth(h http.HandlerFunc) http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                    SetUser(r, "dave")
                    h.ServeHTTP(w, r)
            }
       }
    
       // Routes
       r.HandleFunc("/admin", use(adminHandler, auth)
       http.Handle("/", r)
    
       // Allows you to chain your middleware rather than wrapping functions
       func use(h http.HandlerFunc, middleware ...func(http.HandlerFunc) http.HandlerFunc) http.HandlerFunc {
        for _, m := range middleware {
            h = m(h)
        }
    
        return h
       }
    

You can mod this to work with http.Handler if you want: replace HandlerFunc
and then write r.Handle("/admin", use(http.HandlerFunc(adminHandler), auth)
instead.

[1]:
[http://www.gorillatoolkit.org/pkg/context](http://www.gorillatoolkit.org/pkg/context)

~~~
optymizer1
Interesting. Now, what does SetUser(r, "dave") do?

Since you're passing in 'r', it must keep a map of all requests and their
data, i.e. map[*http.Request]string ("dave" being the string in this case).

This map should be protected by a mutex because it could be written from
multiple go-routines.

Is this the case? If so, how do we avoid contention on this mutex?

~~~
tptacek
No, 'r' is request-specific; it's owned by the goroutine that's dispatching
the current request.

If you had actual shared state that needed to be mutated by concurrent
handlers, the idiomatic solution would be to park it behind a channel on its
own goroutine; that's why goroutines are so cheap, is so you can allocate them
to problems like this. If you want your solution to be general and unfussy,
you'd have the channel be of closures; you'd just pass whatever mutating code
you want to run to the goroutine.

~~~
hundt
Actually
([https://github.com/gorilla/context/blob/master/context.go](https://github.com/gorilla/context/blob/master/context.go)):

    
    
      var (
              mutex sync.Mutex
              data  = make(map[*http.Request]map[interface{}]interface{})
              datat = make(map[*http.Request]int64)
      )
    
      // Set stores a value for a given key in a given request.
      func Set(r *http.Request, key, val interface{}) {
              mutex.Lock()
              defer mutex.Unlock()
              if data[r] == nil {
                      data[r] = make(map[interface{}]interface{})
                      datat[r] = time.Now().Unix()
              }
              data[r][key] = val
      }
    

The map is needed because 'r' is not actually (and cannot be) modified, so for
a later handler (for the same request) to access the data it must be stored in
this map, so 'Get' can get it back. And the map is global, so you need a
mutex.

As a side note, my understanding is that it is not necessarily un-idiomatic to
use a mutex in Go, particularly for shared state. See
[https://code.google.com/p/go-
wiki/wiki/MutexOrChannel](https://code.google.com/p/go-
wiki/wiki/MutexOrChannel)

~~~
elithrar
> As a side note, my understanding is that it is not necessarily un-idiomatic
> to use a mutex in Go, particularly for shared state. See
> [https://code.google.com/p/go-
> wiki/wiki/MutexOrChannel](https://code.google.com/p/go-
> wiki/wiki/MutexOrChannel)

It's definitely not unidiomatic (heck, one of the Go team members inspired
gorilla/context). It's not _ideal_ , but at the same time a context map a) is
simple b) does not impart significant complexity on middleware and c) _should_
still perform well, even with a fair bit of contention. Map access is pretty
fast, and most of the time you're only storing small things in it.

I'd be curious to see/benchmark the results of a more complex, less map-
reliant solution vs. a context map: my gut feel is that the map route wouldn't
have any problems hitting 10,000req/s on a small 2GB VM. Most of the time, the
context map won't be your bottleneck.

------
bfrog
The request context is I think the most annoying part of go http handling.

Oh you want to wrap your handler with something that does
db.Begin()/db.Commit()? Well you need to stuff that tx somewhere...

so then your options are a global map[*http.Request]interface{} with mutexes
or pass it as an arg. Passing it in as an arg means you either lose compile
time type checking because you use reflect or lose the ability to add more
middlewares that build up request context. The map[]interface{} also loses
compile time type checking for the most part unless you do something per type
of context, with more mutexes.

The second option is the sanest, but means either reflect if you have more
than one context arg to build out (sql.Tx + cache + oauth grant + whatever
else maybe?) or you have some common context object that may or may not use
everything every request and your just adding more context members all the
time that may or may not be initialized for that particular request.

Sometimes I miss the dynamic typing and/or compile time type templating.

~~~
codegangsta
I find the args approach the most appealing to me. Yeah you lose some compile
time safety. Like you said the other option is a global context with mutexes
which still somewhat misses some compile time safety. It basically boils down
to reflection over type assertions.

I think Martini handles this pretty darn well if I do say so myself ;)

~~~
mmgutz
I think you get the best of both worlds with `gocraft/web`. I feel args when
you inject 3 or more (db, session ...) is cumbersome. `gocraft/web` is faster
too since it has less reflection.

~~~
codegangsta
Agreed. I really like how gocraft does contexts. Very cool way to structure
your app

------
latch
I find the use of httptest.ResponseRecorder as well as the proposed context
pattern not very elegant.

Just because go's HTTP entry point is an http.HandlerFunc, doesn't mean your
middleware need to implement that same method. I'd prefer:

    
    
          type Middleware func(context *Context) Response
      
          type Context struct {
             Req *http.Request
             Token string
          }
      
          type Response interface {
             Status() int
             ....
          }
    
    

No global variable, and definitely no contention against that global.

(note, I'd make the middlewares nesteable, passing a next middleware to each
middleware, but that wasn't covered in the article).

~~~
sjustinas
Author here. Thanks for your input, but I'd like to clarify some points.

The blog post's intent was not to show how to implement your own middleware
mechanism in a web framework you're writing, but how to create HTTP middleware
using the standard Go net/http interface.

Your proposed middleware type might feel a bit nicer, but it disregards the
standard net/http interface and makes your middleware incompatible with it.
Well, you _could_ in theory make it compatible, but you would need some more
"glue" and hacks. net/http doesn't really work the way you're proposing
(returning a Response object, etc.). Please see my other post on the standard
Go HTTP tools: [http://justinas.org/embrace-gos-http-
tools/](http://justinas.org/embrace-gos-http-tools/)

If you don't want full compatibility with existing net/http libraries, then
sure – there are nicer ways. Again, my intention was to show how to do it with
net/http and I agree that there are certain trade-offs.

"(note, I'd make the middlewares nesteable, passing a next middleware to each
middleware, but that wasn't covered in the article)."

It _was_ covered to some extent. In a model where every middleware is
http.Handler itself, you nest your middlewares like:

    
    
      finalHandler := Middleware1(Middleware2(MyApp))
    

So every middleware does actually get a reference to the next handler that
should run.

~~~
latch
I guess we'll disagree about whether eliminating a global variable [in a
multi-threaded language] is nothing more than "might feel a bit nicer". Same
with using a class which specifically exists for testing.

net/http works fine with a response interface:

    
    
        context := newContext(req)
        response := chain.yield(context)
        // the exact same code you had:
         for k, v := range response.Header() {
            w.Header()[k] = v
        }
        ...
    

This has the added benefit of letting you use different response
implementations. A streaming implementation, an in-memory implementation, or,
probably the most important if you're doing high traffic, one that relies on a
pre-allocated byte-pool.

A little nitpick on the side, new(sync.RWMutex) isn't needed. You can just do
var cmMutex sync.RWMutex
([http://golang.org/src/pkg/net/http/server.go?s=45646:46651#L...](http://golang.org/src/pkg/net/http/server.go?s=45646:46651#L877))

EDIT: Sorry, not trying to be negative (so sick of this on here), but I do
think global variables are really, really bad and we probably shouldn't be
promoting patterns that rely on them.

~~~
sjustinas
Sure, I didn't say it was _impossible_ to convert your proposed Response type
to a proper net/http response when you need it.

But, like I said and you just demonstrated, glue code is needed to do that.
That makes people write differing middleware application mechanisms that are
incompatible with each other.

Sure, ResponseRecorder is placed in net/http/httptest and that makes sense.
But then again, nothing in the actual implementation is integrated with the
testing package or tied to testing in general. IMO there's nothing that makes
it unsuitable for other usage. It's a simple buffer to accumulate HTTP, that's
it. Even if one were to implement a buffering writer like that on their own,
it wouldn't differ too much.

------
tedchs
In the Ruby community, the Rack interface has done a lot to enable Web
middleware to work together. It would be great if there were an analogous
interface for Go's net/http.

~~~
politician
There is, it's the Handler interface.

[http://slid.es/andybons/high-performance-web-applications-
in...](http://slid.es/andybons/high-performance-web-applications-in-go)

