

Choosing Go to implement the new GOV.UK router - benilov
https://gdstechnology.blog.gov.uk/2013/12/09/choosing-go-for-a-new-project/

======
csmuk
Not impressed.

I have nothing against Go (in fact I like it) but they seem to have built
GOV.UK with every damn bit of technology there is available. I've seen
Varnish, Go, Ruby, Python, Scala, Java, Mongo, MySQL, nginx, rails, sinatra,
Django and PILES of Not Invented Here.

Also I'm not sure it's a great use of public money to build stuff like this
when they should have nginx/apache up front and some configuration.

In fact this whole thing stinks of crappy information architecture resulting
in a massive front end router hack that they did in Scala, binned and moved to
Varnish, then binned and moved to Go.

This is not one "cohesive" web front end. It's a rat's nest and an
increasingly expensive one.

~~~
daliusd
It does not look that bad actually. Your list lacks Oracle, Microsoft
technologies to really complain about inefficient usage of public money. I
would love if my country used open source more.

~~~
csmuk
Agree entirely there!

However it doesn't excuse people working on a web site funded by the taxpayer
using it as a technology playground.

~~~
petepete
In their defence, gov.uk is miles better than just about any other government
site/application I've ever used.

~~~
csmuk
Oh definitely agree. I'm not dissing the site itself which is marvellous so
far, but the technology churn is very concerning.

~~~
polymatter
Allowing staff to use "language of the week" can be an excellent way of
retaining staff when you can't pay them market rates.

------
floops
A lot of the comments here seem to be missing something quite important: Scala
and Go are not similar languages.

Scala is a large and complex language, encompassing multiple programming
idioms. Teams programming Scala (as with C++ and many other "large" languages)
often have difficulties nailing down what portion of the language they are
going to use. "Java-without-semicolons" all the way through to idiomatic,
functional Scala.

Go is a relatively small, simple language. You could easily learn Go's basic
syntax and semantics in an afternoon or two. Simplicity and suitability for
programmers is an explicit design goal.

Switching away from Scala due to lack of experience while simultaneously
picking up Go is not as illogical as everyone seems to be implying.

~~~
JazCE
You're right, but i think the point you're missing (and i don't mean this in a
rude way) is that they had a system built in scala, somehow, but now no one on
the team knows scala.

They lack an architectural oversight, that they should really be using
one(ish) language rather than a big mix of languages so that this doesn't
occur again when no one on the team knows GO because the project leads taken a
new job.

~~~
girvo
Retention. Also, I'd suggest that your average dev will learn Go quick enough
to work on this router quicker than they'll learn Scala; I prefer Scala
myself, though! It's a balance between proper tech choice, correct management
choices moving forward, and dealing with management mistakes (like letting
someone build the router in Scala when the rest of the team don't know it)
from the past.

A crappy PHP dev will learn Go a hell of a lot quicker than Scala, IMO. A
safer choice, for their team and position, IMO.

------
thu
That seems weird to me:

    
    
        * No one having a deep knowledge of Scala is mentioned as a problem,
          but they will learn Go.
        * They rule out a language because of its syntax.

~~~
Zenst
I agree, but if you want to learn something today then GO would fit better for
the task at hand and easier to learn. That and the cost involved is really
good value given the training involved with learning Scala over GO.

That and I suspect there was some wishlists on what people wanted to do in the
mix, like COBOL, still used but people just do not want to learn that as it is
deemed so last century in comparsision to say java, though they are altogether
different fish. Bit like the difference between a proxy and a router, what
sounds better on a CV.

I suspect that from a maintenance aspect that the GO code will be much easier
to support and grow as and when needed as well as scaleability.

Still it is most encouraging to see such a young language like GO get so much
respect and use so quickly in a enviroment that has to many in the past and
present been slow to adapts new technology. But there again many Govermental
departments still have XP, albeit used as a fancy terminal to some old
mainframe application that just ticks along.

~~~
enneff
"Go" is not an acronym.

~~~
Zenst
I know, sorry got carried away with that caps key.

------
shawabawa3
On the github page it says the router sits between nginx and varnish and does:

    
    
      * Reverse proxy, forwarding requests to and serving responses from multiple backend servers on a single domain.
      * Redirector, serving HTTP 301 and 302 redirects to new URLs.
      * Gone responder, serving HTTP 410 responses for resources that used to but no longer exist.
    

Does anyone know why they have this as a separate piece of software instead of
getting nginx to do it?

~~~
rossj
You should read the first article in the series -
[https://gdstechnology.blog.gov.uk/2013/12/05/building-a-
new-...](https://gdstechnology.blog.gov.uk/2013/12/05/building-a-new-router-
for-gov-uk/)

~~~
csmuk
I actually read that and thought nginx/mod_proxy.

Looks like they didn't actually try it or know about it which is worrying.

Either that or it was more interesting to do it in Go which is not a valid
reason in a taxpayer funded site.

~~~
floops
Dynamic route updating with nginx is not fun.

~~~
csmuk
No but updating static routes via puppet/ansible and reloading is a piece of
piss.

If your site is volatile enough that the routing needs to change _that often_
then there is something wrong either with your information architecture or
your development process.

~~~
floops
Or maybe you're trying to deal with a huge legacy content migration of several
hundred different government agencies and want to deploy several times a day
without waiting for 10 minutes for nginx to load 1MB of config each time?

~~~
csmuk
2 front end nginx boxes s01-s02:

1\. take s01 out of pool.

2\. migrate config on s01 to new config.

3\. put s01 back in pool.

4\. take s02 out of pool

5\. migrate config on s02 to new config.

6\. put s02 back in pool.

Ansible can handle this quite happily. Scales up to any number of boxes. For
an (n-1)/n capacity reduction during deployment.

I've got a 450k apache config somewhere that takes <1 second to reload so I
don't think that's a major issue.

Also if you have THAT much config, something is wrong with your information
architecture (see my other points).

------
txutxu
Something is wrong with this statement:

    
    
        "Scala is great for performance, but quite bad at resource usage"
    

Maybe it's directly related to the next statement:

    
    
        "No-one in the core GOV.UK team had a deep knowledge of Scala, and particularly how the old router worked"
    

But the first statement, literally, does not make sense on its own.

~~~
mseepgood
> But the first statement, literally, does not make sense on its own.

Why? There are other resources than CPU time, e.g. memory.

~~~
txutxu
"Computer performance is characterized by the amount of useful work
accomplished by a computer system compared to the time and resources used."
\--wikipedia

Is there a different definition I'm missing? I think, a car can be "fast" and
take too much fuel, but (in my book) a programming language cannot be "great
for performance" while being bad at resource usage.

------
pjmlp
So they are ditching Scala, because no one in the team is an expert, while
replacing by a language that everyone has to learn?!?

~~~
rossj
I asked on twitter, and got a reasonable response. Investing in learning Scala
is a much longer term thing than learning Go (which is, in their experience, a
much quicker thing to do).

~~~
CmonDev
Because you always need to use all the features the language provides. Simpler
-> limited.

~~~
rossj
C is _much_ simpler than Scala, is it more limited?

~~~
CmonDev
In it's expressive ability - definitely. In terms of implementing stuff,
everything that is Turing-complete could be used.

------
justincormack
As they are Using Nginx already interested to know if they ruled out openresty
or did not consider it.

~~~
buro9
I'm a big fan of Go and we use it internally, but I too wonder why nginx + Lua
wouldn't be the chosen path.

If it's good enough for Cloudflare it probably is good enough here.

Besides, Varnish also would have excelled (yes I read the first post in the
series) if they had just stored the configuration in a saner format and then
output the required VCL when the config changed and reloaded on the fly.

Both of the technologies I would've picked (nginx and Varnish) were in their
stack.

I love working in Go, but reading this feels like a reinvention of the wheel.
Go would be my choice only if these other things didn't exist already and you
had to start from scratch.

The biggest pain identified was the writing, maintaining and loading of the
router config. Openresty could've solved that.

~~~
intranation
Openresty and Lua are great, but Go is a more generally applicable language
for building web services. So when taken in that context Go makes more sense
if we look at it as a learning exercise as well as a new part of the stack.

~~~
JazCE
Scala wasn't a learning exercise then?

------
CmonDev
"Erlang, Go, Scala, and Clojure are all good fits and represent the current
state of the art." \- why is the choice so limited? It seems to be quite JVM-
focused as well, I would consider F#/Nemerle as well. Erlang was invented in
1986 by the way - so a safe bet rather than "state of the art".

~~~
danpalmer
F# and Nemerle are both based on the CLR, and unfortunately the performance of
Mono is pretty bad, and not really suitable yet for high performance use-cases
like this.

Also I'd Erlang is state of the art, even though it is so old (Haskell too),
but I wouldn't call it an entirely safe bet because it requires quite a
different style of programming and experience, so might not work well for
certain developers.

~~~
CmonDev
Isn't Erlang just a (semi-)functional language with Actor Model baked in? I
might be wrong but it seems that you just need any dev familiar with message
queues and functional style.

