
Gor – A simple http traffic replication tool written in Go - mrsirduke
https://github.com/buger/gor/
======
StevePerkins
Eek... that's a rather unfortunate name choice. The "Gor" saga is an infamous
series of books spanning the past 50 years, basically the sci-fi/fantasy
equivalent of "Fifty Shades of Grey".

[https://en.wikipedia.org/wiki/Gor](https://en.wikipedia.org/wiki/Gor)

[https://www.google.com/search?q=gor&biw=1920&bih=947&source=...](https://www.google.com/search?q=gor&biw=1920&bih=947&source=lnms&tbm=isch&sa=X&ei=RjmDVa21GOHasASDzauwCg&ved=0CAYQ_AUoAQ)

~~~
jackmaney
"There are only two hard things in Computer Science: cache invalidation and
naming things."

\-- Phil Karlton
[http://martinfowler.com/bliki/TwoHardThings.html](http://martinfowler.com/bliki/TwoHardThings.html)

~~~
hpkuarg
Oh, and off-by-one errors.

~~~
jonathanoliver
I still count two problems. =)

------
tmd83
I have always wanted to use such a tool and this one looks good from a cursory
glance.

But what I have always wanted to know how people actually use these properly.
There are two problems that I see in replicating production traffic to
staging/dev

1\. Changes in application structure url/parameter. I think given the change
we make per release, we will get a lot of error. How to gracefully handle
that?

2\. Our application is write heavy so there is a lot of new content. So the
majority of the requests will access content s that don't exists in the
staging/dev environment. We can't even use live replication of DB either since
we usually have a lot of DB changes also.

One solution I can think of is record today's traffic. Take a DB snapshot at
the end of the day, replicate DB, run migration and then replay. Still has to
deal with app change. Am I missing something or this is really challenging. I
can see how this will perfectly work though for infrastructure change like
server software or configuration.

On a side note of the blue/green deployment. Theoretically the reversible DB
migration/parallel deployments of old-new version sounds good. But how many
people can make this possible? Does most app has little DB changes from
release to release. I will hazard a guess that if we try to implement such
things the cost of such complexity will easily overshadow the actual changes
to the app and most likely add severe risk of bug. But people seem to be doing
it, again not sure what I'm missing.

------
laumars
Blog post: [http://leonsbox.com/blog/2013/06/04/improving-testing-by-
usi...](http://leonsbox.com/blog/2013/06/04/improving-testing-by-using-real-
traffic-from-production/)

And previous discussion:
[https://news.ycombinator.com/item?id=5824387](https://news.ycombinator.com/item?id=5824387)

------
atom_enger
We use it to replicate production traffic to our staging and testing
environments at [https://reverb.com](https://reverb.com). It's an incredibly
useful and functional tool. We've used it to shake out exceptions from our
blue/green deploys as well. For instance when we migrated from Rails 3 -> 4,
Gor was paramount to shaking out some pretty nasty bugs by replicating prod
traffic from the Rails 3 cluster to the new Rails 4 cluster.

Gor is awesome!

------
fweespeech
This seems like a super useful tool, especially if you want to do blue/green
deploys where staging/production end up on the same physical machines and you
want to run with artificial load as part of capacity planning/testing.

------
Thaxll
Looks like Tsung replay feature: [http://tsung.erlang-
projects.org/](http://tsung.erlang-projects.org/)

~~~
mrsirduke
I can't find any documentation, but the name suggests, that Tsung would be
doing log replaying.

Gor does packet capturing and duplicates the requests to any number of
environments.

It's a different tool for a different task.

------
needcaffeine
This is super cool. You know what I'd really really like even more? Store this
info somewhere, and let me replay it at a later time or date. Maybe I want to
replay a whole day's worth of data at an accelerated rate to see if my servers
can handle it. It also opens up the ability to do data forensics to try and
reproduce a bug, perhaps.

Thanks for this!

~~~
newman314
It appears to have that support.

    
    
      # write to file
      gor --input-raw :80 --output-file requests.gor
    
      # read from file
      gor --input-file requests.gor --output-http "http://staging.com"

~~~
needcaffeine
Shit, this changes everything.

------
gouggoug
Seems like an awesome tool.

As a side note, however, I would recommend anybody to not blindly apply the
"Tuning" section (at the bottom of the readme) to their system. More
specifically, net.ipv4.tcp_tw_recycle and net.ipv4.tcp_tw_reuse are notorious
for causing problems if mis-used.

~~~
oimaz
Could you elaborate?

~~~
gouggoug
Of course,

net.ipv4.tcp_tw_recycle causes problems with NAT-ed clients.

tcp_tw_recycle (Boolean; default: disabled; since Linux 2.4) Enable fast
recycling of TIME_WAIT sockets. Enabling this option is not recommended since
this causes problems when working with NAT (Network Address Translation).

net.ipv4.tcp_tw_reuse seems fine to use, but literature about its real effects
is sparse.

This link [http://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-
li...](http://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux.html)
mentions both of them.

------
suchitpuri
I have been using VCR ,
[http://www.relishapp.com/vcr/vcr/docs](http://www.relishapp.com/vcr/vcr/docs)
for long time. Though VCR is only applicable to ruby, rails project.

Gor looks promising

------
mrsirduke
I took me a while to find gor, and it seems that there aren't many tools for
traffic duplication/replication out there.

If you know of any tools like gor, I'd very much like to know of them.

~~~
molf
There's httap:
[https://github.com/rolftimmermans/httap](https://github.com/rolftimmermans/httap)

------
LeonidBugaev
Thanks for putting it to the HN, author here :)

------
room271
I've used this and it's great. Very easy to use and works.

