
Going With The Flow: Google’s Secret Switch To The Next Wave Of Networking - nsns
http://www.wired.com/wiredenterprise/2012/04/going-with-the-flow-google/
======
blrgeek
There is literally no equipment available in the market that does what these
Google's switches do. Cisco, Juniper, et al, protecting their technology and
investments into switching over the last 30 years, just didn't have the balls
to kill their old lines by doing this wholeheartedly.

Essentially, they have spent the last 20 yrs building their software which
runs on Motorola, MIPS, PowerPC, etc., running arcane switching protocols -
not always interoperably even. And these 'software-less' switches can be made
by almost anyone, since the software is their secret sauce.

Think of it as going from Minicomputers, which were custom boxes which had
custom hw/sw from a few vendors, to PCs, which have an 'open' design and are
designed for interoperability.

That's what OpenFlow does to the switching/networking ecosystem.

And since none of the incumbents want to commit hara-kiri, a few startups are
trying to do this, Nicira, BigSwitch, etc. Many others have OpenFlow
compatible switches, but nowhere near the scale that Google would need in
their datacenters.

Brilliant stuff. And I'd love to see Cisco die because of this - they've kept
the industry back for long enough.

~~~
CWuestefeld
I'm nothing like a network engineer. But is the centralization described in
this article a good thing for the Internet as a whole?

I can see that within an organization's internal network, they can assess the
importance of different communications, and route accordingly. So on the
internal side, it's potentially a big win.

But across the globe, who can assign the priority of traffic accurately and
impartially? And isn't the decentralized nature of the current architecture an
important feature, because of the way it can route around problems (be they
technical or regulatory) of its own accord, without requiring a higher
authority to tell it how (and thus without being susceptible to the agenda of
that authority)?

~~~
rumcajz
The point is that the switch is programmable. You can implement the
centralised behaviour. You can implement decentralised one. You can implement
anything you want.

~~~
CWuestefeld
Do there exist heuristics for decentralized traffic handling that this is able
to support, but cannot be implemented through traditional approaches?

~~~
joestringer
The idea is to build a globally consistant view of the network. Currently,
each node builds up its own view of the global state and routes based on this.
Sure, distributed protocols allow us to share this information, but it's not
guaranteed that the state will be accurate a few hops away.

OpenFlow allows an entity to keep a global consistant state, and calculate the
rules by which each of the nodes should forward. This logically centralised
control can then enable higher utilisation of the network. Think about traffic
reports on the radio -- If you are driving and know that there is a bottleneck
on one highway, then you can take an alternative route.

EDIT: I use "Global" in this context for within an AS, not necessarily
internet-wide.

~~~
NoKitty
Is that analogy necessarily parallel to networking? The radio station
communicates with an entire city. The traffic jam only affects traffic within
X miles of the bottle neck. I can imagine a car radio that automatically
stitches to a local radio station that only broadcasts traffic jams that are
relevant to cars in that area, eliminating the need for a larger centralized
station.

~~~
joestringer
Well, I think any analogy starts to fall apart when you look too closely. But
you've got the right idea. Sure, there's no reason you couldn't distribute it
out further. I used the word 'entity' above in an attempt to imply that it
could be "one large radio station" or "a group of smaller radio stations"--the
point is that the decision-making is abstracted out to somewhere else.

Note also the use of "logically centralized", not "physically centralized".

------
dons
Note that the openflow protocol has been implemented in Haskell by Galois and
Yale - <http://hackage.haskell.org/package/nettle-openflow> \- and you can
also configure openflow networks - <http://hackage.haskell.org/package/nettle-
netkit> / <http://hackage.haskell.org/package/nettle-frp>

------
ajb
It's interesting that this is now the insurgent, because in some sense, this
looks like networking done the _old_ way, the telco way: centralised
provisioning, aiming for 100% utilisation, etc. The kind of thing that cisco
et all were originally the small rebels against.

~~~
jacques_chester
I was thinking the exact same thing. Google controls their own network, so
they can implement a centrally-managed, circuit-based networking scheme.

Telephone networks tend to use these (on a scheme called SS7[1]) because in
most countries, the telephone networks were built by monopolies. It was
possible to develop the entire network as a single system and thus to obtain
very high efficiencies for certain use cases.

Google goes a step further. What they seem to have done is married circuit-
based networking with batch planning. The network itself is circuit based --
rather than each packet "finding" its own way, it can be routed end-to-end by
a central plan. But the decision of what to move when can also be planned.
Note the reference to "simulating a load". That's similar to what mainframe
batch planning achieves.

As usual, everything old is new again.

[1] <http://en.wikipedia.org/wiki/Signalling_System_No._7>

~~~
mjwalshe
Lol interesting I will have to share this with some of my old school Telco
mates :-)

So whats next Google reinvents x.400 and x.500 (not the special needs version
LDAP)

------
trout
The basic idea of OpenFlow is getting the manufacturers to split between
making good software and good hardware. If you make great hardware you can
continue to operate, someone else will control the forwarding decisions. If
you want to make good software, you can use someone else's hardware.

There will be a shakeup because telling the hardware manufacturers their
software isn't good enough isn't a great way to start that conversation, but
it's inevitable. They will insist that the software is like that for a reason,
and to do so is saying the last twenty years of development has been done the
wrong way.

I think it's highly interesting. I went to school for computer science but
found computer networking very interesting. There seems to be a certain level
of dismissal in the complexity of networking by people who write applications.
Writing a one line java socket that connects to another TCP port is trivial,
but the details are tedious. The same way we forget how difficult it is to get
phone calls to work because the end result is simple - phones ring.

OpenFlow will need to reinvent the wheel unless the existing hardware
manufacturers decide to give them a head start, which is unlikely. If it's
open source it will evolve quickly, however. There are many difficult
decisions and engineering problems to solve, which I suppose is a good sign.

------
jtchang
Google wants to make switching hardware a commodity product. Right now most
switching software does not interoperate, at least not on a level Google
wants.

Because of this if you want the latest and greatest features from Cisco you
have to run all Cisco. Or Juniper. You can't just buy 10 Cisco switches and 10
Juniper switches and all run the same operating system. Compare this to PC
hardware where I could buy any combination and install any OS I want.

~~~
pyre
The hardware is commodity, but you've centralized the software/logic. Is this
really a net win on a global scale? (It's obviously a net win at the micro
level, or else Google wouldn't use it)

~~~
joestringer
The protocol doesn't force this centralization to be physical, just logical --
Hyperflow[1] is one example of how a physically distributed control plane can
be created.

[1]
[http://static.usenix.org/event/inm10/tech/full_papers/Tootoo...](http://static.usenix.org/event/inm10/tech/full_papers/Tootoonchian.pdf)

------
mgw
OpenFlow is really the coolest thing to happen to internet infrastructure in a
long time. Just look at this video from Stanford where they route a wireless
video stream simultaneously over WiFi and WiMAX.
<http://www.youtube.com/watch?v=ov1DZYINg3Y> This technology is really
imperative if wireless networks want to cope with the increased traffic from
all mobile devices. How cool will it be when, having a video call, your iPhone
naturally switches from the cellular network to your home WiFi when you get
home. Or event better splits the traffic over both networks.

~~~
jodrellblank
_How cool will it be when, having a video call, your iPhone naturally switches
from the cellular network to your home WiFi when you get home._

It might be technically cool in the video, but my cellphone contract costs
more than my broadband one and the connection is more stable too. I don't want
to pay a phone company while they get to freeload on my ISP contract, and I
don't want my phonecalls dropping out when my broadband does.

~~~
toomuchtodo
If you're paying AT&T, Verizon, or Sprint per GB, and you're getting
250GB/month from Comcast on your home connection, yeah, you're going to want
your iPhone to intelligently switch to the "cheapest" connection available at
the time of data usage (if roaming between connections is possible).

------
redthrowaway
I found the licensing interesting:

 _License

Copyright (c) 2008 The Board of Trustees of The Leland Stanford Junior
University

We are making the OpenFlow specification and associated documentation
(Software) available for public use and benefit with the expectation that
others will use, modify and enhance the Software and contribute those
enhancements back to the community. However, since we would like to make the
Software available for broadest use, with as few restrictions as possible
permission is hereby granted, free of charge, to any person obtaining a copy
of this Software to deal in the Software under the copyrights without
restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

The name and trademarks of copyright holder(s) may NOT be used in advertising
or publicity pertaining to the Software or any derivatives without specific,
written prior permission._

It seems they're rolling their own license, as opposed to adopting any of the
other open licensing schemes out there. Any thoughts why they might do this?

~~~
bradleyjg
It is legally identical to the MIT license. The prefactory clause is purely
hortatory.

~~~
redthrowaway
It does appear to be largely identical. Is it not common practice to include
the name of the MIT license with the distribution, as is the case with the GPL
and Apache licenses?

~~~
jcurbo
No, not that I have seen. It's only called the MIT license because MIT used it
for their software (X, Athena, etc). The BSD license is a similar situation;
it doesn't say BSD in it anywhere, it's only called that because it was used
by BSD. Plus these days, neither of those licenses are overseen by who named
them - vs the Apache and GNU GPL which have version numbers and caretakers, if
you will.

------
MichaelGG
"In the user-facing network you also need every packet to arrive intact —
customers would be pretty unhappy if a key sentence in a document or e-mail
was dropped."

Why bother trying to illustrate with examples, if you're gonna write things
like that?

~~~
politician
It's arguably accurate; a UDP text-based protocol _could_ drop that key
sentence. Nonetheless, it's wired. What did you expect?

~~~
plink
Accurate for maybe a text-based talk client, but you wouldn't use UDP for docs
or email. I expected more from Steven Levy.

------
zby
We used to run routing on commodity computers - then came hardware routers
because you could do things faster when you specialized. Now we are back to
writing new software routers to run on commodity hardware because hardware is
now fast enough. Is that right? Looks like
[http://blog.gardeviance.org/2012/03/tens-graphs-on-
organisat...](http://blog.gardeviance.org/2012/03/tens-graphs-on-
organisational-warfare.html)

~~~
sophacles
Sort of but not really... openflow keeps the specialized network hardware, in
the form of switching fabrics etc, but allows for a common set of programming
instructions to be applied, so routing decisions can be made based on simple
rules in the hardware, or exported to a computer to decide routing (for the
packet or flow) without needing the computer itself to process all the
traffic. This is different than using a computer with N ethernet cards, where
all the data had to be marshalled around.

------
joestringer
Nick McKeown's OpenNetworking Summit 2011 presentation entitled "How SDN will
Shape Networking"[1] explains very well the abstraction of ideas that
Software-Defined Networking provides. OpenFlow is a protocol which implements
the idea of SDN.

[1] <http://www.youtube.com/watch?v=c9-K5O_qYgA>

------
noibl
The article uses the analogy of road traffic congestion being defeated by
autonomous vehicles with swarm-like intelligence enabled by centralised
computing. Which is interesting.

------
tectonic
This also implies that Google has private fiber between their data centers,
right?

~~~
jrockway
I think this is fairly common. We had this at Bank of America too; our own
non-Internet IP-based network. It even did multicast, almost.

~~~
jcurbo
Yeah, but did BoA buy actual fiber in the ground? You can have your own non-
Internet IP network easily if you lease dedicated physical circuits from
telcos. (This is how the military does it)

------
ForrestN
I like that Google is working on both sides of the car traffic analogy.

------
electrichead
They can probably extend this system (judging purely by the analogy since I am
not a network guy) to their self driving cars one day. One would think that a
similar mechanism would be needed to manage traffic.

~~~
joestringer
Perhaps. I would propose that the application of this paradigm to traffic
flows would have each intersection making the decisions for the drivers. A car
says "I'm heading to east 42nd", and the intersection says "Sure, go left."

Furthermore, traffic controller(s) would constantly gather information about
traffic conditions from each intersection, and tell them how to direct the
traffic to optimise road usage.

This way, the traffic controller knows everything about the optimal way
traffic should flow, and it only has to make decisions based on the number of
intersections rather than the number of cars. If you consider that one
intersection could have tens of thousands of cars per day, that provides a
huge saving in computation.

This differs from the analogy in the article; Intersection = Router, Car =
Packet.

------
eiji
Probably a stupid question: Does OpenFlow possibly change my home network
setup in the future too? Or for a small bussiness? Coffee shops etc.

~~~
wmf
There's hasn't been much research in this direction, but there are probably
potential uses for OpenFlow in home networks. For example, apps or devices
could use OpenFlow to give your router advice about how to handle their
traffic, e.g. firewall or QoS rules. There's already an OpenFlow
implementation for WRT54G routers, so it wouldn't be too hard to experiment
with it.

------
luminaobscura
so openflow is an alternative to ipv4/v6 ?

~~~
mgw
No. The idea behind OpenFlow is having simple routers which only implement
basic switching capabilities in hardware. The complex routing rules can then
be set in software which allows for much easier and more capable traffic
routing. It also allows more innovation in this part of the infrastructure
which has traditionally been very resistant to it because of the incumbent
players. You can, for example, experiment with new routing protocols without
having to do a firmware or even hardware upgrade.

~~~
tuxguy
@mgw, thanks for your insightful comments.

Traditionally the argument has been specialized hardware doing function X,
will be faster (but not customizable, upgradable) compared to doing function X
in software on commodity hardware (slow but upgradable).

And fpga were pointed out as somewhere in the middle ( some hw customization
using sw)

What are your thoughts on the argument that by using commodity hw &
implementing routing algos, etc in sw, will be more flexible but _slower_.

Is there a significant performance/speed cost when you implement core
networking features in sw ?

~~~
mgw
Specialized hardware will be faster than software for a long time. OpenFlow
ist smart enough to go around this problem. The routing still takes place in
specialized hardware and is therefore just as fast as in a standard device.

The routing layer in hardware acts on a set of cached rules which are very
simple. Simplified, you can imagine them to be of the form Matching Rule ->
Routing Action. The matching rule selects by the packet fields such as source
port or destination IP. The routing actions could be "forward to port" or
"drop packet". All of this is just as fast as in every other commodity router.

What is special in OpenFlow is another possibility for the "routing action"
field (or for unmatched packets): You can send certain packets up into the
software level, to the OpenFlow controller. This can be a centralized server
and the logic is implemented in software. The software decides about the
routing of these kinds of packets and sends the answer back to the router.
Here the rule is cached again and from now on the routing for this is as fast
as for all the other packets.

This last bit is the only part which is slower compared to commodity routers.
A really great solution in my opinion.

------
jbert
What's the benefit of keeping the data plane on the cisco/junpier hardware?
#ports per box?

In my naivety, I'd expect the main benefits of openflow to be on the WAN
links, so you could get away with 6 or so ports in a PC-like chassis running
software routing, with dumb local switches?

What am I missing?

~~~
blrgeek
#ports/box or more accurately, #ports/rack unit & gigabits of throughput/rack
unit.

The switching hardware is ASIC based and super optimized with CAM based
switching on VLAN tags, destination addresses, etc. 1U switch could have 36
ports @ 1Gbps, and needs to switch 36Gbps within the switch, and perhaps
10Gbps upstream.

So line rate switching of 36Gbps+ in 1U, possible today only with ASICs,
typically from Broadcom.

~~~
jbert
So openflow is useful within a cabinet?

I was thinking you could have have "dumb box, many ports" and "smart box, few
ports". Each cab needs one of the former, but you could get away with not many
of the latter?

~~~
blrgeek
OpenFlow is useful _across_ cabinets.

Cisco/Juniper boxes were smart, many ports, but smart only within each box.
That is you had to configure each one individually. As in you could do static
provisioning with a single tool across multiple boxes, but on the order of
minutes between changes. If you had to create a VLAN with ten boxes, across 5
different racks, you would be spending quite a bit of time doing that, since
you would need to find spare VLANs that are unused across the fabric,
configure all the switches in between, etc.

Now with OpenFlow, all the switches are controlled by the central controller.
And flow configuration is dynamic - ie you don't need to fiddle with
individual configs, when the flows start, the controller is queried, and if
there's an appropriate flow setup on the controller, it will be implemented,
with local free VLANs, etc.

Basically the entire fabric becomes as dynamic & smart as the controller,
instead of each switch being smart and static.

------
keeptrying
Finally a real world use case for active networks.

I built a proof of concept for soemthing like this for my master thesis -
2000.

I wish it had occurred to me that unlike the user being in control (in active
networks), the network operator could be in control.

Anyhoo ...

------
jpdoctor
Anyone seen any router speed metrics for this? They are absent in the article,
and I can't tell if that's just Wired or whether OpenFlow is still not fast
enough.

------
ilaksh
Sounds like most of the OpenFlow concept is a no-brainer improvement, but when
they talk about using it for centralized control, that is worrisome.

~~~
packetslave
Think "centralized control over all the routers and other devices that make up
a company's global network" (as opposed to hundreds/thousands of routers all
making their own decisions)

This is not about Dr. Evil-style centralized control of the whole Internet.

