

Will We Ever See InfiniBand in Desktop Computers? - kungfudoi
http://clusterdesign.org/2014/07/will-we-ever-see-infiniband-in-desktop-computers/

======
reportingsjr
Why switch to infiniband? Ethernet can go just as fast (not as good of
latency) and the cables are cheap and ubiquitous. The only thing stopping
higher speeds is cost. I have been watching 10GbE PCI-E cards for the last
year or two and they have dramatically dropped in price ($200 from newegg).
You can get server motherboards with 2 integrated 10GbE ports for $500 as
well. 10GbE does run over cat6 by the way.

~~~
mattzito
I think for general-purpose workloads, it's not relevant. However, some of the
IB-specific capabilities like RDMA are extremely interesting for clustering
use cases. You can dramatically reduce the overhead of moving data around
between nodes by bypassing the kernel entirely.

~~~
reportingsjr
I understand that sort of use case, but this article is asking about desktop
computers. It almost seems like the author only sees it used in high
performance situations so clearly it is better for all situations.

------
skynetv2
There is no practical or commercial motivation for IB on consumer devices.
Sure one may make a case that a home server or a media center can use IB to
stream photos and movies to other devices. At this point, we have to stop and
think like an average consumer and not an IT professional. And an average
consumer prefers no wires forget running copper or fiber. And what benefit
would it offer over wireless even if they were to run the cable? None from the
consumer perspective. You can stream HD over wireless without any problems.

Sure, we all prefer wired over wireless when available. But not in home, and
not by an average consumer.

Intel will integrate IB with their future processors, but they will not be
consumer based SKUs.

~~~
Spooky23
There are niches -- some people (interested parties for sure) even advocate
for fiber to the desktop as a cost savings:
[http://www.thefoa.org/tech/allfiber.htm](http://www.thefoa.org/tech/allfiber.htm)

But for regular consumers, wireless is the driver. Power users will care about
cat 6. The rest is just niche.

------
myrandomcomment
10GBaseT is cheap. Look at Arista networks. This idea is just not well
reasoned. Ethernet had all the "stuff" needed to make it work when you plug it
in. IB requires you to do a lot of configuration on the IB switch side, etc.

------
Nexxxeh
You can do InfiniBand on the desktop, but it necessarily doesn't give the
performance you might be expecting. I think I read about this on HN
originally...

[http://www.davidhunt.ie/infiniband-at-home-10gb-
networking-o...](http://www.davidhunt.ie/infiniband-at-home-10gb-networking-
on-the-cheap/)

Edit: From the edit on the follow-up post about getting it working on Ubuntu:

>Note: iperf maxed out at 1.2 gbps, and on the current linux install, I
couldnt get netperf client working at all. netserver would work, but only
showed a throughput of 25mbps from a Win7 client. HOWEVER, when I set up the
raid with 6 old 160G drives, the “hdparm -t /dev/md0p1″ showed 250MB/sec
reads, and I got the same from the Win7 machine using samba across the
infiniband fabric. This seems to indicate that iperf and netperf are
completely unreliable for testing this type of connection. Bear in mind though
that I did have netperf running on the previous ubuntu install, but that
installation was so messy I don’t know what drivers and user-space software
was running. I reckon it’s the kind of think that may be fixed in the stock
Ubuntu install in the near future. For the moment, just go with real-world
testing, i.e. copying large files from ramdisk to ramdisk, for example.

------
bhouston
In visual effects, there is need to transfer around massive data sets. WE have
found that 1 Gbps Ethernet just doens't cut it, but right now 10 Gbps Ethernet
solution are just too expensive for now, or require specialized cables (fiber
optics).

[http://en.wikipedia.org/wiki/10-gigabit_Ethernet](http://en.wikipedia.org/wiki/10-gigabit_Ethernet)

I would really like those prices to come down and also to use just regular CAT
cables.

Right now it seems that the switch to faster than 1 Gbps networking is just
too costly and not easily incremental (because of the switch to optical
generally.)

PS. There is 10GBASE-T but I think there is zero adoption or promotion of this
substandard:
[http://en.wikipedia.org/wiki/10-gigabit_Ethernet#10GBASE-T](http://en.wikipedia.org/wiki/10-gigabit_Ethernet#10GBASE-T)

~~~
caw
10GBASE-T isn't popular because your cable runs are cut short.

Costs have come down a lot on fiber. A short range SFP is like $120 or so, and
short range cables are like $3 per meter.

What a lot of people do as a stopgap is binding ports together, using
something like LACP. There's trade-offs of course. A 4 port 1000BASE-T card
could be used to boost a few important servers.

~~~
josh-wrale
10GBase-T also entails more power consumption per port and significant latency
over other 10GbE standards.

------
varelse
I don't mean to be cynical, but other than for niche clusters in the home,
wouldn't such technology be hobbled instantly by incoming bandwidth?

Even in Germany, where the author lives, the top bandwidth appears to be about
50 Mb/s:

[http://www.pcmag.com/article2/0,2817,2457284,00.asp](http://www.pcmag.com/article2/0,2817,2457284,00.asp)

Which although it is still much better than the states:

[http://ispspeedindex.netflix.com/results/usa/graph](http://ispspeedindex.netflix.com/results/usa/graph)

They're both well below 1 Gb/s, no?

~~~
zymhan
I don't think the use case envisioned here is for home computers with
InfiniBand, it's for work stations in corporate environments. Transferring
large data sets to local servers is quite common, and can take a (relatively)
long time even over Gigabit Ethernet.

~~~
varelse
Corporations can afford to buy pricy network hardware for the people who need
it (or they can skimp on engineering resources and congratulate themselves
over and over again over how thrifty they're being).

I read desktop to mostly mean consumer, perhaps erroneously?

------
sahyee
Interesting. I think it's possible that Infiniband will one day work its way
to desktop computers, but there are a lot of players in the game that have to
agree to make it work and make it popular.

------
walshemj
Would you want every workstation to have direct access to the cluster - ok I
get for loading data it might be useful but IN that case surely you would have
special purpose nodes for that

------
sp332
How would Infiniband be so much faster than Ethernet over the same cables? And
would a 5x faster Inifiniband be less than 5x as expensive?

~~~
zymhan
Going off of the [Wikipedia
page]([http://en.wikipedia.org/wiki/Infiniband](http://en.wikipedia.org/wiki/Infiniband)),
I think it's due to the "switched fabric topology" of InfiniBand.

~~~
sp332
Ethernet is mostly switched now (when was the last time you used a hub?), and
the topology doesn't change the line speed.

------
blutgens
No.

