
Hacking Ethernet out of Fibre Channel cards - zdw
https://blog.benjojo.co.uk/post/ip-over-fibre-channel-hack
======
noja
"I have roughly 2 tesco bags of Emulex FC 8Gbit cards and 1 tesco bag of
QLogic."

Brilliant.

~~~
nicktelford
If anyone is wondering about this standard unit of measure, the volume of 1
tesco carrier bag is equivalent to 17.9L (litres)[1]

1:
[https://www.reddit.com/r/AskUK/comments/3tt95u/how_many_litr...](https://www.reddit.com/r/AskUK/comments/3tt95u/how_many_litres_is_a_standard_uk_supermarket/cx9fwlv/)

~~~
jdsnape
Ah, well it depends a bit if it’s a standard carrier bag or a bag for life - I
think the bag for life is bigger.

~~~
philjohn
Which bag for life? One of the sturdy plastic ones, or the hessian variety?

~~~
OJFord
'Oh yeah, a _hessian_ bag for life maybe, but not a sturdy plastic one, that's
my point.'

------
eqvinox
Fun fact: you can connect more than 2 systems in FC without a switch. Just hop
along from one TX to the next RX and build a ring. (It's called FC-AL,
"Arbitrated Loop".) Works with fibre if you break the duplex cables apart to
make simplex connections.

Obviously the entire thing fails if any node is powered off or has any kind of
problem, but it's quite funny to do ;)

~~~
wyldfire
> Obviously the entire thing fails if any node is powered off or has any kind
> of problem, but it's quite funny to do

Indeed it was funny enough to have that be a very common network topology back
in the day. I've never had to investigate a token ring network failure but I
can pretty much guarantee it was a frustrating exercise.

~~~
phs2501
I was going to say that I thought token ring MAUs handled disconnected
stations but then I read this (including the "setup aid" with an included
battery) and now I'm not sure any more:

[https://en.wikipedia.org/wiki/Media_access_unit](https://en.wikipedia.org/wiki/Media_access_unit)

(un)fortunately I didn't experience this period of network history first hand.
:)

------
anilakar
A few years ago I and a few friends found a pile of old PCI Express Fibre
Channel cards from some manufacturer I can no longer remember. 2.5 gigabits
per second being faster than one and people being on an undergrad budget, we
soon started wondering whether they could be repurposed as network interface
cards in our ad hoc server colo built in an old abandoned storage room, which
didn't even exist in the original dorm floor plans.

A bit of googling turned out that it was possible and supported by the
manufacturer - if you ran Linux 2.6 and were prepared to compile the kernel
drivers yourself. We quickly ditched the project when we found out that the
old driver made newer kernels panic and setting up link aggregation on normal
gigabit ethernet cards wasn't that much of a hassle anyway.

~~~
non-entity
> We quickly ditched the project when we found out that the old driver made
> newer kernels panic

I'm suprised they ever compiled on a newer kernel.

------
duxup
I worked with FC for a while, and IP over FC was always a wonky kinda
situation. It really made no sense that if you had a big expensive FC fabric,
that you'd funnel your IP traffic through it.

It wasn't that much of a lift for most organizations to just have an IP
network for IP type traffic, and leave their high performance FC fabric alone
to do its thing.

The few situations when it was done was when someone had some rando
traditional IP systems that you'd connect just because, but these were corner
cases / kinda forgotten bit of equipment that you just were waiting to replace
/ time out kinda situations.

~~~
itsjustjoe
FCoE (fibre channel over ethernet) was similar. Not only was it an
amalgamation of arguably incompatible standards, but it was an unusual use
case. If you needed data security / reliability, etc you could go FC. If you
needed a cheap data protocol you could go iSCSI.

~~~
ArchOversight
FCoE was (and may still be) pushed heavily by Cisco. It was neat, being able
to have your Cisco UCS devices talk to both Fibre Channel NetApp and Ethernet
all on the same link.

It worked fairly well once it was up and running, and you set the QoS
correctly so that FC traffic won out over Ethernet because Fiber Channel does
NOT tolerate drops very well.

~~~
duxup
Cisco's FC solutions were always a weird effort to merge their own IP tech in
here or there and it just always felt half thought through to me.

~~~
ArchOversight
Bugs... so many bugs in the various pieces. But I can say it was fairly rock
solid once it was up and running.

~~~
duxup
Yeah Cisco's FC products were pretty buggy. I preferred Brocade.

FC is solid generally once you've got it running, the protocol kinda ensures
that.

------
nigwil_
IBM AIX (IBM's variant of the UNIX operating system) has long supported IPoFC
(IP over Fibre Channel).
[https://en.wikipedia.org/wiki/IBM_AIX](https://en.wikipedia.org/wiki/IBM_AIX)

------
erinnh
I have used FC 8Gbit transceivers before in my homelab as a replacement for
Multimode 10Gbit transceivers. They work quite well. No problems with those
Ive still got running.

~~~
mmastrac
Any recommendations on card brand? One of these days I'd love to give this a
shot.

~~~
erinnh
I used whichever were cheapest to buy on ebay. WHich currently were Mellanox
ConnectX-3 and HP branded Qlogic cards. Both work without a problem in my
homelab.

I do have one server with a built-in Intel card, that is more annoying, so I'd
steer away from those in the future. The Intel card did not take all
transceivers. They only wanted Cisco branded transceivers (from the ones I had
available. I'm gonna guess, Intel ones would be fine too).

------
ShradhaSingh
Here we have a network card which normally pretends to be a disk, but here
it's a network card pretending to be a scanner pretending to be a network
card.

Top-tier stuff.

------
quercusa
This is a great hack! Searching for "over fibre channel" and "fibre channel
over" will demonstrate that FC was seen as the answer to every need.

