
The Battle of the InfiniBands, Part Two - rbanffy
https://www.nextplatform.com/2018/04/13/the-battle-of-the-infinibands-part-two/
======
gnufx
Whatever they say, Omni-Path isn't (compatible with) Infiniband just because
it supports the Openfabrics interface, like some Ethernet technologies.
Qlogic's Infinipath was, so you could mix Mellanox HCAs and Qlogic switches,
for instance; I assume that also applies to Truscale, but I haven't used it.

DFT programs like VASP are likely dominated by time in mpi_alltoallv at scale,
and probably depend on the implementation of that more than the fabric,
assuming similar latencies. I don't know how/if that's helped by offloading; I
(and a colleague at another site) couldn't make a previous generation of the
support work properly and promises of help never materialized. Also, real
usage may not reflect IMB-type mpi_alltoallv benchmarks. (VASP is a
particularly unfortunate example, since the licence requires you to be a
member of a specific research group, as opposed to general support staff or,
as far as I could see when I read a copy, vendors.)

At least without the complication of offloading, according to
[http://simgrid.gforge.inria.fr/](http://simgrid.gforge.inria.fr/) a few P2P
metrics characterize the network well for simulating real MPI
programs/libraries.

