I just use iSCSI at home but using Mellanox’ RoCE which is pretty well performing.
One thing I’m noticing is that most of these storage protocols do, in fact, assume converged Ethernet; that is, zero packet loss and proper flow control.
I haven't experimented with it yet, but I expect that over TCP things degrade more gracefully. It seems earlier iterations of storage over networking didn't want to pay the overhead of TCP and lost out on the general purpose benefits that it brings. IIRC some RoCE iterations aren't routable for example. In theory, you could expose your NVMe over TCP device over the internet.
It seems to me applications taking advantage of NVMe are focused on building out deep queues of operations which may smooth out issues introduced by the network. But only way to know is to benchmark.
One thing I’m noticing is that most of these storage protocols do, in fact, assume converged Ethernet; that is, zero packet loss and proper flow control.
Is this also the case with NVMe over TCP?