
Training Bert, ResNet, Mask R-CNN, and Transformer with 16 GPUs and NVSwitch - remyg
https://lambdalabs.com/blog/announcing-hyperplane-16/
======
p1esk
Cost: ~$250k. For this price you can build 12 quad GPU or 6 eight GPU machines
(RTX 6000). Would not scale quite as well for multi-node jobs, but 48
distributed GPUs are still a lot faster than 16 local GPUs for most scenarios.
Not to mention flexibility and convenience 48 GPUs provide for larger research
teams.

