> But can a 10-node-iphone-cluster really match an A100 for pure flops?!
Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats.
Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.
Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats.
Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.