Hacker News new | comments | show | ask | jobs | submit login

EFS is terrible, the latency and throughput issues makes it useless for anything involving scientific computing/data processing. I could not disagree more, using it conjunction with GPU's makes zero sense. S3 and GCP/GS on the other hand are far better. Tensorflow "natively" support S3 and GS, i.e. if you path begins with "s3://" or "gs://" it will transparently pull from S3/GS, When used with conjunction with tf.dataset it allows you to have stateless training/processing. Tensorflow also support s3/gs for storing trained models.

Recommending EFS is a great way of signaling your ignorance about I/O.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: