Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm sure someone can come along with a better description, but it's kinda like devops/sysadmin but specifically for data storage and access.

You could also look it up...



At some point, people realized that 90% of data science work was building pipes and keeping them clean. And then data engineering was invented so that the data scientists wouldn't have to get their hands as dirty. :)


A snappier version which gives the flavour: Data scientists use {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}. Data engineers write {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}.


I'd include database engineer in that bin. We used to call people DBAs but that seems a bit limiting now.


That's a good description for a data engineer from my end.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: