Hacker News new | comments | show | ask | jobs | submit login

Scheduling jobs by distributing docker container? Looks really heavyweight.

There is a good reason most of Map-Reduce frameworks are written on top of JVM - it is VERY easy to serialize Java/Scala code ship it to remote host and execute there.

This is not trivial in compiled languages ( I believe some hacks for Haskell exists )

What we need in Big Data space is more people that understand how computers really work. Some effort must be put into supporting TCP/IP Stacks in Userspace using new 10GBe cards or support for IB verbs. Also the long forgotten art of writing zero-copy code.




> Some effort must be put into supporting TCP/IP Stacks in Userspace using new 10GBe cards or support for IB verbs

Obviously my understanding of networks is not at that level yet. What would a layman do to become somewhat vested in this? Any books or resrouces?



Thank you. :)


I wouldn't consider StaticPointers a hack. I believe this is the feature you are talking about.




Applications are open for YC Winter 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: