Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I write a lot of code that runs in funny environments like the Compute Node Kernel running on Blue Gene systems, with offset-mapped memory (no TLB) and no over-subscription. It's fantastic for reproducible performance, but it's a specialized environment and people want to do things like run Python (as "glue" for some scientific applications) which needs dynamic loading. Since all IO is exported to separate IO nodes, and due to file system consistency semantics, dynamic loading is a tremendous bottleneck. This has escalated to the point where people burn a quarter million core hours to load Python (plus C extension modules) once on a large machine. To really solve the scalability problem, we needed to make dlopen() avoid touching the file system (by patching ld.so and implementing the POSIX file API with collective semantics served over the fast network).

The point here is that there are good reasons to want to use different systems together and then you end up putting in lot of effort working around the limitations of the specialized environments.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: