Hacker News new | past | comments | ask | show | jobs | submit login

Maybe I was wrong on some of this, but my use case (due to my employer's industry being finance) is lower latency, where Linux absolutely and positively crushes anything else.

Actually, while we're on the subject, SmartOS with CPU bursting from illumos is the leader in low latency trading:


That is a slick platform they've built, but I still don't see how it is competitive with Linux for very low latencies. He mentions trading at microseconds, but we're building microwave radio networks to trade at nanoseconds. Unless this has changed extremely recently, Solaris/Illumos and hence SmartOS still don't have tickless kernels. I recall Solaris having a 100hz tick by default which you could change to 1000hz with a special boot flag. Linux has had dynticks since fairly early 2.6 kernels and with the modern 3.x kernels (RHEL7+), you've got full on tickless via the nohz_full options. Without this, the kernel interrupts the application to use cpu time.

Additionally, I don't believe (Experts please correct me if this is wrong) SmartOS has an equivalent to Linux's isolcpus boot command line flag (or cpu_exclusive=1 if you're in a cpuset) to remove a cpu core entirely from the global scheduler domain. This prevents any tasks from running on that CPU, including kernel threads. Kernel threads will still occasionally interrupt applications if you simply set the affinity on pid 1 so that does't count.

These two features, along with hardware that is configured to not throw SMIs, allow Linux to get out of the way of applications for truly low latency. As far as I'm aware, this is impossible to do in Solaris/SmartOS. I'm not even getting into the SLUB memory allocator being better or the lazy TLB in Linux massively lowering TLB shootdowns, etc, etc. There is a reason why virtually every single major financial exchange in the world runs Linux (CME in Chicago, NYSE/NYMEX in New York, LSE in London, and Xetra in Frankfurt), it is better for the low latency use case.

You asked for an expert to correct you if you're wrong, so here it is: this is just completely wrong and entirely ignorant of both the capacity of the system and its history.

On timers: we (I) added arbitrary resolution interval timers to the operating system in 1999[1] -- predating Linux by years. (We have had CPU binding and processor sets for even longer.) The operating system was and is being used in many real-time capacities (in both the financial and defense sectors in particular) -- and before "every single major financial exchange" was running Linux, many of them were running Solaris.

[1] https://github.com/joyent/illumos-joyent/blob/master/usr/src...

Thank you Bryan for the correction, I did after all ask for it :)

One final question while I've got you that your response didn't seemingly address. Does the cyclic subsystem allow turning off the cpu timer entirely ala Linux's nohz_full? If so, I stand corrected.

Yes, it does -- the cyclic subsystem will only fire on CPUs that have a cyclic scheduled, which won't be any CPU that is engaged in interrupt sheltering via psradm.[1] This is how it is able to achieve hard real-time latency (and indeed, was used for some hardware-in-the-loop flight simulator systems in the defense sector that had very tight latency tolerence).

[1] https://illumos.org/man/1m/psradm

You should really chat to some HFT folk in NYC before making that conclusion.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact