One of the reasons I didn't go with D3 or any of the existing charting frameworks is that they simply do not seem built with that quantity of data in mind. Not only the rendering, the whole model: data is all packed in trees of JS-objects (at least the examples I see).
I have simple data that can be represented as dense typed arrays, with all the performance benefits of such, and no library seems to make good use of that. If I read the documentation for Stardust correctly, it's an exception to that rule (which makes sense, because typed arrays were mainly introduced for the sake of WebGL).
Similarly they also seem to do smart things with avoiding having to re-upload data to the GPU. I really hope this lives up to my expectations, it would save me a whole lot of work!