
Show HN: uPlot.js – An exceptionally fast, tiny time series chart - leeoniya
https://github.com/leeoniya/uPlot
======
remon
After having a look at the code and the list of "non-features" (which is a
rather opinionated list I might add) the "exceptionally fast" part primarily
comes from it simply not doing as much as most charting libraries do. And
especially when it comes to things like "No DOM measuring" you're essentially
not doing something in the library that practically every implementation will
have to do outside of it. It's your project so you're perfectly free to make
the design choices you want but that puts the benchmarks resoundingly in
apples vs oranges territory. "Lightest car on the market, engine and bodywork
not included for weight optimisation reasons"

~~~
leeoniya
from a perf perspective it's more like excluding the mirrors and windshield
wipers. the amount of measuring [of label text] that might be required for
this bench is absolutely dwarfed by the time needed to render the data; it's a
rounding error.

> After having a look at the code and the list of "non-features"

it's quite obvious that you didn't understand the code you looked at, because
you would have noticed that the bench code does not use any of the "non-
features" (in any lib).

you should present stronger evidence that the benchmarks are BS.

~~~
remon
And you should calm down a bit. I was referring to your benchmark hitting
exactly the functional scope of your library while it's invoking more generic
libraries that by extension have to do more work for the same functional
result. Based on average feedback here that position is supported by other
real world users of these sorts of libraries.

As for label rendering/measuring; choose. Either it's a rounding error in perf
in which case don't claim it's a performance hog. Or claim it's a performance
hog and don't call it a rounding error to support an argument.

~~~
leeoniya
> I was referring to your benchmark hitting exactly the functional scope of
> your library while it's invoking more generic libraries that by extension
> have to do more work for the same functional result

ok that's fair but that overhead is immense, and you end up paying it
regardless of whether it's needed or not. obviously uPlot is opinionated and
is fast because of this. i'm not here to pry anyone's favorite charting lib
from them. all i'm showing is what's possible if you dont need to be generic
and cater to everyone.

i dont think i ever said label measuring/collision detection is a performance
hog. compared to the work of doing the plot itself, it's insignificant.
avoiding that work is more of a code size and complexity reduction for uPlot,
rather than a perf opt.

how else could i possibly present a benchmark of uPlot without limiting the
alternatives to its own functional scope? that would be like comparing 0-60 on
a formula 1 vs a mini-van and claiming the results to be invalid because the
minivan can fit 7 people but the formula 1 cannot. that fact is made
abundantly clear upfront.

~~~
remon
> i dont think i ever said label measuring/collision detection is a
> performance hog. compared to the work of doing the plot itself, it's
> insignificant. avoiding that work is more of a code size and complexity
> reduction for uPlot, rather than a perf opt.

Okay, fair enough

> how else could i possibly present a benchmark of my lib without limiting the
> alternatives to its own functional scope?

Can't think of a good way to do that. That's why I don't think your benchmark
is BS or in bad faith. All I was trying to say (but clearly didn't communicate
effectively) is that the value of the benchmark is limited as comparison data
for me and I assume others because it's not really comparing the same end-to-
end functionality if in the vast majority of cases the implementor would have
to manually add some things that it doesn't do out of the box (thereby burning
the same CPU cycles you're avoiding).

Anyway, let's close this thread. The above footnote aside it looks like a
super clean library. Well done.

------
gitgud
Under "Non-features that _won 't_ be added" it says

> No validation of inputs or helpful error messages - study the examples, read
> the docs.

I agree that validation is probably redundant, but why would you promote not
having helpful error messages? Seems a bit elitist...

~~~
leeoniya
certainly didn't mean it to come off that way.

i might make a devmode version that has some of this in the far future. but
it's not a priority before API stabilization, perf optimization, code golfing,
tests, docs, examples and v1.

if anyone wants to contribute this, i would certainly entertain a PR, but no
one is paying me to write extra code (or any code, really).

~~~
solidasparagus
I don't disagree, but good vs bad error messages can be the different between
a project that people love and contribute to and a project that people find
frustrating and bombard with issues.

Keras is great about error message[1] and it's part of what makes Keras
beloved (and heavily contributed to).

[1] [https://blog.keras.io/user-experience-design-for-
apis.html](https://blog.keras.io/user-experience-design-for-apis.html)

~~~
leeoniya
honestly, i already maintain too many open source projects (and quickly
resolve bug reports) to be worried about people not using more of my free
work.

i write open source code primarily for myself. doing anything i dont consider
critical takes the joy out of doing it.

if people want features, they can fork and contribute code and then maintain
that code, too...forever. of course, once you shift that burden to them,
suddenly no one wants to get involved.

sorry for putting it this bluntly but it's the truth. i am one human, with a
9-5 and a life outside of coding. people seem to feel entitled to opinions on
free open-source projects having contributed little-to-nothing.

~~~
journalctl
I’m pretty sure the parent was just making an honest suggestion that wasn’t
meant to attack or hurt you. And why post on HN if you aren’t willing to
listen to others’ suggestions/criticisms of your work?

~~~
leeoniya
look, i get it. i'm not offended. but i want to keep lib size minimal and
adding validation and error messages will bloat it. a lot.

i typically add them (or update the docs) once users encounter problems and
open Github issues (reactively). doing it proactively is a waste of time (for
me).

i accept the criticism. i just don't have the necessary free cycles to address
it.

------
leeoniya
i figured this was far enough along in the dev cycle now to get some feedback.

most charting libs try hard to do as much as possible; uPlot tries hard to do
as little as necessary. if anything, it demonstrates how fast raw Canvas can
be in the absence of extra baggage, like mem allocation and inefficient
algorithms.

i'm still fleshing out the programmatic API and also considering adding bar
chart/category functionality, since it captures the other type of useful chart
which cannot be represented as a trendline. eg:
[https://doc-0c-2g-docs.googleusercontent.com/docs/securesc/h...](https://doc-0c-2g-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/7mvbi2r6mu40k6gst3q1e4t2kvg5bne8/1570629600000/06662872918353485003/*/1oqsebLsZH8pj4exAVbDthaRhERXqg8mN)

~~~
onion2k
_if anything, it demonstrates how fast raw Canvas can be in the absence of
extra baggage, like mem allocation and inefficient algorithms_

Obviously using an efficient algorithm is important, but your point about
memory allocation is more interesting. The closest you can get to malloc in a
browser is a fixed length typed array, and if you're working with a canvas
that's probably Uint8ClampedArray(xSize * ySize * 4) for 24bit color. They're
_really_ fast. Why wouldn't you use that? It's definitely not "baggage".

~~~
leeoniya
i considered using typed arrays for the input data, but most data is gonna
come from JSON parsing, which will automatically allocate at least the size of
a non-typed array, so i used that as the lowest common denominator for the
demos.

my point was more that many charting libs use record-based datasets like
[[a,b],[c,d]] which would need to allocate possibly hundreds of thousands of
arrays or objects. uPlot uses something closer to a column store for its data
format, which saves a lot of mem, in addition to not duplicating the same
timestamps for every series.

------
dbranes
Can you state what's the mechanism that makes it fast? Is this a breakthrough
in rendering optimizations or does it introduce smarter data structures? Or is
it an accumulation of small optimizations everywhere?

~~~
leeoniya
i think a better question to ask is why the others are not faster. honestly i
cannot answer that question without digging into the source of each to find
their bottlenecks.

allocating a ton of small objects or arrays is a very common source of
slowness.

~~~
dbranes
No part of this comment answers my question.

~~~
leeoniya
no, there is no breakthrough that i'm aware of.

i took a raw canvas and a raw data structure of a single array per series,
plus one for timestamps and made a loop to draw lines on a canvas. it turned
out to be very fast. the mousemove interaction has rAF throttling applied and
does a binary search over the timestamps plus some basic arithmetic. there's
no "secret sauce" that makes it go fast. maybe the way it calculates scales is
more efficient than the others. i honestly don't know without looking into
what the others do, nor do i care enough to look into it.

------
seanlaff
Awesome!! It's been disheartening to see flotjs (which was essentially
unmaintained for 5 years) still beat out scores of other charting libraries in
terms of raw speed.

Over the last few years we've tried a lot of different techniques, eventually
settling on using Vega with our own interaction layer ontop- but I'd like to
give this a try. I'd love if this library handled the sizing of all the
elements as vega would, but I understand the desire to keep this small... I
browsed the entirety of the source code on my phone!

One of our use cases is streaming high frequency data- do you still see uPlot
performing better than the alternatives in that scenario?

~~~
leeoniya
> Awesome!! It's been disheartening to see flotjs (which was essentially
> unmaintained for 5 years) still beat out scores of other charting libraries
> in terms of raw speed.

i should add it to the benchmarks :) probably based on
[https://www.flotcharts.org/flot/examples/axes-
time/index.htm...](https://www.flotcharts.org/flot/examples/axes-
time/index.html)

> Over the last few years we've tried a lot of different techniques,
> eventually settling on using Vega with our own interaction layer ontop- but
> I'd like to give this a try. I'd love if this library handled the sizing of
> all the elements as vega would

i'll add Vega, too based on [https://vega.github.io/editor/#/examples/vega-
lite/line](https://vega.github.io/editor/#/examples/vega-lite/line)

as far as label/element sizing goes, right now uPlot's labels are just
absolutely positioned divs; vega's are canvas text. in addition, uPlot simply
blows away and re-creates all labels on each zoom/unzoom action - far from
ideal, but more than fast enough for manual ranging/zooming. i could
potentially move to canvas text as well as a starting point, but there are
many trade-offs to consider or whether it's necessary at all. i would like to
see what type of label sizing you guys take advantage of with Vega to
understand how extensive the implementation would have to be to offer
something similar.

> One of our use cases is streaming high frequency data- do you still see
> uPlot performing better than the alternatives in that scenario?

that's a loaded question. and the answer is that it really depends on the
frequency and the amount of data. if you can give me a function that simulates
the type of data feed & rate you're dealing with, along with how much data you
expect to be shown at any one time, then i can give you a better answer. right
now, if you open the benchmark in the repo and toggle the series on/off while
recording perf in devtools, you'll see that it takes ~12ms to redraw the 170k
point chart, which includes auto-scaling. this is enough to do 60fps, but does
not leave much frame budget for much else. in a real streaming case, i would
expect the actual numbers to be much lower than 60hz data updates and much
fewer than 170k concurrently displayed points. in addition, i would probably
turn off auto-scaling, since it would be very distracting to have the chart
rescale constantly.

if you'd be interested in providing a data stream simulation, then we can work
through an example. feel free to open an issue if you're interested in
fleshing this out.

~~~
shusson
> i'll add Vega

I don't think you should really add Vega as a comparison. It's more about
describing charts than rendering them (Vega-lite comes with a renderer baked
in). In fact it would be nice if uPlot could be a renderer for Vega.

~~~
leeoniya
contributions welcome! :D

...but probably wait for 1.0

------
tastroder
Looks great, nice work! Quick heads up, there seems to be a slight bug in the
zoom/selection functionality on desktop Chrome.

On the example page
[https://leeoniya.github.io/uPlot/bench/uPlot.html](https://leeoniya.github.io/uPlot/bench/uPlot.html)

\- Right click on the plot, click on any context menu item

\- The zoom selection area highlight appears, a second click highlights an
area but does not zoom. That highlight stays visible through zooming out by
double clicking and only goes away with a regular zoom in action.

~~~
leeoniya
yep, i need to add filtering for which button was clicked. thanks!

------
OJFord
Are there any (I speak as not-really-a-JS-person) frameworks on the opposite
end of the spectrum, for when performance isn't (yet) as important as fully
featured interactivity?

I'm imagining a sort of library of pre-made chart styles for different
purposes with different sorts of interactivity or sub charts already built-in,
such that using it is as simple as `fxSecurityWithCandlesticks(mydata)` or
`cmpRegionTrend(region1data, region2data)` etc. for when I just want a working
chart and I'm happy to mangle my data to fit the prescribed API, not to build
it up to fit my data.

Everything I've seen has varying levels of simplicity, and may make it quite
easy to add bundles of options for candlesticks or a second line or another
chart underneath allowing the range to be easily selected, or whatever, but
even if there's examples of different usages, it's still 'copy and paste'
rather than 'use this function'.

~~~
rejschaap
There is a list of alternatives in the performance benchmark you could check
out:

[https://github.com/leeoniya/uPlot#performance](https://github.com/leeoniya/uPlot#performance)

Personally I use highcharts a lot, it has many chart types out of the box and
allows for enough customization to fit my needs.

------
1wheel
Have you tried running simplifying the lines before rendering? Rendering 150k
points is cool, but there aren't enough pixels to see that level of detail.

[https://www.jasondavies.com/simplify/](https://www.jasondavies.com/simplify/)

~~~
leeoniya
i had a prototype using [https://github.com/mourner/simplify-
js](https://github.com/mourner/simplify-js) and it did a bit better in low
quality/low precision mode, but worse in high quality mode. at some point the
trade-off is probably worth it but i've noticed some not-so-great artifacts
even in hq mode, so had to ditch it.

another major reason for ditching it is that all the series must be x-value-
coalesced, so it's impossible to remove/merge datapoints along any single x
without incorrectly removing/merging them in an unrelated y.

since the path is drawn directly from the data without any intermediate
data->path conversions & allocations, i dont think there will be a reasonable
point at which general path simplification would provide a net positive.

i'm pretty sure dygraphs does some form of simplification which seems to
render well, but overall it ends up slower (not necessarily due to this, but
did not check). it was also written at a time when Canvas was not as fast as
it is today, so maybe it made more sense back then.

~~~
1wheel
Interesting! I wonder if anyone has worked on simplification for when we know
x is monotonically increasing.

One really simple idea: group points by their x pixel, connecting the max and
min y values for each with a vertical line.

~~~
leeoniya
uPlot requires a csv-like data structure as seen at the top of
[https://jsfiddle.net/v439aL1k/](https://jsfiddle.net/v439aL1k/)

feel free to experiment, but i suspect that any workable solution is not going
to be cheap enough and is likely to not be simple and high quality.

you must be able to simplify each data series as a stream (during the path
drawing loop itself) rather than allocating another set of 3 x 50,000-element
pixel offset arrays, which will eat up all the benefit very quickly.

~~~
1wheel
Twice as fast! You might be able to get interactive dragging working with
this.

[https://bl.ocks.org/1wheel/0e7f71ac0325b9be92c9dd19fe4bcc44](https://bl.ocks.org/1wheel/0e7f71ac0325b9be92c9dd19fe4bcc44)

~~~
leeoniya
interesting. it looks less accurate tho. also, i cannot use this without
supporting data gaps and sparse data. it could be worse when you get it to do
everything it needs to without just having this case be a dedicated additional
code path. once your neat uniform loops start branching, perf usually takes a
nose dive.

im on a phone, but will look into it later, thanks!

if you want to open an issue in the repo to work on porting this to the lib
for actual apples-to-apples comparison, that would be cool, too :)

~~~
1wheel
The column version looks like it has less detail along the top of the plot,
but I think that's a rendering bug with the moveTo version: there aren't any
data points above 1.

I added code for gaps. The tight loop isn't disrupted that much; the number of
interruptions is bounded by the x resolution.

added a placeholder issue
[https://github.com/leeoniya/uPlot/issues/15](https://github.com/leeoniya/uPlot/issues/15)

~~~
leeoniya
awesome, thanks!

------
3dfan
Why is this needed in the codebase?

    
    
        export const getFullYear = 'getFullYear';
        export const getMonth = 'getMonth';
        export const getDate = 'getDate';
        export const getDay = 'getDay';
        export const getHours = 'getHours';
        export const getMinutes = 'getMinutes';
        export const getSeconds = 'getSeconds';
        export const getMilliseconds = 'getMilliseconds';
    

To be seen here:

[https://github.com/leeoniya/uPlot/blob/master/src/fmtDate.js](https://github.com/leeoniya/uPlot/blob/master/src/fmtDate.js)

~~~
leeoniya
it reduces code size.

d.getHours();d.getHours(); cannot be compressed but
d[getHours]();d[getHours](); can compress to d[a]();d[a]().

do this enough times and you've saved 1kb.

you should see the kinds of compression hacks the Preact team does ;)

~~~
pintxo
Don't js compressors do this sort of optimization on their own?

~~~
leeoniya
if you enable a bunch of unsafe options they can do some of them, but they
tend to err on the safe side when it comes to native built-in methods like
these on Date objects.

also, some minifiers optimize for gzip size where the effect is not as
drastic, but the browser still has to parse the un-gzipped source afterwards.
it depends on the weighted costs built into each minifier.

------
sb8244
Would it be possible to have 2 charts with synced crosshairs on this? I have a
little open-source application that grabs stats from Elixir nodes and its
plotting library is pretty slow. Lining up multiple crosshairs allows
pinpointing a moment of time to an issue.

~~~
leeoniya
once the API is a bit more developed it will definitely have a way to hook
into cursor onmove events and a method to set the cursor to specific x/y or
data offsets. that's pretty much all you'll need for sync.

~~~
reacharavindh
This was also my immediate thought - how lovely would it be to have
synchronised charts built based on this to view system perf stats. Looking
forward to the day this library gains that ability! Cheers on this project!!

~~~
leeoniya
this is done.

[https://leeoniya.github.io/uPlot/demos/sync-
cursor.html](https://leeoniya.github.io/uPlot/demos/sync-cursor.html)

------
xiphias2
Zooming on my mobile (Chrome) works for dygraphs, but not for uPlot.js.

I tried the benchmark example and it doesn't zoom in for me.

The other ,,example and API'' demo doesn't work at all.

Anyways, thanks for making something fast, it's rare these days that people
care about speed!

~~~
leeoniya
> Zooming on my mobile (Chrome) works for dygraphs, but not for uPlot.js.

i haven't added touch events yet. right now zoom reset works on dblclick event
which i get for free with a mouse, but there's no free doubletap event, so
it's not terribly trivial to just add it quickly and with little code. but
i'll have to figure out what to do eventually :)

------
mosselman
Nice and fast!

There is a tiny bug, if you want to call it that: When you drag to select a
period and go over the edge of the chart itself and go back, you can't finish
selecting the period and have to start over by clicking a few times.

~~~
leeoniya
thanks!

i should bind the mouseup event at the document level instead of chart level
and clamp the min/max ranges.

------
calmconviction
I really like the packing implementation in data.json. Is this homegrown or
more of a 'standard' method for saving having to send all the json mechanics?

I'd like to use this elsewhere.

~~~
leeoniya
it's basically the same idea as
[https://github.com/WebReflection/JSONH](https://github.com/WebReflection/JSONH)
except i don't unpack to objects

------
edf13
> it can create an interactive chart containing 150,000 data points in 50ms

Very impressive in a world of over engineering and over designed charts!

------
catacombs
How does this compare to d3.js? I'm surprised to not see it included in the
benchmark comparison table.

~~~
leeoniya
i could not find a simple/minimal d3 timeseries example. these are huge and
very explicit:

[https://bl.ocks.org/d3netxer/10a28b7aee406f4e7fce](https://bl.ocks.org/d3netxer/10a28b7aee406f4e7fce)

[https://bl.ocks.org/robyngit/89327a78e22d138cff19c6de7288c1c...](https://bl.ocks.org/robyngit/89327a78e22d138cff19c6de7288c1cf)

if you know d3 well enough to faithfully match what the benchmarks do, then
please submit a PR. or at least point me to a minimal demo.

maybe i can do one using c3.js, which is d3-based:
[https://c3js.org/samples/timeseries.html](https://c3js.org/samples/timeseries.html)

this demo uses SVG, so it will very likely be poor. i don't know if there's an
option to use a d3 canvas-based backend?

it does not appear so:
[https://github.com/c3js/c3/pull/1436#issuecomment-302930456](https://github.com/c3js/c3/pull/1436#issuecomment-302930456)

------
edgarvaldes
This is great, awesome. I'm going to toy around with the lib. Also love the
focus of the project.

------
qwerty456127
Can it draw candlestick (OHLC) series?

~~~
leeoniya
i'll probably support that through allowing custom markers at the datapoints.
beyond that i'm not sure how much will be baked into the lib itself, but it
will be possible to draw your own candlesticks or rubber duckies at x/y
coordinates for any given datapoint by a callback that accepts the necessary
positions, data values and access to the canvas context.

------
superPorridge
You just made me register to say thanks, I'm a noob currently exploring time
series and it's usage. You helped me a lot

