I think this hasn't been update since 2017, and most of it is from 2016.
I pretty much only use javascript for d3, and I put it down for a while. Recently I decided to do something with d3 v5.0[1], and got confused about the state of ES6 modules. Has anyone gotten rollup to work with d3? Is it possible to just use ES6 modules natively in the browser if you don't care about older browsers?
My other issue was with async/await. It seems like when I used async await on a locally hosted csv file, it could take between 1 and 2 seconds to load. For my colleague sometimes minutes. When I switched to d3 v4.0 and the old callback approach, it loaded instantly, even for my colleague. What gives?
> Is it possible to just use ES6 modules natively in the browser if you don't care about older browsers?
Yes. Note that you need to use
<script type=module>
Personally though, I am using Google Closure compiler to turn the code of all my ES6 modules into a single, packed file without modules. Additionally I have told Closure compiler to output a sourcemap so that I can debug my code still. There are some issues with this from time to time. Not sure about what the debugging story is like currently with plain ES6 modules in the browser.
Most ES6 features are available in Chrome and Firefox. Not sure what the issue is with async and await, all it is, is syntactic sugar really. It shouldn't have a lag time any differently.
Babel (transpiler) is super easy to setup if you have any problems with compatibility.
It's not all sugar, there are some actual differences; and especially around async/await there are optimizations that can only be made in the non-transpiled code.
He said async/await is syntactic sugar over promises which it is. And that babel is easy to setup, which it is. Those statements are independently true. No one said the state machine that babel transpiles your async/awaits to is performant, he wasn't making that assertion.
The difference between 1s & 1m lag in D3 has got to be a D3 bug.
For me using D3 is kinda like using regex. Whenever I need to use it... it always take a long refresher to get back up-to-speed. But, it's one of those amazing, invaluable tools that does some pretty complex stuff without sweat.
In addition to the online tools that other commenters have pointed out, I strongly recommend finding a regex "cheat sheet" (there are dozens online) and actually printing it out - I have one pinned to the corkboard next to my monitor and it is invaluable. It will save a lot of time from having to remember all the character classes, which characters do/don't need an escape \, and crazy lookaround syntax ("was it ?<! or ?<=")
I love D3 but my biggest complaint is that many of the examples floating around appear to have been coded by people who don't care about reusability (and in some cases, ability for others to understand the code).
Nowadays if I need to do interactive visualisations I typically end up using Mobx + SVG.js. It works really well and I rarely get other developers asking me to explain how it works so that's a HUGE plus.
I played with D3 every now and then for maybe the past 6 years to scratch some data visualization itch, but I never really understood it. I always wound up trying a code snippet from a random blog and got frustrated when I couldn't get it to bend to my needs.
It finally clicked for me a after I read the first few introductory tutorials on the D3 Github wiki. It made me realize that, while it was probably built for data visualizations, D3 at its core is a browser DOM manipulation library with a (data-driven) declarative API. It's a slightly higher level of abstraction compared to the DOM manipulation part of jQuery (though, many frontend developers these days consider jQuery to be a low-level library).
Of course, it also helped that I learned a lot over 6 years. I was a university student back then, and now I'm a professional software engineer who has spent the last year or so doing frontend development. D3 makes a lot of sense within the context of my current understanding of how browser DOM APIs work.
that's pretty much the only way to do it though.. if you go any higher-level then it's just pre-baked out of the box visualization. As low level as d3 seems to be, there is probably no more intuitive data visualization that is bound to web display technology.
This seems so defeatist. Is each graph really that much of a special snowflake that we can't have frameworks that cover the 80% use case but allow you to drop to a lower level when needed?
I am disinclined by temperament to use low level tools for tasks that really should have a sensible generic abstraction. I know some people's brains err in the other direction - each to their own.
(I'm pretty sure tools exist. I used https://bokeh.pydata.org a few years ago when I needed something along these lines. I looked at d3 and had a similar reaction to the grandfather comment)
If what you want is a tool that basically does Excel-style static charts for data, then d3 is absolutely the wrong tool. Where d3 shines is when you need to handle smooth, dynamic updates.
In practice, handling dynamic updates requires so many hooks into the data that you can't really define a sensible abstraction layer and have it work well for both simple cases and complex cases. D3 is the successor to Protovis, itself the successor to Flare, itself the successor to Prefuse. And having used all of them save Flare (since I've never had the ability to write Flash code nor the desire to do so), d3 is the only one that I've actually enjoyed using.
I guess I'm one of those people who thinks that d3's api is an amazing level of abstraction. It took a bit for it to click for me, but that was mostly because it was my first exposure to functional programming.
If I ever have to make a chart with another language, I cry because I can't use d3. I have reimplemented portions of d3's api in other environments like native Android before. Digging through the docs to find whatever configuration variable I'm supposed to set reminds me of the days of jquery plugins.
I will say though that it's been a side-project idea of mine for several years now to build a JSX-like templating language that would generate d3 code. Never got around to it.
I'd be happy to. Can I ask first what you've tried and what you had trouble with? I think a lot of why d3 is considered low-level is that there are lot of external things you do need to know. For example, for me, it wasn't d3 itself but functional programming that I didn't really understand. For those who don't do web development much, it might be DOM (and in particular, SVG) manipulation.
The d3 specific parts are the idea of mapping data to DOM elements (which is the part I think is beautiful), how it uses the data join to do so, and how you use the scale functions to map from data space to pixel space to set the element positions and other attributes.
Feel free to email me if you like -- it's on my profile.
You can still use d3, but use a d3 charting library on top of it like nvd3 [1]. It gives you a prebuilt chart but you can modify it using d3 commands. It’s pretty useful and high level, but if you need the low level api it is there.
As far as using other abstractions, you're at the mercy of the developer(s) who made the abstraction, and I've found having to do lots of extra work to figure out how to inject desired customization that makes it generally worth it to just use d3 in the first place
Yes, I used D3 directly everyday for four or five years as did the whole graphics team of the Financial Times where I worked. Most of the print edition charts were created via D3 at some point in the process too. If you're making datavis everyday there's not much to compare (ggplot is good but much of the time the visual quality of the output is not quite there). "Higher level" libraries i.e. those that impose a taxonomic view of visualisation tend lack the necessary flexibility and expressiveness.
Nice timing. I just found this website the other day and love it. I've just recently been getting into data visualization and have fallen in love with d3.js. Another good resources is http://alignedleft.com/work/d3-book/. Maybe a bit introductory for some of the smartypants on HN but I've been away from front end dev for a while and so find blasting through the chapters pretty helpful. Hilarious tone, also. Combined with d3indepth.com's examples and the API docs I've been able to get a good grasp of d3.js.
As someone who mostly sticks to python but is interested in data analysis and visualization, I still haven’t figured out what problem d3 solves, and which use cases it makes sense for?
You can’t run Python (or R, or SAS or whatever) in a browser.
If you want to do dynamic visualisation with something like Python or R in a browser you need need a server, and that incurs significant latency.
D3 also allows much more fine grained control over visualisations than any visualisation library or package I’ve worked with in any language, because you have total control of every line, point, every fill.
No matter what crazy design the design team comes up with for your website, D3 will let you bind data to it and turn it into a workable chart.
It was written by an editor at the New York Times, and it is designed for producing bespoke visualisations on public facing showcase pages where appearances and speed have to be state of the art.
> If you want to do dynamic visualisation with something like Python or R in a browser you need need a server, and that incurs significant latency.
Not true. Bokeh (http://bokeh.pydata.org) lets you build interactive web graphics that run purely embedded in a static HTML file. No server needed.
(Of course, it also has a server-based mode that lets you build interactive analytical viz apps, with browser interactions automagically driving Python callbacks)
Or to try and explain that better... It's not a chart library, but a framework for wiring up dynamic data to collections of DOM nodes (either HTML or SVG) in any way you like. eg you can write specific chart libraries using it.
It is oriented around the lifecycles of instances of data - eg entry and exit hooks to run as data appears and disappears, and it has a lot of powerful lower level functionality.
It offers a bunch of functions that help make svg manipulation digestible. Try manipulating an svg with vanilla javascript, it's a pain to do anything more than basic shape drawing.
But it is static. The visual output is usually in jpg form. That's pretty much the definition of static output.
The only output I can find in Jupyter that is not static is coming from Bokeh which takes Python and generates javascript code to run in the browser. Jupyter itself calls this "a style of D3".
...The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.
Calling it "live" seems just slightly inaccurate to me. Every time code needs to run an http request needs to be made. (Unless there's some new tech here I'm unaware of)
This is a limitation that may make no difference in some cases, but in others it is important for code to execute in the client.
A good example of this is making a visualization that seamlessly responds to the user's mouse pointer.
I pretty much only use javascript for d3, and I put it down for a while. Recently I decided to do something with d3 v5.0[1], and got confused about the state of ES6 modules. Has anyone gotten rollup to work with d3? Is it possible to just use ES6 modules natively in the browser if you don't care about older browsers?
My other issue was with async/await. It seems like when I used async await on a locally hosted csv file, it could take between 1 and 2 seconds to load. For my colleague sometimes minutes. When I switched to d3 v4.0 and the old callback approach, it loaded instantly, even for my colleague. What gives?
[1] https://github.com/d3/d3/blob/master/CHANGES.md