I played with D3 every now and then for maybe the past 6 years to scratch some data visualization itch, but I never really understood it. I always wound up trying a code snippet from a random blog and got frustrated when I couldn't get it to bend to my needs.
It finally clicked for me a after I read the first few introductory tutorials on the D3 Github wiki. It made me realize that, while it was probably built for data visualizations, D3 at its core is a browser DOM manipulation library with a (data-driven) declarative API. It's a slightly higher level of abstraction compared to the DOM manipulation part of jQuery (though, many frontend developers these days consider jQuery to be a low-level library).
Of course, it also helped that I learned a lot over 6 years. I was a university student back then, and now I'm a professional software engineer who has spent the last year or so doing frontend development. D3 makes a lot of sense within the context of my current understanding of how browser DOM APIs work.
that's pretty much the only way to do it though.. if you go any higher-level then it's just pre-baked out of the box visualization. As low level as d3 seems to be, there is probably no more intuitive data visualization that is bound to web display technology.
This seems so defeatist. Is each graph really that much of a special snowflake that we can't have frameworks that cover the 80% use case but allow you to drop to a lower level when needed?
I am disinclined by temperament to use low level tools for tasks that really should have a sensible generic abstraction. I know some people's brains err in the other direction - each to their own.
(I'm pretty sure tools exist. I used https://bokeh.pydata.org a few years ago when I needed something along these lines. I looked at d3 and had a similar reaction to the grandfather comment)
If what you want is a tool that basically does Excel-style static charts for data, then d3 is absolutely the wrong tool. Where d3 shines is when you need to handle smooth, dynamic updates.
In practice, handling dynamic updates requires so many hooks into the data that you can't really define a sensible abstraction layer and have it work well for both simple cases and complex cases. D3 is the successor to Protovis, itself the successor to Flare, itself the successor to Prefuse. And having used all of them save Flare (since I've never had the ability to write Flash code nor the desire to do so), d3 is the only one that I've actually enjoyed using.
I guess I'm one of those people who thinks that d3's api is an amazing level of abstraction. It took a bit for it to click for me, but that was mostly because it was my first exposure to functional programming.
If I ever have to make a chart with another language, I cry because I can't use d3. I have reimplemented portions of d3's api in other environments like native Android before. Digging through the docs to find whatever configuration variable I'm supposed to set reminds me of the days of jquery plugins.
I will say though that it's been a side-project idea of mine for several years now to build a JSX-like templating language that would generate d3 code. Never got around to it.
I'd be happy to. Can I ask first what you've tried and what you had trouble with? I think a lot of why d3 is considered low-level is that there are lot of external things you do need to know. For example, for me, it wasn't d3 itself but functional programming that I didn't really understand. For those who don't do web development much, it might be DOM (and in particular, SVG) manipulation.
The d3 specific parts are the idea of mapping data to DOM elements (which is the part I think is beautiful), how it uses the data join to do so, and how you use the scale functions to map from data space to pixel space to set the element positions and other attributes.
Feel free to email me if you like -- it's on my profile.
You can still use d3, but use a d3 charting library on top of it like nvd3 [1]. It gives you a prebuilt chart but you can modify it using d3 commands. It’s pretty useful and high level, but if you need the low level api it is there.
As far as using other abstractions, you're at the mercy of the developer(s) who made the abstraction, and I've found having to do lots of extra work to figure out how to inject desired customization that makes it generally worth it to just use d3 in the first place
Yes, I used D3 directly everyday for four or five years as did the whole graphics team of the Financial Times where I worked. Most of the print edition charts were created via D3 at some point in the process too. If you're making datavis everyday there's not much to compare (ggplot is good but much of the time the visual quality of the output is not quite there). "Higher level" libraries i.e. those that impose a taxonomic view of visualisation tend lack the necessary flexibility and expressiveness.
