Hacker News new | past | comments | ask | show | jobs | submit login
D3 7.0 (github.com/d3)
180 points by cltsang 46 days ago | hide | past | favorite | 61 comments

The one critique I have of D3 is that it is becoming increasingly closely paired with Observable, such that when I'm searching for examples of how to make something in d3, I can only find Observable examples that require the Observable runtime in order to function properly.

I think the examples should have a JS native version in addition to the Observable implementation.

That's part of the reason I started my book "Visualisation with D3.js"[1]... it's early access and I'm trying to find more time to work on it, but it's all JS native

[1] https://datacrayon.com/shop/product/visualisation-with-d3/

It's a real pain. I work with lots of "data science" types who are often trying to concurrently learn JavaScript and d3 - often with JS as their first or second programming language. It's difficult enough for an experienced programmer coming back to d3 after a few versions to pick out what's d3 vs. what's Observable - and very difficult to help a relative newbie through the process of turning what's shown in an Observable notebook into something that can run in an ordinary web page.

Even more difficult to deal with that and conforming it to Vue/React SPAs

Indeed - I don't like the pairing with Observable. I don't like Observable in general because the full experience can't be self-hosted. However, both projects belong to Mike, and D3.js is free - so I can't complain.

Didn't they have an export function that allowed self-hosting something you developped on Observable? (Or is it the development part that you want to self-host for "full experience"?)

You can self-host the runtime, and use code from notebook cells in whatever context you like. The closed part is the “notebook chrome” (editor, history view, social features, etc.).

What’s the difference between the two? As someone who only dips into D3 every few years and just discovered Observable, I wasn’t aware that they were different. I thought Observable was just a way of sharing D3 visualizations.

Observable is like Jupyter for Javascript. A general JS notebook environment, with the added twist that it uses reactive dataflow ideas to evaluate cells in a "right" and deterministic order (a common footgun in Jupyter where cells might be reevaluated manually, resulting in unexpected state).

D3 is a JS library for data visualisations.

Both are the brainchild of Mike Bostock. That's the connection. :)

Someone should make an open source version of the Observable ecosystem or integrate it into Jupyter tooling better. The notebook format is not encrypted or anything.

I posted this on HN a little while back [1], it's an open source alternative editor/runtime for Observable as a plugin for Starboard.

[1]: https://starboard.gg/nb/nfwK2VA


That's a bit like complaining that there are examples on codepen.

I'm super grateful that they provide such good documentation, which is second to none, in a nicely explorable environment.

The difference is that examples in Codepen are easily reproducable in another environment.

d3, on the other hand, has converted most of their examples to Observable, which requires extensive research to convert back to JS (in the best case) and in the worst case requires reverse engineering the runtime.

I'm not the only one with this opinion, there are plenty of others who have struggled through similar issues and discussed it.

"The process of converting an Observable notebook to standalone html and/or js is clearly not as simple and copying and pasting. The use of Observable notebooks for all D3 examples has made the introduction to the library much more challenging for this reason." [0]

[0] https://stackoverflow.com/questions/53155957/convert-d3-obse...

[1]: https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w...

I work with both D3 and Observable full time.

The claim that one has to "reverse engineer the runtime" is simply absurd.

The people that complain about D3 on observable always turn out to have taken neither the time to understand Observable nor D3 and are looking for a way to just copy paste something into their codebase, without understanding it, and without any willingness to really learn the tools they are using. Which would have never worked in the first place, because D3 is not a plotting library like vega or observables plot, it's a library library, building material from which you can build your own bespoke visualisations.

Observable is in essence reevaluating the entire code with each "requestAnimationFrame", no fancy diffing a la react or other magic, if it runs in observable, it will also run in your code as a static thing. Copy paste the cells out of observable, prepend them with a "const" and you're done, not more work than removing and rewriting the boilerplate that's in every codepen.

> taken neither the time to understand Observable nor D3 and are looking for a way to just copy paste something into their codebase, without understanding it, and without any willingness to really learn the tools they are using.

That's how many of us learn, by dipping our toes into a new SDK by copying and pasting into our bare HTML/JS page and tweaking things to see what happens. Eventually we will get around to reading the full documentation.

I have made d3 examples and played around with the toolkit since back when it was in its infancy (called protovis or something). It has definitely become more and more difficult over the years for a beginner to jump into.

(Background: I have met Mike since I got my PhD from the hci/graphics lab at Stanford, but we did not overlap.)

This is my experience as well - when deciding whether to use a certain library or framework, I don't start reading the reference manual. Rather, I highly prefer looking at and tweaking existing examples, mostly to understand what the typical usage looks like and most importantly where the limitations lie.

I feel like the last point is often overlooked by documentation authors: When evaluating whether to use a library, I'm not just interested in what it can do, but also what it can't do! That's why I really enjoy the "you should use X if you want to do Y"/"you shouldn't use X if you want to do Z" sections present in some manuals.


I am a beginner, and have spent hours in frustration, because I couldn't figure out the examples.

In every other project I would copy and paste some examples and start playing with it to understand.

You make my point. What better environment to learn and twiddle, than a reactive, constantly reevaluating interactive notebook?

I don't know why others down voted you so I upvoted this comment.

All I am saying is that the d3 team needs a "getting started" page aimed at allowing a dev to include one script tag (or a single npm install) and then copy and paste examples from the example gallery to get it quickly working on their own test website.

I recently tried getting back into d3 and I gave up (opted for another low threshold SVG toolkit). It's fine if d3 is aimed at other people who want to use observable.... It's just not targeted at me anymore.

I think that's actually a result of how D3 has changed over time. It has become much more modular and flexible, but also less "plug & play". In a sense it's a pro tool for custom data visulaisations, where everything is possible, but nothings easy.

I'd recommend Plot instead: https://observablehq.com/@observablehq/plot

It also works as a pure JS library, just like D3, but is based on a grammar of graphics idea, like ggplot. It's not infinitely flexible, but flexible enough, and build to get quickly to a chart, with as few lines as possible.

I guess it depends on your goal. If your goal is to learn how to code a nice reactive environment like a notebook is great. If your goal is to learn how to take a library and use it in a piece of software to release to the world, the notebook is next to useless.

If you think that the trivial differences between the two environments makes either of them useless to learn the other, franky I'll doubt you'll have the skill to learn D3 properly at all.

Taking a dom node and hooking it into the dom is not magic in D3. All that obervable does is hook in any dom node you return for you.

I wasn't sure whether I was going to comment, but this kind of comment is rather unhelpful and poor form. Some may prefer not to have a (mock) notebook runtime in their application for good reason that has nothing to do with their skill.

I see you are passionate about defending Observable, and that's cool - it's a great piece of tech, but please remember that everybody's requirements are different and valid too.

Then don't use the runtime and rewrite the code to plain js. All you have to do is declare cell contents as variables, and replace the first D3 select, that attaches to a reified dom node, with an #id selector. D3 is a series of tools, where the difficult parts are completely orthogonal to what observable provides.

It's like complaining that you can't learn how to drive in the evening, because there is a different sky color, than during the time you'll take your morning commute. The sky color is not the relevant nor difficult part.

I have my own (different) complaints about observable, but I'm sick and tired of bottom of the barrel programmers complaining about free stuff not being pre-digested enough for them.

Whatever environment each user prefers.

Being softlocked into Observable is not a pro.

That's like trying to force every vim user into Jetbrain IDEs (both are great, not trying to start a flamewar here).

Sure an IDE will have more features (like Observable vs simple HTML file), but it's important to understand different people have different ways of doing things.

We're talking about a javascript repl here, don't blow it out of proportion. The things transfer trivially between the envionments. If having a javascript repl is such a show stopper for learning D3 for you, maybe learn javascript first.

But if the example transfers trivially, why is there no convenient "download example project" button that offers an archive containing all the necessary files I need to run the example?

There is such a button.

Like I said, everyone I've seen complain so far, always turn out to not even want to learn basic stuff.

Ahh I must have missed this button. I will try to go look for it and report back when I have a d3 example working in my bare HTML page.


It’s literally just a matter of reordering variables, adding or removing a pair of parentheses, and removing the “return svg.node” call.

People love to complain about free open-source documentation. Especially for d3 it seems.

Shout out to Mike Bostock for all his awesome work.

Knowing what to re-order, remove/add is non-trivial for a newbie.

Relevant discussion: [0].

Let's look at the edit distance between [1] and [2]. Previous frustration at [3].

[0] https://news.ycombinator.com/item?id=27415725

[1] https://stackoverflow.com/questions/66966625/converting-obse...

[2] https://observablehq.com/@d3/contours

[3] https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w...

A newbie could simply use the export functionality in observable, which automatically generates iframe code, or vanilla-js code or react code for you.

Quoting other people that don't want to spend 5 minutes on their problem, doesn't make your not wanting to spend 5 minutes on your problem any more valid.

If you want a no code solution for data viz, use Excel, if finding the export button, or reordering variable declarations in an order, where things come before the things they are used by, is already too difficult, then you're not gonna understand D3 anyways.

Your post comes off really condescending and shitty. I don’t think it’s necessarily true, either.

I think a lot of newbies might just find the Observable stuff a bit overwhelming. Maybe their background isn’t JavaScript. That does not mean they aren’t gonna understand.

What are you even suggesting. They aren’t capable? We must all look like ants to you, up there.

As a dev with over a decade of experience in the front-end and no issues picking up d3 on my own, I still sympathize with overwhelmed newbies. Mike is trying to shove Observable down everyone’s throats in hopes of getting a good valuation and selling it some day, and I don’t blame him wanting to cash in D3 for this purpose. But let us recognize it plainly, and recognize the adverse consequences that has on a subset of the userbase.

And it’s fine if he doesn’t want those people on for the ride. It may also just mean a long-term atrophy of the D3 community.

> Mike is trying to shove Observable down everyone’s throats in hopes of getting a good valuation and selling it some day

That's an allegation that completely contradicts mikes and observables behaviour to date. They are insanely obsessed with making the barrier of entry of observable as low as possible, give most of the code away as open source, and are more concerned about empowering minorities than having a working business.

But what has it earned them? A bunch of ungrateful brats that couldn't code their way out of a hole if their lives depended on it.

Can you imagine people from any other profession being this whiney and entitled? "Heres a free plan for a bridge! BuT It'S iN MeTrIC SySteM :(!", "CiVIl EnGinEErINg is NoT MAtH!", "I doN'T WanT tO Do A DeStrUCtiVe TeST, CaN'T We JuSt ShIP iT???"

I come off as condescending? Good! I'm tired of catering to the absolute minimum. Observable is f** simple, if you don't have the ability or drive or whatever to learn it, and appreciate the help it provides in learning D3, you won't be able to learn D3. That's not me being mean, that's just realistic. D3 is super f*n hard, I have to look up stuff 24/7, because it's not build to be easy, it's build to be the most generic thing you can imagine to make visualisations.

Let's be real, the people don't complain about observable because they're overwhelmed newbies, but because they don't want to do their job. It's the people that copy paste stuff into their codebase without a glimmer of an attempt to understand it. Spend 10 minutes inside an interactive playground understanding the tool you're gonna use? MoNKeY WAnNt CopY PAsTe NoW!

The living personification of legacy code, the kind of coworker some poor person that actually takes the whole software _engineering_ thing seriously has to clean up after.

Ironically it's do-gooders like Mike and Melody that brought us into this mess in the first place, with their "everybody can be empowered to program" and "always say 'yes, and...'" bull**.

You act as if D3 documentation in observable was some kind of burden, when in reality observable is a bunch of training wheels.

Importing the library is not the difficult part, neither is selecting the DOM node to attach to, the difficult part is the selections, scales, axis, tics, interaction helpers, drawing helpers, simulations, layouts and whatnot.

But sure take the training wheels off! Shaving those 2 pounds of your bike will surely make it go a bit faster. But since you were unwilling to learn how to ride it in the first place, it's just gonna make you fall on your face more quickly.

Oof. Man. Your attitude is what makes software less diverse and hospitable.

I know there are some folks who maybe come with sincere laziness, but the hard push with observable complicates an already complex thing. You don’t have to agree.

Keep being mad or whatever and shitting on newcomers to the field. I’ll stick to welcoming folks and trying to hear the merit in their complaints.

Don't turn this into a diversity argument. We need more diversity in software from more diverse backgrounds. We just need less people in software in general. Especially less mediocre ones.

Guess who drives out female talent? Mediocre male white nerds who think they know it all and who don't have the open mindedness to learn tools like observable.

ObservableHQ is one of the most impactfull diversity pushing companies I've seen, but you accuse them of some ulterior motives for having the audacity of writing and hosting amazing D3 docs.

Haha, alright bud. You enjoy your weekend.

It is trivial. I just told you what to remove, and tracebacks will inform you of the ordering.

I would advise the ‘newbie’ you speak of to learn how to read a basic error message and how to use the backspace key before tackling d3!

(For reference, just this week an intern on my team needed some d3 boilerplate for a fairly complicated plot. I wrote it for her in observable, with some guidelines on where to go next. She finished what was needed and cr’d directly to our codebase - at no point did either of us worry about the supposed difficult translation between observable and a js file.)

Yeah I encountered this problem recently as someone who was once reasonably familiar with d3 and came back to it after several years away.

I find myself wanting to start with a working example I can riff off of more in d3 than with many other tools I've used, so I totally understand the appeal of interactive examples in Observable but find it super frustrating and cumbersome when it comes to actually rolling what I'm working on into an existing application.

> Adopt type: module. #3501

Standard JS modules, very nice! D3 now works in all playgrounds and libraries even as dependencies without non-standard tooling!


I do wish they didn't even publish UMD, that's a recipe for duplication if some libraries import the module and some import the UMD.

So, is d3 tree-shakeable yet? If I import { scaleLinear } from 'd3' in a webpack-built project, will I get only the scaleLinear code? As far as I remember, in v6 this will import a lot of (or perhaps the rest of) d3 as well.

You can also just import it from d3-scale.

Yeah, I know.

I am probably being unreasonable; but it's kind of nice to be able to have a single third-party dependency out of which you can pick what you need at the moment, rather than a host of small third-party dependencies. Kind of like lodash, from which you can at least import individual functions (import merge from 'lodash/merge') rather than installing all those functions as individual packages.

I agree with you and wish that were an option, but I do think most build tools will tree shake for you nowadays.

Exciting to see ESM everywhere. It works so well with Observable. I'm trying to put live playgrounds at observable for my packages as part of docs [0] - just starting, nothing exciting to see there yet, but I think the concept of playground as part of docs is quite awesome.

[0] https://observablehq.com/@mirek/tsql-examples

Great! Nice to see very few breaking changes, the Dreamcast intro animation still works https://datacrayon.com/posts/visualisation/visualisation-wit...

Thank you Mike!

This x1000. D3 is such a great tool. Just the scales/coloring/map project libraries would be impressive standalone. D3 is incredible.

Used this in a data viz class and I’d say it’s overkill for most visualizations. Also very difficult to learn and frustrating to implement with all the “enter, exit” stuff. Excel, Tableau, etc so much easier and most of what you need to do visualizations in the corporate setting. Obviously it’s excellent for very specialized unusual visualizations but it’s rare those are really needed.

All the "enter, exit" stuff is what makes d3 beautiful. From a core handful of ideas, Mike is able to express a vast world of visualizations. I think it is the most amazing framework I have had the pleasure of using in my 30+yrs of software development. Ya, you could grab bar chart library and just give it data and get a graph out, or, you could spend a little time with d3 and actually learn something so powerful you can make bar charts - or virtually any other kind of visualization you will ever need. Well worth working past the frustration, imo.

If you want the polish of d3 and some of its power without all the complexity, try Vega Lite and related tooling: https://vega.github.io/vega-lite/

Apples and oranges.

D3 is a low level visualisation library where you write code.

Excel/Tableau are business tools.

Why can't we have both? -- I had a similar frustration and got to work on https://hal9.ai, you get prebuilt blocks to easily import/transform/visualize data, while also being able modify the D3/Plot sources to your hearts contempt.

You actually don't have to use enter and exit and I agree it is cumbersome and unnecessary although I understand the desire for inband and out of band separation... https://medium.com/d3js-tutorials/d3-without-enter-91934575d...

> Excel, Tableau, etc so much easier and most of what you need to do visualizations in the corporate setting.

D3 is the best way to make visualizations online. Yes, it requires learning the library and building things from scratch, but it's light years better than the dog shit modules Tableau offers.

You're comparing Paint to Photoshop.

Feels like less than a year ago that we had D3 v6!

D3 6.0 - https://news.ycombinator.com/item?id=24288497 - Aug 2020 (86 comments)

D3 v6 was released on 27th Aug 2020, so your feeling is right. We seem to get a major release about every year now, compared to every two years before v5.

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact