Hacker News new | past | comments | ask | show | jobs | submit login
G9.js: Automatically Interactive Graphics (omrelli.ug)
631 points by nnx on Aug 9, 2023 | hide | past | favorite | 46 comments



The fact that you can pick a node which is rendered as a fifth or a sixth level of a recursive tree, move it around, and the rest of the tree would react, looks highly magical to me. It works without a function that explicitly infers the initial parameters of the tree from an arbitrary leaf node position.

I wonder how is it achieved, and how well does it work when positions of points are not connected by reasonably smooth functions.


If I had to guess: 1. Get partial derivatives of each parameter's influence on the moved object by sampling the function twice. 2. Perform a bog standard gradient descent. 3. go back to 1 until convergence is achieved.

Or you could be lazy and just throw an off-the-shelf optimizer at it.



They do quite a bit more sampling in order to dynamically determine a good delta value. But apart from that, I dare say that I got it broadly right.

It feels to me like there's also quite a bit of temporal coherence that could be leveraged in order to accelerate the process.


Requires the relationship from the screen coordinates to the variables in the program to be differentiable.

It is a very nice idea though.


It requires the final function to have a numerically reasonable finite difference gradient, which is somewhat different from what is commonly referred to as "differentiable" - eg. the insides of that function could still use non-differentiable/non-analytic functions.

It seems to be based on numeric.js, which is based on the classic Fortran UNCMIN [1] optimizer.

[1]: https://perception.lab.uiowa.edu/UNCMIN


I am guessing it's similar to "Inverse kinematics".

There are many methods of modelling and solving inverse kinematics problems. The most flexible of these methods typically rely on iterative optimization to seek out an approximate solution, due to the difficulty of inverting the forward kinematics equation and the possibility of an empty solution space. The core idea behind several of these methods is to model the forward kinematics equation using a Taylor series expansion, which can be simpler to invert and solve than the original system.


First thought was cool, clean simple API. I like it better than d3 to do the equivalent. -- Then I read this:

"When someone interacts with the graphics, for example by trying to drag an element to a new position, g9 optimizes over the space of possible values for your data to find a set of values that comes closest to creating the change. "

That is some highly magical DOM jimmying. High science weirdness. The applications I can think of where you'd need to manipulate a chart to modify data are all pretty niche (except for fun, which is universal). When the use case does come around this is going to be a delight.


Because it's not for charts. It's for interactive demos; have you tried playing with them on the rest of the page?


Yeah I did. They're fun.


As far applications go, there are a lot of STEM concepts that can be communicated effectively by allowing interactivity.


This is a good point.


I wouldn't call that high science. Average the velocity and vector of a drag event over the first few frames, keep averaging with more frames, and start moving everything to the predicted end-point with a nice ease-in-out, you're most of the way to "optimizing over the space of possible values".


I don’t think so. It looks like it’s using gradient descent optimization which does not require quotation marks around it and requires college level stem math for most people at least to really understand.


You're right, it turns out to be a gradient descent function adapted from numeric.js. Seems like a long way to go to interpolate mouse events into a range of values. Still suffers from gimbal lock in the cube demo. I guess since it doesn't allow you to manage the draw routine, you couldn't use quaternion to Euler conversion. I'm not sure I see the purpose of it. But what do I know, I don't have a stem degree.


There were some interesting comments when this was last discussed on HN (7 years ago): https://news.ycombinator.com/item?id=12572412


I would love an explanation on how it works because this feels like black magic to me.

If you know you have two points, one at (0, 10) and (10, 0) and drag the latter to (20, 0), how does the system tell that the first needs to be mirrored by the same distance?

Again, this is magic, so very well done to the author. Great stuff


Define a function that measures the error between the position of the drawn element and the new position required by the drag action. Compute partial derivatives of this function with respect to the data variables. Minimize this function to find the new data values. Re-render everything with the new optimal data values.

It's a bit confusing, because typically what's called 'data' here would be called "parameters" in mathematical optimization, or "weights" in statistics / machine learning. Also ML would call "data" what's fixed, so here the position of the dragged element.


Any sufficiently advanced mathematics is indistinguishable from magic, where "sufficently advanced" turns out to be a low bar.


Looks pretty cool!

One nitpick is, there are no markers/selection, so it's not very obvious that you can drag the objects, unless you stopped to read the description first.

I'm also bit puzzled on how readjusting the object's position works. Usually, you'd define a pointer handler and transform based on the input, but if I understand correctly, the engine tries its best guess to automate this for you?


I got excited about this a couple years ago and ported it to Pyodide in to run in the browser with a pytorch like library:

https://srush.github.io/g9py/

It's a little choppier than javascript, but crazy to me that it works at all.


This feels like witchcraft. I'm incredibly impressed.


It seems to have things in common with 2D CAD constraint based sketching tools?


Not sure what I would use this for, but it's crazy clever.


I guess it could potentially be used for any drag and drop interface? It's such a different way to think of the problem, I'm going to have to try it for something just to wrap my head around it.


Well, that was 8 hours ago - you get anywhere noodling over this?

I'm trying to go through the same thought process - this is neat, how do i translate into more practical applications. It seems like such a powerful paradigm if one can figure out that mapping.


No :).

The interesting thing is that drag and drop interfaces are not typically purely directly mapping the drag operation to moving the object selected. There's a lot more going on -- there's hover and activation effects, limits, often animation.

This approach gets rid of the annoying leg work of mapping pointer movement to changes, but I'm not sure how to fit the rest of drag drop UI into it (I do see examples of limits in the doc).

I think it's going to have to be something in the back of my head for awhile, and when I see interesting dragdrop experiences I'll think about how it could be achieved with this approach.


This could be extremely helpful for conveying trig or other math concepts in an education setting. I know a lot of visual learners that struggle with mapping the math to the visual space. Sprinkling in interactivity is a great way to help cement understanding.


How is this different than p5.js?


"When someone interacts with the graphics, for example by trying to drag an element to a new position, g9 optimizes over the space of possible values for your data to find a set of values that comes closest to creating the change."

This clearly isn't trying to be Processing. Obviously it's not revolutionary in terms of being productive, but it's a super cool concept and seems unique to me! The dragon example is particularly telling. You can grab any node and it tries to fit the fractal to wherever you move it.


I'm not familiar with p5. Can you pick an arbitrary rendered point of a p5 picture, and manipulate it?


Yes, it has full interactivity support: https://p5js.org/learn/interactivity.html

P5 has an imperative/immediate mode style of graphics API so you have to code the result of your interactivity (moving a point, etc.) yourself. Something like three.js, which has a full scene graph for rendering and interactivity, might be a better choice if you want a lot of interactivity out of the box.

I'm kinda scratching my head at g9 as it seems to be a mish mash of scene graph that's implicitly generated and imperative rendering APIs. It's not exactly clear to me why you'd want this mix except for very specific scenarios the author created support for with their implicit scene graph generation for interactivity.

In my mind you either want something that's like three.js with a full scene graph style of rendering and interactivity, or a p5 style imperative API for both. This seems to be some in-between thing to scratch some itches.


Have you tried dragging one of points?


Yes, I understand what is happening here with interactivity. Like I said I don't follow why I want some implicit generation of an interactivity scene graph (i.e. what points are connected or move relative to other points) when I can just be explicit about the scene graph interactivity with other similar libraries.


Because it's a fast way to get that interactivity, rather than having to explicitly code it yourself.

A good example are modern math videos on youtube, like this one (not saying that they're using G9.js in particular): https://www.youtube.com/watch?v=spUNpyF58BY

It's 20 minutes of changing graphics. You could code all of that with P5, and it would be an excruciatingly long process. G9.js would get you there in a fraction of the time.


Dumb question -- I know what d3 stands for. What are the 5 p's in p5 and the 9 g's in g9?


I was curious too, took a little bit of digging :)

"the original domain of [P]rocessing was proce55ing.net, so people used to sometimes refer to processing as proce55ing or P5 or p5 for short. they still do sometimes. p5.js is a reference to that."

from https://github.com/processing/p5.js/issues/2443


this is crazy. I wish they explained better how they solve the constraint to find the new point position.

Regardless this would be an awesome tool for Math Visualization. The book "THE ELEMENTS OF EUCLID" [1] had a website with java applet but this is now dead since browser have dropped.

[1] https://www.c82.net/euclid/#books


this is pure magic. Amazing stuff.


The animations don't seem to run on mobile


Work perfectly in WebKit on iOS.


They do for me (Android, Firefox).


Reminds me of the game Euclidea, where you manipulate geometric relationships (exactly like the circle example)


This is simply stunning!


very impressive, the last demo of a working crank-shaft-engine shows you can use it to make complicated models as well.

great start, best wishes with the project, looking forward to more amazing demos.


This is lovely




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: