Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Hook – a simple graphical C/C++ runtime editor (github.com/abolinsky)
73 points by fercircularbuf 9 months ago | hide | past | favorite | 35 comments
While helping one of my students create and iterate on a game in C using raylib, I realized there was quite a lot of time spent on experimenting with level design and colors, how the player moves, gravity, etc. and a lot of time was wasted making a change, exiting the game, recompiling, running the game, deciding his change wasn't right, and going through the whole cycle over and over. I could tell it was demotivating, and this process was creating a barrier that prevented him from experimenting to his heart's content.

I started this project to solve this problem by giving a simple gui to a few core features of lldb through the lldb api. Experimenting with changing colors, where blocks in the level go, how tall the player is, gravity is all now just a simple toggle or text field edit. The variables are modifiable while the program is running.

The project is still in its early stages, and so far only targets macOS.




Hey, looks like we had the same idea! I've run into the same issue and wanted to stop building debug UI's that do the same thing.

In my case, I went to implement this for Windows, but bit more advanced with inline scripting support. Basically, you insert a snippet anywhere in the code that can call various functions and one of them is `view()` which will show all locals (or a chosen object out of scope) in a Visual Studio window (it's a VS extension [0], website [1]). Calling it again will update the view with the new state. The scripting language underneath is Angelscript and it understands symbols inside your entire project (e.g you could do `view(Namespace::g_variable)` or even call functions from the program etc. The snippet hook is then JIT'ed into the target function by relocating it using info from the PDB. This took me ~1y to implement, and it's still a research phase project.

[0] https://marketplace.visualstudio.com/items?itemName=donadigo...

[1] https://d-0.dev/


This is amazing work! Thank you for sharing! This is what I was envisioning when I embarked on this project.


I did a similar thing once in Haxe by using macros to annotate members and rendering control widgets in an auto generated debug UI. This is basically what Unity/Unreal/Godot/etc do with their fancy editors.

The best experience I've had with this kind of thing has been Godot. Not only do you get to tweak values at runtime, you can also change the code at runtime and see changes reflected live. It's so awesome being able to hit a breakpoint in a running game, step through some code, change the code, and then continue stepping to see the changes.


I think it might be more useful for your students to learn about making simple GUIs with Dear Imgui or something similar, that make this type of stuff intentional and much more stable/foolproof. A large part of game dev is making tools appropriate for making the game and all of this falls into that.


I didn't see your comment before I commented but yes I agree with you completely on this one.


I'm trying to wrap my head around why build a hook into lldb instead of writing a library that would allow your students to simply spawn an imgui window with the variables exposed in exactly the same way?

This is generally a tool you would likely need to build into a game regardless for exactly the reason you mentioned, rebuilding to change gameplay variables isn't exactly a great experience.

I personally just use a simple terminal that I can open and close to adjust variables but eith hoe easy imgui is I don't see any reason to not do a gui window for that.


Interesting! My baseline expectation would be that this wouldn't work at all, debugging C++ usually tells me variables have been removed by the optimiser. Perhaps this requires `-O0 -g` or similar to work effectively, which games devs are usually not open to.

Optimising IR without dropping debug info is difficult. I don't believe it to be intractable, some of the Sony developers are active in improving this for LLVM. Debug info is roughly a bunch of side tables of data from which one can recompute values which were in the original program and are not in the executing one. I would speculate that a compilation mode which treated any degradation of that as a miscompile could be done with minor to no runtime cost, at serious engineering cost.

Perhaps easier would be to restructure the program so that specific variables are always available to the debugger. Deliberately hoist them to escaped global variables, roughly.

The other interesting line of attack here is a JIT. The students probably don't change all the parameters simultaneously. Call graph walk + incremental compilation of parts affected by a given variable is probably tractable. A sibling mentioned hot code replacement, that's a similar sort of hack (you block function inlining, pad functions with some nops, then overwrite in place if the new definition fits in the space).

Thanks for the link, interesting stuff.


> Perhaps this requires `-O0 -g` or similar to work effectively, which games devs are usually not open to.

There are also plenty of game devs expressing the idea of "never use STL, because that makes performance unusable in debug builds". Which implies that many of them are not completely against debug builds (for debug/development purpose).

The problem of variables being optimized away is mostly for local variables. This tool will most likely used on global/long lifetime objects representing overall game state and configuration, instead of temporary variables while stepping through a function. (used)Global variables and class properties are unlikely to be optimized away or mangled in away that makes them hard to inspect.

This was created to help students while learning. In those kinds of projects performance is somewhat to be less of an issue, unless you make a mistake which would ruin performance regardless of optimization level.


> Perhaps this requires `-O0 -g` or similar to work effectively, which games devs are usually not open to.

Well, not for the shipping/release/prod builds, sure. But for dev? I dunno, from my vantage would rather keep it, if not for debug/inspect then just for compile speeds and ensuring a certain baseline perf even-without-optimizations... (but then, I'm not triple-A or have any games to my record =)


One of the neatest hacks I've seen and think about occasionally but have never used is a library that defines a TWEAK macro to do this at the source level:

<https://blog.voxagon.se/2018/03/13/hot-reloading-hardcoded-p...>

In other words, rather than having a graphical interface for tweaking values at runtime, you interface with it the same way you normally would—by changing the constants in the original source file, but the difference between the ordinary stop-and-recompile way of doing it is that the changes get picked up live, so there is no stop-and-recompile. (This isn't exactly novel; webdevs have had hot code reloading to various degrees for a while, but the conventional static languages folks have generally needed to rely on support in sophisticated IDEs to get the same thing, and many end up going without.)

For those who want the graphical control panel, I suppose you could do the complement: when a value is tweaked in-game, it could overwrite the existing value in the source file so you don't have to tweak in-game and then manually copy them out to source once you've landed on the values you like.


> but the conventional static languages folks have generally needed to rely on support in sophisticated IDEs to get the same thing, and many end up going without

Not really, building and using shared libraries to do this has been trivial for decades.


Can you point to examples that are in widespread use (or widely known)?


I don't think any game engine in use today except maybe the ultra-general ones would be without sections that are recompiled and reloaded live for gameplay code in this fashion. For all I know the ultra-general ones probably do it some stupidly complex way that sucks because they're overthinking things.

Most setups will likely stream both code and assets to wherever the game is running while you're developing as well, but I wouldn't necessarily call that trivial. Recompiling a shared library and hot reloading it if it's changed is (on the order of 20 or so lines), though, and I wouldn't expect any game to be developed without at least some code being reloaded this way.

While I haven't seen it as much, developing servers this way is also very doable, depending on how much you have to change data structures that have to be kept around.

Edit:

See https://www.youtube.com/watch?v=WMSBRk5WG58 for an example of how trivial it is to get this using just your OS facilities (`stat` + `LoadLibrary`/`dlopen`). None of this stuff is new and it's certainly been used with C/C++ or any language like it for a very, very long time.


What are you talking about dude?

I don't understand what's happening here. I asked for examples. I get in return two replies that have no examples and barely make sense in context.

Do you have examples of what you said?


I'm sorry, but if you don't understand what is being said to you I think the odds are pretty good that you were speaking out of turn when you were making proclamations about what developers usually do with static languages with regards to hot reloading code. If you don't know about dynamic loading of shared libraries and can't even parse these replies I think maybe you should just admit that you just didn't know what you were talking about when you said that. I don't have examples of this in open source projects because I don't have some list of open source games or whatever that I can point to (and honestly no one should have to give an example of something so trivial and well used).

I sent you a video of someone doing this in less than 2 hours without even using non-OS dependencies. It's an example of someone actually doing it and showing how trivial it is to dynamically reload code when it changes in statically typed, ahead-of-time compiled languages, a class of languages you claimed don't have hot reloading of code without "complicated IDEs".


> if you don't understand what is being said to you I think the odds are pretty good that you were speaking out of turn

Since I have you, do you have any personal insights on why so many programmers retreat to this territory—simply deciding that the other person is a dumb and confused noob and proceeding to more or less declare that that's the case? (It reads as insecurity/projecting, but it's hard to nail down without more concrete information about the other person. Maybe it's amenable to just trying to get a response to a direct question? Having said that, the amount of effort here to get a succinct response to a far more direct and less abstract question, "Can you point to examples [of what you're talking about]", makes me doubt.)

I've been programming for 20+ years. I know how shared libraries work (and dlopen—mentioned, finally, in your edit). I'm certainly confused—but the confusion is why you're bringing it up. It's totally unsuited as a drop-in replacement for what I described in the first comment you responded to. Did you read the blog post describing the actual mechanics of how the TWEAK macro works? I'm guessing no, since (a) your comments don't make sense in context and (b) that post references mollyrocket (Casey Muratori, the guy recording the screencast you linked to in your edit) by name.

If your solution is to refactor your code into a separate module and use dlopen, then we're talking about different things, and I'm not watching a 2 hour video just to find out whether it is the person I'm interacting with who is in fact the one not grokking the breadth and depth of the discussion.


> Since I have you, do you have any personal insights on why so many programmers retreat to this territory—simply deciding that the other person is a dumb and confused noob and proceeding to more or less declare that that's the case? (It reads as insecurity/projecting, but it's hard to nail down without more concrete information about the other person.

I think it's your attitude that does it, to be honest. For someone who seems to have issues understanding how `dlopen` and friends solves the same problem you're awfully dismissive yourself. I understand the blog post just fine, and it's honestly a way more complicated and less solid solution to something that has better solutions, including the one I mentioned (and yes, they solve the same problem).

You stated statically typed languages have issues hot-reloading code and I've explained to you that this is patently not true (and obvious to anyone who has spent the minimum amount of time you need in order to reload DLLs on-the-fly). You not being willing to watch a video of this isn't really a problem I'm trying to solve or otherwise handle.

> I know how shared libraries work

Then you should have no problem understanding why the responses you've received are relevant for hot-reloading. Maybe you should actually try to understand how shared libraries work more, then, because your confusion and your lack of understanding of it point to you not really understanding how you could use them yet.

Just to be clear, I'm trying to not be snarky, but your attitude about this is really getting on my nerves when coupled with your lack of understanding when you're the one with the original BS line about hot-reloading. I don't know why you can't just say "Oh, hey, I was basically just bullshitting because I've never done hot-reloading in a native language". It's not hard to recompile a DLL on file changes and reloading them when they change in a program. It's also not hard to compile a DLL. You implying that any of these things are hard shows that you really ought not to speak so definitively on any of them, and maybe you should tone down your dismissive tone when faced with proof of your misunderstanding.


> For someone who seems to have issues understanding how `dlopen` and friends solves the same problem [...] Maybe you should actually try to understand how shared libraries work more

See? You're still doing it. But I'll bite. Let's ignore the overhead of having to refactor your code for a quasi-plugin architecture in the first place. I'm going to ask you a very simple question:

How do you reload a shared library using dlopen after changing a source file without recompiling it?

> You stated statically typed languages have issues hot-reloading code

The wounded tone suggests that you're reading way too much out of the (true, and what should be totally uncontroversial) statement that, in order for edit-and-continue to work when debugging an arbitrary program they're working on, "the conventional static languages folks have generally needed to rely on support in sophisticated IDEs". This was not a challenge in a static-vs-dynamic (or PL-vs-PL) holy war, and your taking it as such has you behaving with insolent irrationality.

> you're the one with the original BS line about hot-reloading

Just to underscore the source of conflict here: you don't seem to understand that hot code reloading (especially hot code reloading with "no stop-and-recompile"—a constraint that was explicitly mentioned not once, but twice in my original comment) is a far more general subject than hot module reloading, which is what you're describing.

> It's not hard to recompile a DLL on file changes

And there it is, HNers. You hardly ever fail to move the goalposts.


Let's examine exactly what you said from the beginning so that we can put the goal post back from where you moved it with this ridiculous paragraph:

> Just to underscore the source of conflict here: you don't seem to understand that hot code reloading (especially hot code reloading with "no stop-and-recompile"—a constraint that was explicitly mentioned not once, but twice in my original comment) is a far more general subject than hot module reloading, which is what you're describing.

...

> In other words, rather than having a graphical interface for tweaking values at runtime, you interface with it the same way you normally would—by changing the constants in the original source file, but the difference between the ordinary stop-and-recompile way of doing it is that the changes get picked up live, so there is no stop-and-recompile.

You've had it explained to you that "stop-and-recompile" isn't even a factor because you can pick up these changes live regardless and you've been shown how.

> Let's ignore the overhead of having to refactor your code for a quasi-plugin architecture in the first place.

You keep pushing this idea that this is somehow difficult and takes a ton of time. It's not and it doesn't. How am I supposed to assume you're competent and knowledgeable when what you write says you're not and you aren't?

> How do you reload a shared library using dlopen after changing a source file without recompiling it?

You recompile it because recompiling it and reloading the code isn't an issue.

> The wounded tone suggests that you're reading way too much out of the (true, and what should be totally uncontroversial) statement that, in order for edit-and-continue to work when debugging an arbitrary program they're working on, "the conventional static languages folks have generally needed to rely on support in sophisticated IDEs". This was not a challenge in a static-vs-dynamic (or PL-vs-PL) holy war, and your taking it as such has you behaving with insolent irrationality.

This word salad suggests that you're butthurt about having your programming chops challenged when you make bullshit statements about reloading code.

I'm sorry, but I'm well and truly done with this. You say to assume competence and stuff but you seem to not understand at all that the things you're saying go against that entirely. You've now sort-of redefined what the original problem was to make it seem like hot-reloading shared libraries doesn't solve it.

Honestly, just learn to not make these silly statements from the beginning instead when you're out of your element.


I'll note that you didn't answer the question.

> You keep pushing this idea that this is somehow difficult and takes a ton of time. [...] recompiling it and reloading the code isn't an issue

No, I didn't. But you can continue trying to move the goalposts, and I will continue pointing out that that's what you're doing.

> You say to assume competence and stuff

Really? Where?

> You've now sort-of redefined what the original problem was to make it seem like hot-reloading shared libraries doesn't solve it.

Perhaps you and I have a different understanding of the word "redefined". Or maybe you disagree on what "recompile" means (and/or what it means to say that there is "there is no stop-and-recompile").

I'll (re) ask a simpler question: did you read the post describing TWEAK that I linked to[1]—that is, before you replied (and subsequently dove headlong into trying to change the subject with ad hominem attacks on the competence of the person you responded to)?

1. <https://blog.voxagon.se/2018/03/13/hot-reloading-hardcoded-p...>


Plugin systems in a lot of software effectively are just dll/so files loaded at runtime, you could even enable reloading them at runtime.

Usually you would want to hot reload the code sections though and allow the application to change the data section which is significantly more intuitive than changing source code to change data.

Edit:

On that note most game engines serialise state to disk, pretty sure it would be easy to just read the state from like a yaml file and update the state if you really don't want to write an interface.

TLDR: There are significantly easier solutions to "how do I update the value of a variable at runtime" than hooking into the running process and changing the variable in memory.


... what? Why is this posted as a reply to my question for examples?


These are examples


There are zero examples named in your comment, and your TLDR ("hooking into the running process and changing the variable") has no relevance to either the question you're ostensibly responding to ("Can you point to examples") or the subject of the blog post I originally mentioned ("a TWEAK macro to do this at the source level"). It's a baffling reply all-around.


Or you could just set a breakpoint somewhere and edit the values in your debugger.

This looks cool but I'm not sure how it is especially superior to functionality that is easily available through your favorite debugger (note: I'm speaking from a standpoint of Qt Creator + gdb; YMMV depending on your IDE/debugger setup), especially given that Hook seems to be mainly targeting macOS.


May I ask why the compilation and linking is taking 20 minutes?

Is it the include of LLDB.h?


It must be the LLVM project in the git submodule (https://github.com/abolinsky/Hook/tree/main/external), because all other dependency code (GLFW and Dear ImGui) should compile in a couple of seconds.


Thanks.


Looks really interesting. How much effort do you think it would take to get this running on Windows? I might give it a shot.


Not to downplay on the author efforts as this is quite hard to get right, if you want something today, Visual C++ already does hot code reloading.


How does this compare with attachable debuggers and memory windows in vscode?


So, it's a debugger? tthat can't break/single step...


This looks great! Are you aware of any similar projects?


Visual C++, https://learn.microsoft.com/en-us/visualstudio/debugger/hot-...


This is the most well known one: https://liveplusplus.tech/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: