Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Factorio – Statistics improvements, Linux adventures (factorio.com)
141 points by ibobev on April 26, 2024 | hide | past | favorite | 53 comments


> Most desktop environments will allow windows to supply their own decorations if they wish but will provide a default implementation on the server side as an alternative. GNOME, in their infinite wisdom, have decided that all clients must provide their own decorations, and if a client does not, they will simply be missing

I'm sure there was/is a good reason why this design decision was made, but it certainly seems a bit odd. Imagine if every game had to write decorations for every platform and desktop environment...


Their reason was that applications want to control everything about how they look, so they don't want compositor-provided decorations that clash with the rest of the application window. Whether that's a *good* reason... Well lots of electrons have been used up in that discussion already.

https://gitlab.gnome.org/GNOME/mutter/-/issues/217

https://gitlab.gnome.org/GNOME/mutter/-/issues/1143

... and probably many other places.


Ok, so make it optional to use custom window chrome/decorations, rather than make it mandatory.

GNOME devs are very arrogant. “Do it our way or go away.”

They make it very hard to hold onto any hope that there will ever be a reasonable, cohesive, Linux desktop experience.


KDE is reasonable and cohesive


I really like the hearty amount of configurability they make available thru the control panels. They're not afraid of giving you a lot of options, yet it's organized well, easy to use, and the defaults are sensible.


I think them and fyne fight over who can make a worse gui library experience.


I covet that guys job big time. However, I'm actually too busy playing factorio to do his job so there's that.


I'm a heavy user of and minor contributor to the Allegro library. We thought Factorio had got rid of Allegro long ago, but apparently some vestigial component remained. So long!


Allegro has been around for decades. I still remember using it with DJGPP in the mid-90s, which is at least four forevers ago.

One thing I remember making was a fire/heat visualization, not optimized at all but that didn’t matter much because I was targeting 320x240x8bpp.


Thanks for making such a great library! It served us well for many years.

Edit: My brain skipped over the "user of and minor" part of the sentence. Still, thanks for contributing to Allegro!


Using fork to save the game state in the background is really clever. love it.


I wonder how they deal with multi-threading. Forking a multi-threaded process is very error-prone (e.g. allocator locks can remain orphaned).


The good thing about this usecase is that they do not need to solve the general problem of forking a multithreades program.

The child process is doing a very specific job that amounts to serializing the contents of memory and writing it to a file. Once that is done is can simply exit, and all the orphans are cleaned up.

While this is admittadly more complicated, it is the same general idea behind fork+exec. Sure, your serialization logic cannot call malloc, but how often do you really need dynamic memory allocation. And if you really do need to, you can use a simple mmap backed bump allocator (or any other custom allocator you want)

For datastructure locks protecting gamestate; just wait until the gamestate is in a consistent state before forking.

Of course, if you try reusing existing code, or forget about the limitations while writing new code; it would be easy to introduce non-deterministic bugs.


It still seems super error prone:

It is very difficult to write general purpose C++ code that does not call malloc somehere.

Something fails and you want to display a message `"Error: " + reason`? Bam, malloc right here, serialisation process may hang forever in a malloc lock.

You quit the game, the parent process exits, and the serialisation process gets reparented to init, invisibly using up your RAM until you reboot.

fork()+exec() works in C because C has no invisible memory allocation, and even there you'd usually try to not call any function in between the two to be very sure.

Using fork() without being 100% sure there can be no malloc usually means inviting years of rare, hard-to-reproduce random weird bug reports.

Beyond that, as the post mentions, fork() needs "requires a significant amount of RAM to work" if many pages are touched due to copy-on-write, and copy-on-write also slows down the main game.

It seems much safer to use a thread for saving the game state.


In principle, using a thread for saving the game state is a much more difficult problem, since the game state itself will mutate. You need to be extremely careful to assure that the state you serialize is consistent and is at least a plausible game state (even if there is a bit of wiggle room to allow for game states that technically never existed). This complexity invades every aspect of game logic; and bugs here tend to be subtle corruption of your save data that will take a while to notice; thereby obscuring the relationship to the corruption, and the save/reload cycle.

In contrast, forking with its COW semantics is conceptually easy. You just fork. The main process can continue running, and the child process gets a frozen snapshot. There is a bunch of overhead from the copy part of copy-on-write. However, most of that overhead will likely be spent in the first frame; which is still a significant improvement over the pause time associated with stop the world saving. In practice, coding for the child process is tricky. However, it is self contained and responsible only for a relatively simple problem. No complex problems to solve, just a relatively small amount of code that needs to be written carefully.

The RAM usage is a real trade-off inherent in the approach.

> You quit the game, the parent process exits, and the serialisation process gets reparented to init, invisibly using up your RAM until you reboot.

Or until the short-lived child process finishes its work and exits on its own.


> Or until the short-lived child process finishes its work and exits on its own.

I was talking about the malloc-deadlocked child. That will not finish.


How do you know that the game state is in a consistent state in the child?

A stopped thread may be in the middle of mutating an array or struct.


If they have the ability to pause the entire game in a consistent state while its state is being saved (in the foreground save case), then they certainly have the ability to pause the entire game in a consistent state so they can fork. Just the latter pause will be much much shorter.


Because you forked while being in a consistent state?


Ah, I see. I was assuming that the achievement of consistent state must somehow be achieved in the child.

If the parent can achieve consistent state (e.g. doing the equivalent of pressing "pause"), why not do the following instead:

While paused, memcpy the current memory to a buffer, then simultaneously {resume game, spawn thread to write the buffer to disk}. In C++ the memcpy might be even more convenient with the copy constructor.

This will introduce a short delay for the copy, at the speed of RAM bandwidth.

But that copy will need to be done anyway, straight away, as the parent poster says:

> that [copy] overhead will likely be spent in the first frame

With fork() it just happens in the kernel instead of in userspace, thus likely slower (1000s of individual sequential page faults, instead of a single contiguous allocation).

So if the fork() approach can somehow do it faster, I'd be curious via what mechanics.


They mention that there's a rare freeze bug. That does sound like it's most likely one of those orphaned lock problems or similar.


The post was already getting long so I didn't mention this, but I already found an fixed a potential deadlock due to each process not closing the redundant ends of the pipe, which was specifically touted as being a potential cause for freezing. However, someone reported the bug again after that change went out, so there's still SOMETHING else causing it to hang that I haven't tracked down yet.


Is that relevant if the forked process only has to save the game-state to disk and then die? I'd think reading the game-state wouldn't require some mutex lock.


Will never cease to be amazed by the performance that Factorio gets.


For anyone who hasn't played this before, you should know that the references to addiction are not jokes (for many people, at least). I played for a while, and had a blast doing so, but after I had achieved the simplest nominal 'victory' and noticed I was still compelled to disposed of a couple of good weekends towards meeting some self imposed challenged I felt it was time to stop.

I didn't have the discipline to just play occasionally, but I did have the discipline to delete the game and all my saves and blueprints one day, and I'm very glad I did. I recommend any player at least consider whether they should do the same.

Repeat, I had a great time, the community is great too, but the only way I could sustainably stop myself from turning that into a big waste of time was to delete the game.


Is this about the upcoming expansion or the base game? I am not reading FFF to avoid spoilers and instead get the full wow effect the first time I play the 2.0

Edit: expansion I guess, since the last 3 releases don't mention statistics (https://forums.factorio.com/viewforum.php?f=3)


Every FFF since they started doing them again has been about the expansion or base game updates that won't be released until the expansion, so I would continue to not read them :)

The way they are doing it is a bunch of engine changes to the base game that will be released as 2.0 for free. Then the actual expansion is implemented as mods. So some of the FFF topics like this are about things that will come to the base game when the expansion launches.


Everything I talked about here will apply to Factorio 2.0 as a whole, whether or not you have the expansion DLC.


Many of the changes affect the base game as well. Accumulator charge graphs were very overdue.


When they say they removing their Allegro dependency "eliminated 123,024 lines of code from the game" I assume this is because they'd vendored in the code?


Yes, we vendor and statically link as much as possible to avoid dependency hell across distros and platforms.


Yes. I believe they also vendor SDL, because they've mentioned in the past making local changes to SDL for their needs. There's also a comment in the Reddit discussion by the author of the Linux about deleting a large number of lines of code when he removed their vendored FreeType, so it's likely the case for Allegro too.

https://old.reddit.com/r/factorio/comments/1cdifrh/friday_fa...


Vendored or not, it's code that ends up as part of the game.


But they probably wouldn’t have bothered to count the lines of code in the library if they hadn’t checked a copy of it into their source code.


> The game has decorations now, but the theme doesn't match. Thanks GNOME!

Yeah good luck with that one :)


If they drop X11 support that will finally be the end of my cracktorio addiction.


Apparently I didn't word that section very well, because many people took away this impression. My apologies.

Factorio will continue to support X11 for as long as SDL does, in other words, essentially forever. The only change here is that X11 is no longer required for the game to launch at all. SDL will load whichever video driver is available at runtime, or if you have both, you can choose which one to use in the graphics settings.


Excellent news, thank you for the clarification and all of your work!


I don't see anything to indicate that they're dropping X11 support? It sounds like they're dropping dependencies that would require X11 to be installed.


No, it doesn’t sound like they’ll be dropping X11 support any time soon. At least, not before the 2.0 release. It is nevertheless one of those worrisome things.


They’re dropping a hard dependency on X11 and instead using a library that interfaces with X11 or Wayland for it.

It still runs fine on X11.


There is a comment in the forum discussion by the author of the Linux section that X11 support is not going away, simply because SDL supports it.

https://forums.factorio.com/viewtopic.php?p=609071#p609071


It's time to make WaylandX, for running Wayland applications on X. Well, I happily use Wayland so I won't use it, but maybe you would.


Looks like your wish has been granted. https://news.ycombinator.com/item?id=40175731


That's already a thing, called XWayland, and it's the only way to make Wayland actually usable on Linux desktop.


XWayland is the exact opposite of what GP is asking for.


I don't think they are planning to do this; they are at least maintaining support for whatever MacOS (Metal?) and the Nintendo Switch use in addition to Wayland, which suggests to me they are dropping X11 as a hard dependency not as a usable backend.


Just run the last version that supports X11?


You could use Proton to play the Windows version in x11 just fine, I'm pretty sure.


Same here.


Mandatory CSD was always a bad idea imo and this is yet another reason


Sorry, what is CSD? I can't find any reference to this acronym.


Client-side decorations: The title bar, the min-max-close buttons, and the window border.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: