Ask HN: Why is Yocto so slow but so popular?

[c0deR3D]

I was using Scarthgap (released April 2024), which is my first try. I realized the following:

1. Everytime a bb file is changed, it takes minutes to finish, for example, parsing recipes.

2. Building of a specific component takes minutes at the Yocto init things.

3. Invoking a Linux menuconfig takes minutes at the Yocto prologue and epilogue (e.g. resolving dependencies).

I see that Yocto manages the package in a way that provides good extensibility, modularization and maintainability. But it just way too slow.

I've heard that there is a database in Yocto which hinders the scalability, but why it seems that everyone using with Yocto doesn't mind waiting for minutes just for a minor config change?

Well, maybe I was wrong because I'm using Xilinx flavored (PetaLinux) Yocto.

How do you think?

[not_your_vase]

I'm quite heavily invested in Yocto, both professionally and on hobby level. Not saying that bitbake is blazing fast, but these minutes long parsing and init steps will be something specific to your machine. On my home desktop (Ryzen 5600G + 32GB RAM) Bitbake init including all parsing doesn't take more than 10 sec. It is similar on my laptop (Ryzen 4700U + 16GB RAM).

It takes only more if I change some global config, but even then parsing doesn't take more than 40-50 seconds, before it would actually start cranking on the tasks.

Have you tried asking around the Yocto mailing list?

[c0deR3D → not_your_vase]

Your scenario makes me suspecting that it is Xilinx flavored Yocto causing the problem. I think that removing some unused Xilinx-specific layers/recipes can reduce the prologue and epilogue execution time.

[not_your_vase → c0deR3D]

Well, Xilinx layers are definitely not the most light-weight ones, though they still shouldn't cause parsing/task init to take minutes. By any chance, are you using WSL? I have heard some complaints about storage I/O performance, when it comes to this...

[c0deR3D → not_your_vase]

Nope, mine runs on a native Linux box featuring Intel i7-1370P and 32GB RAM. Maybe Xilinx has some tuning which makes Yocto become slow at parsing the recipe dependency and the likes.

I've tried asking the Xilinx community, and got only a reply saying that there is a database in Yocto which limits the scalability.

[Terr_]

I used Yocto/Bitbake at an old job where they needed to make a custom distro for a bespoke device that went into extremely large vehicles. IIRC the biggest hit for rebuild speed involved certain settings that had ripple-effects across many different recipes or artifacts.

It was challenging since I somehow got saddled with babysitting the system (despite having no C/C++ experience) as the team gradually broke apart for other projects. I was probably too amiable, I need to move more towards the "give me a cool sexy project or I walk" end of the spectrum.