
Yocto: Tools for building a custom embedded Linux distribution - based2
https://www.yoctoproject.org/
======
lowercased
[https://www.yoctoproject.org](https://www.yoctoproject.org)

"The Yocto Project (YP) is an open source collaboration project that helps
developers create custom Linux-based systems regardless of the hardware
architecture."

~~~
Jemm
Press releases should really have a description of the project or at least a
link

~~~
umvi
Most people in the industry know about the two big players - yocto and
buildroot. It's for when you need a custom minimal Linux distro for your
gadget (IoT, router, modem, SoC, etc.)

------
swiley
I personally haven’t been impressed with yocto: It’s about as intuitive as
gentoo and reliable as buildroot.

Maybe I was using it wrong though. I think it’s a good idea.

~~~
ernst_klim
>I personally haven’t been impressed with yocto

That's a very mild way to express my feeling about it.

Maybe I'm asking too much, but Yocto/openembedded is the worst piece of
software I've ever dealt with.

The build system is a huge pile of opaque and fragile scripts. It's so
fragile, that if you don't put a space before a list of options, it would be
concatenated with other options without a space, leading to incorrect options.

Because there are no lists, lists are just strings of space-separated words.
Who needs proper data types?

There are no modularity either, everything is configured through strange
variables with names like CONFIG_image_graphics_mesa_package=... God knows how
this naming works.

Of course nothing is documented, and the discoverability is extremely low:
everything is done through hundreds of opaque shell scripts, python scripts,
half-baked python scripts for their build system.

The overall design of the build system is abysmally bad and fragile, and it's
harder to get a notion how the things work than to just build everything with
your own scripts.

Oh, and when something is wrong, build system just returns python stacktrace.

Is it that hard to use proper abstractions, data types, docstrings,
introspection mechanisms so that everything would become transparent, obvious
and sane?

~~~
usr1106
> Of course nothing is documented, and

This is plain wrong. There is a whole bunch of manuals
[https://www.yoctoproject.org/docs/](https://www.yoctoproject.org/docs/)

And there are several books
[https://www.yoctoproject.org/learn/books/](https://www.yoctoproject.org/learn/books/)

Without understanding the philosophy behind the whole system and the recipe
language just trying to "discover" something seems indeed mission impossible
to me. Yes, the learning curve is steep. The recipe language is not always the
least surprise if you don't understand it. On the other side, there is a lot
of magic support in the system. If you know what you are doing writing recipes
for new packages does not require much code.

~~~
ernst_klim
> There is a whole bunch of manuals

Ah, yeah, the manual. I forgot another abysmally bad part of Yocto: the
manual.

Yocto documentation is the single worst documentation I've ever dealt with.
It's a huge giant wall of text which tells you... nothing concrete whatsoever.

Wanna know how to change an image fs or image format? Go figure with the
manual, just look at these entries:

[https://www.yoctoproject.org/docs/3.0/ref-manual/ref-
manual....](https://www.yoctoproject.org/docs/3.0/ref-manual/ref-
manual.html#ref-images)

[https://www.yoctoproject.org/docs/3.0/ref-manual/ref-
manual....](https://www.yoctoproject.org/docs/3.0/ref-manual/ref-
manual.html#var-IMAGE_FSTYPES)

So concise, but even more useless. Which filesystems could I use? How to know
that? Ah, don't bother.

The whole manual is a huge wall of text telling you what classes, tasks,
recipes and dozens of other overengineered concepts are, but nothing about how
to use them beyond the basic examples. Wanna build an fs for the device with a
local storage and external ssd, good luck to figure out how: decoupling where
machine starts and image ends, and which part should you modify.

> Without understanding the philosophy behind the whole system and the recipe
> language just trying to "discover" something seems indeed mission impossible
> to me.

Yeah, it's hard to introspect something which have dozens of concepts, yet no
other form of introspection but file reading.

And I'm fine with file reading, but with yocto half of important stuff is
concealed within the build system, leaving just strange config variables and
gluing scripts on the surface, making it incomprehensible for a sane person.

~~~
usr1106
> Which filesystems could I use? How to know that?

A simple

    
    
      git grep IMAGE_FSTYPES
    

brings you up a long list. File meta/classes/image_types.bbclass seems
particular promising. And there it is

[https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/cl...](https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types.bbclass?h=zeus#n260)

I'm very far from claiming that Yocto is easy to understand. But you get much
more than you pay for. What alternatives do you suggest with a similar
functionality?

Do you think the Linux kernel documentation is better? The systemd source code
easy to grasp? Still, they are widely used. I tend to say the price you pay
for those is that you need to employ an expert and the expert needs to spend
quite some time on it.

If you buy a product you are entitled to request something from your vendor.
Good luck if you are a small company. I even tried when working for a 50,000
people corporation in the past and we got very little after many years of
paying huge fees.

In open source it's take it or leave it. Preferably take it and contribute to
make it better.

------
m0zg
I wish they'd generate toolchains I didn't have to install in order for them
to work, like Buildroot already does. With Buildroot I can just tar/gz the
toolchain and libs and integrate them into a build system. This lets me
support cross-build in a relatively pain-free way. Buildroot toolchains do not
care where they are located on the filesystem, so e.g. Bazel can download and
unpack them without any issues and they work fine. In contrast, Yocto
generates toolchains that have to be _installed_ and in which the installer
has to _rewrite the locations of dynamic libs_ during installation. This makes
Yocto toolchains effectively hostile to build automation.

~~~
nrclark
If you find yourself wrestling with this in the future, you might try
experimenting with Yocto's SDK generation capabilities. Most image recipes
have a 'populate_sdk' task that will generate a self-extracting shell script
that includes a full cross-compilation toolchain and a set of libraries for
the target. You can also add custom components to the SDK very easily.

Our CI system generates the SDK along with every build of our Yocto image.
Another CI job grabs the SDK and uses it for some out-of-tree compilation
work. It's very straightforward to automate that way.

~~~
m0zg
Yes, and that shell script can't be simply unpacked. It has to run, and when
it does, it rewrites paths to libs that the toolchain needs to pull in so that
they point to the correct installed locations. In contrast, Buildroot
toolchains require that only relative paths match, and do not require that
absolute paths are embedded into the binaries and *.so's.

~~~
nrclark
You're right, of course - I wouldn't say that Yocto's SDK generator is
perfect. But it does work fine, and you can install the SDK anywhere you want.

From a CI system's point of view, what's the logical difference between [tar
xf toolchain.tar.gz] and [./install_sdk.sh -yd .]?

------
nrclark
Thanks to the Yocto team for a great release. I'm excited to start playing
with the new hash-equivalency support, which is the biggest addition in this
release IMO.

Keep up the good work! Yocto is an amazing project, and I'm a heavy-user in my
day job.

------
karmicthreat
I was really hoping for big usability/learning curve improvements with 3.0.
I've tried using it and other engineers I know have tried. It just ends up
being a complete cluster.

------
usr1106
Yocto steep learning curve is a fact.

[https://www.youtube.com/playlist?list=PLD4M5FoHz-
TxMfBFrDKfI...](https://www.youtube.com/playlist?list=PLD4M5FoHz-
TxMfBFrDKfIS_GLY25Qsfyj) is an attempt to help.

Disclaimer: I have not watched them yet.

------
Const-me
When I needed an embedded Linux, I wrote a 3-pages long shell script to
customize vendor-supplied Debian image. Most of these 3 pages are sudo apt-get
purge commands (I don't need much except kernel).

Pretty sure I would have spent at least 10x more time building my own image
from scratch.

------
louthy
Got excited then, because I thought it was a follow up to the Yocto 808 [1]

[1] [http://www.e-licktronic.com/en/content/25-yocto-
tr808-clone-...](http://www.e-licktronic.com/en/content/25-yocto-tr808-clone-
tr-808)

------
qiqitori
I sometimes use Yocto at work to rebuild certain recipes in an embedded Linux
distribution, but I'm really confused about what "Yocto" and what "Poky" mean.
If anyone knows and has time to dispel my confusion, that would be awesome.
(I'm trying to answer a bunch of my own questions in here -- please let me
know if I got anything wrong.)

"BitBake" is easy -- that's the command you run to rebuild "BitBake recipes"
(a recipe is something like a Makefile, except that it's more "meta" \-- it
defines what to run in order to build software packages, i.e. how to find and
build a source directory that may have Makefiles/CMakeLists/whathaveyou in it
-- so I think it's a bit like e.g. an RPM spec file). So you run stuff like
'bitbake -c clean packagename' or 'bitbake -C compile packagename'. (BTW, I
often want to do things in a single command -- is that possible? Reloading the
whole cache every time gets old after a while.)

Fine okay, "Yocto" is presumably the name of the entire project. But what's
Poky?

> Poky is a reference distribution of the Yocto Project®. It contains the
> OpenEmbedded Build System (BitBake and OpenEmbedded Core) as well as a set
> of metadata to get you started building your own distro.

Hrm, what's OpenEmbedded Core?

> OpenEmbedded-Core is a layer containing the core metadata for current
> versions of OpenEmbedded. It is distro-less (can build a functional image
> with DISTRO = "nodistro") and contains only emulated machine support.

Hrm, okay. (BTW, for the people who don't know -- "metadata" most likely
refers to a collection of "BitBake" recipes (i.e. a directory containing
loosely related BitBake recipes), so the "core metadata" would likely be the
recipe for at least the Linux kernel.) (BTW2, in BitBake, you customize
recipes using "layers". The first layer contains recipes that build things in
a generic way, and then you add other "layers" that contain "bbapend" files
that override variables or commands in the lower-layer recipes to build
something the way you need it.)

Okay, so is the "OpenEmbedded Build System" still part of the Yocto project?
From Wikipedia: "BitBake is co-maintained by the Yocto Project and the
OpenEmbedded project." Okay, that explains that a bit. But why are they
separate in the first place?

Why does "Poky" have such a weird name, and what kind of recipes does it bring
to the table? Looking at the repository
([http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta](http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta)),
that's quite a lot of useful packages.

One other question that just popped up in my mind: are there any Desktop Linux
distros that use this as their build system?

~~~
wdfx
> Why does "Poky" have such a weird name

"Poky" in English means "small". The idea being that the distro is engineered
to build small OS images for small embedded devices.

~~~
kadoban
Though that definition is correct (had to look it up) the more common one
seems to be "annoyingly slow". at least that's the only one I actually knew or
ever heard.

~~~
shakna
It might be a slightly older phrasing, but to match your anecdote: I've never
heard of poky meaning slow.

More:

> It's located in a pokey little village in the middle of damn near nowhere.

~~~
zcrackerz
You've never heard of a slow-poke? Similar usage, you can be pokey or you can
be a slow-poke, both mean slow. The "small" definition is new to me. Seems
like difference between British/American English.

~~~
shakna
> You've never heard of a slow-poke?

Sure, but that doesn't mean "slow slow". The poke part of that that means
something similar to "yoke".

A slowpoke is something that's slow and dragging you down with it.

------
based2
src: [https://linuxfr.org/news/yocto-zeus](https://linuxfr.org/news/yocto-
zeus)

