
The semver trick (2019) - aazaa
https://github.com/dtolnay/semver-trick/
======
kazinator
> _In Rust (as in C, for that matter), two structs are not interchangeable
> just because they look the same._

In C, two structs with different tags in the same translation unit are indeed
incompatible. (That includes two structs with no tag that look the same,
because, effectively, each has some internal, machine-generated tag.)

Within the same translation unit, looking the same is not a consideration at
all; it's all based on the tag symbol.

Between translation units, two complete struct types that are exactly the same
(same members, of the same type, in the same order, with the same names, and
other details like bitfield configuration) are compatible are if they both
have the same tag.

But they are also compatible if neither has a tag: and in that case, they
would be incompatible if they were in the same translation unit.

Basically, two machine-generated tags for anonymous structs are considered
equivalent if they are in different translation units, in which case
compatibility is purely down to structural equivalence.

In practice, C compilers do not police struct tags at all between translation
units; you can get away with it if the same object is known as "struct point {
double x, y; }" and "struct pointe { double x, y; }" in another translation
units. You can even change the member names, for that matter.

You will run aground with a C++ compiler, though, due to type safe linkage
which pulls in class names.

FFI implementations obviously don't care about any of this. If I'm declaring
an FFI type that is to be compatible with the C "struct stat" in a Lisp, so I
can call the _stat_ function, these concepts have gone out the window. The
memory layout compatibility is all that matters: correct sizes, alignments,
conversions.

~~~
forrestthewoods
What is a tag in C?

~~~
formerly_proven
The name of a struct ("struct foo", the tag is "foo").

------
andybak
Not a Rust user but this all sounds remarkably painful. Is it common? The only
other compiled, type-safe language I use regularly is C# (via Unity) and I
don't recall this level of upheaval.

~~~
geofft
It's only really relevant when a) you release semver-incompatible upgrades
(e.g., 1.x to 2.x), b) people use types from your library in their APIs, i.e.,
their types depend on yours, and c) your library is sufficiently widely used
that people are likely to have transitive dependencies on it by multiple
routes, such that a couple of their direct dependencies will have upgraded to
your new version and a couple wouldn't have gotten around to it.

The 'libc' crate was a perfect storm of all three. It was still at 0.x, and
the whole point of the library is to define types/bindings for others to use.

It also requires the FOSS model of independent owners, I guess, since that's
why some consumers haven't upgraded. Inside a company and especially in a
monorepo, someone - probably the person making the breaking change - would
have just upgraded everyone at once. And a redistributor of FOSS like a Linux
distro would have probably just patched everyone at once or held back upgrades
until they could - in fact, this is the same shape of problem as upgrading
from OpenSSL 1.0 to 1.1, or GTK 2 to 3, or libstdc++'s new ABI, or whatever.

~~~
yjftsjthsd-h
> Inside a company and especially in a monorepo, someone - probably the person
> making the breaking change - would have just upgraded everyone at once.

Ideally yes, in practice not always. The easy way to break it is of course
political problems, but it's perfectly possible to be stuck in a situation
where unfortunate design decisions early on leave you stuck unable to make
breaking changes even if everyone is on the same side (at any rate, not
without taking a massive service disruption that would violate our contracts
with customers).

~~~
samus
At work we had that situation with Spring. In the far future, we would like to
upgrade to Java > 8, which makes an upgrade to Spring 5 necessary. But the
frontend module uses Spring Flow Portlets, which are deprecated and were
removed in Spring 5. This was not really a concern though, until too many
security issues were found that made the upgrade necessary. In the end we
settled on leaving the frontend on Spring 4 and upgrading everything else. It
is very ugly, but the only real issue so far is that Spring 4 cannot parse
Spring MVC 5's REST encoding of exceptions. Fortunately, we will build a new
frontend this year...

------
gazarullz
would have been nice to tag the post as a rust + semver regarding post instead
of only semver as the title implies

~~~
mjw1007
You could use the same trick for other languages and packaging systems, as
long as they support linking multiple major versions of the same library into
one program.

------
sixstringtheory
Wouldn’t this still break a consumer who doesn’t realize the trick is being
employed? Doesn’t this assume the consumer is making the requisite changes in
their library as part of upgrading to the “nonbreaking” upgrade that slips the
new type definition in via its dependency on its own breaking upgrade?

~~~
Tuna-Fish
No. Someone using the old version can continue using it, completely unaware of
the fact that anything changed.

------
diegocg
Wouldn't this be solved easily with symbol versioning?

~~~
kibwen
Possibly (I've never seen a concrete proposal of per-symbol versioning, so I
can't say for certain), but in that case all downstream consumers of your
library would have to pre-emptively declare the version for each and every
symbol they use.

Now obviously not all users will be using every symbol from every library that
they depend on, but, to use the OP's example of libc, which contains over
4,500 symbols... that starts to look unwieldy.

Of course, you could technically do this _today_ by just having every symbol
in its own crate. And while that seems like quite a stretch, I think it _is_
the consensus that the libc crate in particular is too big, and should have
been split out into multiple crates in order to better facilitate these sorts
of upgrades. So there might be a practical middle ground by having one crate
for "crucial, fundamental symbols" and a separate crate for "ancillary
symbols", where each could be versioned separately. That might get close
enough to the precision of per-symbol versioning without getting unwieldy.

------
nixpulvis
I would be very interested in how this could be created automatically by
`cargo` itself. A cargo semver tool which can bump versions, and create two
versions on major/minor breaking changes like this post recommends would be
really cool to see.

------
dmitriid
> Servo found themselves coordinating an upgrade of 52 libraries over a period
> of three months

And yet... people only complain about npm having lots of dependencies ;)

~~~
ilammy
Well, looking at various crates pulling in lots (dozens) of dependencies, I'd
say crates.io is clearly moving somewhere into that direction of
microlibraries with extensive code reuse.

~~~
ChrisSD
The trouble with monolithic crates is that they're monolithic. People complain
about the massive amount of code they have to pull in just for a few
functions, and the effect this can have on compile times.

The trouble with small crates is they're small. People complain when the
number of dependencies grows larger and mockingly reference "leftpad".

~~~
josephcsible
The issue with leftpad wasn't that it was small. The issue was that so much
production code relied on it not going away, despite npm letting the author
make it go away.

~~~
tsimionescu
Yes, the problem that broke everything was what you mentioned. But what struck
everyone more was that this all happened because of a few line function that
should have obviously never been a dependency in any sane project.

------
Lammy
> The Rust library ecosystem has a history of traumatic library upgrades. The
> upgrade of libc from 0.1 to 0.2 is known as the "libcpocalypse".

Somebody should come up with a commonly-agreed-upon versioning scheme where we
can indicate that breaking changes should be expected so people could avoid
putting themselves in situations they regret.

~~~
steveklabnik
The versioning scheme is not the issue here.

The issue is that there can only be one copy of certain kinds of libraries,
and when there's a major version bump, it creates a fork in the ecosystem.

