
An alternative to shared libraries (2008) - entelechy
https://www.kix.in/2008/06/19/an-alternative-to-shared-libraries/
======
dagenix
> I like static linking. But code these days is getting extremely complex and
> bloated, so people needed an alternative. Instead of focusing on making
> their code more cleaner and lean, they started thinking about they can share
> this huge piece of complex and bloated code across several applications. If
> you think about it, if your code is small and clean, you wouldn’t feel the
> need for shared libraries.

One person's "bloat" is another person's "critical feature". When I see this
word thrown around, I'm always left worried about what is going to come next.
And then the author proposes avoiding shared libraries in favor of what sounds
like a single-system microservices model. Such a model doesn't resolve any of
the versioning issues of shared libraries but does add in all of the issues
faced by a distributed systems combined with dramatically higher invocation
overhead.

The stuff about plan 9 is interesting - but I'm left wondering why exactly
this is being reposted now.

------
_bxg1
I've done mercifully-little C(++) programming in my life, but the very idea of
dll's has always seemed extremely fragile to me. Depending that deeply on the
system configuration for basic functioning... ylech.

I also can't help but wonder if they're obsolete these days, given how
incredibly cheap disk storage is? Can anyone who works in that sector speak
more to that?

~~~
drwells
At least one C++ library that I rely on (that produces an incredible number of
template instantiations in the object files) ultimately results in a 2 GB
shared object file for the debug version (generating DWARF info for 300,000+
objects takes up a lot of space). The test suite for this project compiles and
runs about 5,000 executables that link, in some way, to that giant shared
object library. Statically linking the test suite would consume on the order
of terabytes (I am not sure the exact number; I have never tried) of disk
space which just is not feasible for a workstation, so dynamic linking is the
only reasonable option. Dynamic linking also makes testing easier since I do
not have to recompile tests as often.

That said, DLL hell is definitely real (as anyone who has ever used windows
knows). Things are generally better (though not perfect) with OSs like Debian
where dependencies are centrally managed across the whole system. In general,
doing C++ development, I have found it advantageous to dynamically link:
recompiling one library does not necessitate recompiling everything that
depends on it.

~~~
abenedic
> Statically linking the test suite would consume on the order of terabytes (I
> am not sure the exact number; I have never tried) of disk space which just
> is not feasible for a workstation

I read complaints like this a lot. Have you ever tried using -ffunction-
sections -fdata-sections and --gc-sections as in
:[https://elinux.org/images/2/2d/ELC2010-gc-
sections_Denys_Vla...](https://elinux.org/images/2/2d/ELC2010-gc-
sections_Denys_Vlasenko.pdf)

I find static linking to take up far less disk space when done properly, but I
keep seeing comments like this.

~~~
ihalip
MSVC linker also supports removing unused functions/data from the output
binary: [https://msdn.microsoft.com/en-
us/library/bxwfs976.aspx](https://msdn.microsoft.com/en-
us/library/bxwfs976.aspx). These can be applied irrespective of compiler
optimizations, so you should still be getting good PGD debug symbols.

Haven't tried it personally, but it's worth a try.

------
theamk
So an alternative to shared libraries is RPC over FUSE-like transport?

It is a sane idea and may work well, but I think the page overcomplicates it.
Why bother with filesystem -- having cryptofs implement "getattr" and
"readlink" seems like a total waste. Why not use the things _designed_ for
RPC?

For example, one may use raw unix sockets (this is how Xorg, and pulseaudio
work), or D-Bus (this is how disk mounting in modern desktop linux works), or
HTTP-based transports (this will be known as "microservices" then).

~~~
wiml
Or just… executables? My shell is "dynamically linked" to every executable on
the system. Git is "dynamically linked" to my $PAGER. My IDE is "dynamically
linked" to my compiler toolchain (which is itself composed of several
pluggable components which are "dynamically linked" to each other).

The low overhead, easily composable, encapsulated and abstracted nature of
processes is one of the things that made Unix an unusually functional
operating system in the first place. Shared libraries are great, but I think a
lot of the time people end up producing and linking a .so when defining an
old-fashioned shell-compatible interface would be perfectly good and far less
trouble.

------
ridiculous_fish
This proposal is crazy town.

 _If you think about it, if your code is small and clean, you wouldn’t feel
the need for shared libraries._

Not so. The key feature of DLLs is to enable software components to evolve
independently. When Apple changes the way a button looks in UIKit, all apps
get the updated behavior, because UIKit is dynamically linked. This has
nothing to do with the app size or its hygeine.

Replacing dylib calls with filesystems and fread/fwrite is absurd. IPC is much
harder than library calls. There's the obvious expense of turning every
library call into multiple system calls. It's also less flexible. When the
interface is a function call, the function can switch implementations from
eager to lazy, add or remove caches, etc. But when your interface is fread()
you must provide your data up front. Switching a field to computed on-demand
is a breaking change.

But the versioning issues are the worst. Reading from Plan9 /dev/mouse returns
49 bytes. I can't add a new field without breaking apps. The equivalent of
adding a method in an OO language is now a _breaking file format change_ in
this scheme. The post attempts to addresses this:

 _With filesystems, it’s trivial to add functionality without breaking
applications depending on older versions of your FS. That’s because all the
compiler sees is a bunch of fopen /fread/fwrites and is not going to complain
if the version of the filesystem changes because it doesn’t know._

The app can't break because the compiler can't detect the breakage. Huh?

 _Alternatively, if you’re thinking of modifying the behavior of your
filesystem; consider providing a version file in the root of your FS right
from the beginning. Applications would then write the version number they
expect to be working with in that file as a way of initializing the
filesystem_

This sounds very bad:

1\. Every API needs to be able to ingest and emit all past versions of
themselves, in perpetuity. This is obviously a huge maintenance burden.

2\. This does nothing about the reverse problem: how does an app run on
multiple versions of the OS? Instead of, say, using reflection to test if a
method is present, apps must be prepared to parse the product of every API
they use against every OS version they want to support.

3\. Applications do not have a single "version number they expect." Apps are
built out of multiple components written at different times which consume
different versions of the same API. There's no affordance for that.

So we're forced to use some structured data format with named fields, rather
than raw bytes. How did this come out of a desire for a "lean, efficient and
small" solution?

~~~
king_nothing
Exactly. It’s utopian “freedom” of tyranny like libertarianism. If every app
bundled all of its own shared libs, and the FS deduped files and shared
library loader deduped memory pages, there would be still be isolation, with
sharing AND add the ability to upgrade any piece of software without having to
maintain forests of NIX packages to or Habitat individual special libraries to
reference count. And, to delete an app, just delete the app, which decrements
dedupped files and deletes singular ones... no shared components to clean up!
ZFS lz4+dedup plus GNU stow

------
Iwan-Zotow
And where, f.e., SO/DLL allocated memory supposed to live? In SO only text
segment is marked ro and mapped shared between many processes. Data (whether
is it .data or malloced) is in the particular process memory and is NOT shared

------
axaxs
As Docker has proved to me, people don't care about saving space anymore. The
time of shared libraries has come and gone. And yes I realize you can use
shared libraries in Docker, but at that point it's acting more like a static
library oddly enough.

Side note - anytime I see Drepper mentioned I cringe a bit. I don't like to
badmouth people, but I can't think of a worse figure in recent open source
history.

