
Propeller: Profile-guided, optimizing, large-scale LLVM-based relinker - uluyol
https://github.com/google/llvm-propeller
======
saagarjha
> It has scalability issues and to rewrite a binary with a ~300M text segment
> size

I'm curious what kinds of tasks require a binary with a 300 MB text size…

~~~
cwzwarich
> I'm curious what kinds of tasks require a binary with a 300 MB text size…

C++ lends itself to bloated binaries. Template instantiations are duplicated
even for types where the instantiations could have identical code. Compilers
have traditionally favored overly aggressive inlining over whole-program
optimization.

Rust didn't solve the problem; it just duplicated the C++ approach, and made
it worse by effectively compiling with the equivalent of C++ exceptions
enabled. The next great systems language should target binary size and CPU
frontend overhead.

~~~
jcelerier
> Template instantiations are duplicated even for types where the
> instantiations could have identical code.

compilers and linkers had identical code folding optimizations for a looooong
time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC
(-fipa-icf))

~~~
saagarjha
I was going to reply with a similar comment, but I'm unable to actually get
this to happen when I tried it now:
[https://godbolt.org/z/4VkEQc](https://godbolt.org/z/4VkEQc). Perhaps the
optimization needs more work?

~~~
jcelerier
No, it's just that the assembly output of godbolt happens before these
optimization phases.

On my machine I get no unsigned instantiation, only the signed one :

    
    
        $ objdump -D a.out | c++filt | grep foo         
         563: e8 21 01 00 00        callq  689 <bool foo<int>(int)>
         56c: e8 18 01 00 00        callq  689 <bool foo<int>(int)>
        0000000000000689 <bool foo<int>(int)>:
    

and the main :

    
    
        0000000000000560 <main>:
         560: 53                    push   %rbx
         561: 31 ff                 xor    %edi,%edi
         563: e8 21 01 00 00        callq  689 <bool foo<int>(int)>
         568: 31 ff                 xor    %edi,%edi
         56a: 89 c3                 mov    %eax,%ebx
         56c: e8 18 01 00 00        callq  689 <bool foo<int>(int)>
         571: 0f b6 d3              movzbl %bl,%edx
         574: 48 8d 3d b1 01 00 00  lea    0x1b1(%rip),%rdi        # 72c <_IO_stdin_used+0x4>
         57b: 0f b6 f0              movzbl %al,%esi
         57e: 31 c0                 xor    %eax,%eax
         580: e8 cb ff ff ff        callq  550 <printf@plt>
         585: 31 c0                 xor    %eax,%eax
         587: 5b                    pop    %rbx
         588: c3                    retq   
         589: 00 00                 add    %al,(%rax)
         58b: 00 00                 add    %al,(%rax)
         58d: 00 00                 add    %al,(%rax)

------
chmaynard
Propeller looks very interesting. Judging from the number of commits, the
project has been around a long while. Does anyone know its history?

~~~
ddcc7
It's a fairly recent project, you can see the original announcement from the
team on the LLVM mailing list in September 2019:
[https://lists.llvm.org/pipermail/llvm-
dev/2019-September/135...](https://lists.llvm.org/pipermail/llvm-
dev/2019-September/135393.html)

------
dang
Related current thread:
[https://news.ycombinator.com/item?id=22654229](https://news.ycombinator.com/item?id=22654229)

------
fulafel
Now if we could only do the same profile guided layout optimizations with
data...

------
fulafel
Can anyone summarize differences with previous work like

[https://cseweb.ucsd.edu/classes/sp00/cse231/dynamopldi.pdf](https://cseweb.ucsd.edu/classes/sp00/cse231/dynamopldi.pdf)
?

~~~
Joky
The closest previous work would rather be Facebook BOLT I believe:
[https://github.com/facebookincubator/BOLT](https://github.com/facebookincubator/BOLT)

~~~
fulafel
Right, I misremembered what Dynamo does - from the Bolt paper references I see
that I was actually thinking of Spike
([https://www.usenix.org/legacy/publications/library/proceedin...](https://www.usenix.org/legacy/publications/library/proceedings/usenix-
nt97/full_papers/cohn/cohn.pdf)).

Interesting stuff!

