In the reconfigurable world, place and route engines run metaheuristics to find good-enough solutions in an intractable search space. I'm imagining if there's a compiler stage which implements even a single optimization heuristic, any prospect of hash-wise reproducibility goes out the window?
One potential for nondeterminism is if the compiler or linker is multithreaded, and multiple threads could update a shared data structure in no fixed order.
Diving a little bit deeper down the same rabbit hole, David A. Wheeler (as he emphatically insists on being cited) also made note of threads with non-deterministic scheduling. Lots of notes on hardware-centric implications as well.
Definitely turns my perspective around on what it means to trust software. I suppose the effort is a proper graduation from masturbating to safe sex, so to speak.
See: Buridan's Principle. Also: http://research.microsoft.com/en-us/um/people/lamport/pubs/b...
This was a real issue in hardware design in the 60's.
Most compilers aim for reproducibility (except for things like timestamps) anyway, as your customers would rightly complain if compiling their programming produced different speed executables each time, and debugging compiler problems would become even harder!
I suppose that's what I was seeking to gain clarity on.
The remark on execution speed is a bit confusing though, considering most software applications do not operate in a strict cycle-accurate, real-time environment, which is further exacerbated by the caching schemes of modern processors.
From your perspective, given the same repeatable test input, what threshold qualifies an executable's speed as being different?
Here is a random example (you can find lots of others):
Which would you consider to be the surprising part: ~0.75% average execution speed slowdown, or on the order of tens of milliseconds slowdown?
W.r.t. page alignment, thought you may find this to be an interesting counter example.
Assuming you compile with the same options and in the same environment, the machine instructions should be the same.
EDIT: The compiler in question though is whatever Visual Basic 6.0 uses, so I wouldn't call it modern today.
Could be the same or similar issue you are facing, given the age of the compiler.
I don't have a reference to link to though, Roger Dingledine told me this in person at a conference when discussing issues related to reproducible builds. But there should be an issue in the tor bug tracker.
The build-benchmark-repeat process is effectively what place and route metaheuristics in the FPGA world automatically do.
I'm imagining not all compilers are created equally, so I threw all handwaving general assumptions on determinism out the window.
Roslyn already has a switch to do this, /deterministic.
There is a spec for this though, https://www.microsoft.com/en-us/download/confirmation.aspx?i...
> A stamp that is used for different purposes in several places in a PE or COFF file. In most cases, the format of each stamp is the same as that used by the time functions in the C run-time library. For exceptions, see the descripton of IMAGE_DEBUG_TYPE_REPRO (section 6.1.2). If the stamp value is 0 or 0xFFFFFFFF, it does not represent a real or meaningful date/time stamp.
> The presence of an entry of type IMAGE_DEBUG_TYPE_REPRO indicates the PE file is built in a way to achieve determinism or reproducibility. If the input does not change, the output PE file is guaranteed to be bit-for-bit identical no matter when or where the PE is produced. Various date/time stamp fields in the PE file are filled with part or all the bits from a calculated hash value that uses PE file content as input, and therefore no longer represent the actual date and time when a PE file or related specific data within the PE is produced. The raw data of this debug entry may be empty, or may contain a calculated hash value preceded by a four-byte value that represents the hash value length.
I'm not familiar with the concepts in this article. Would this explain the sort of issues I had?
On the other hand, reproducible builds are /important/. The better we can trust our tools and verify that they're exactly what we think they are, the less we need to worry about the security of our tools, and the more we can get on with /using them/.
In the end, despite enabling the continued use of a deeply proprietary and anti-freedom platform, this is a good thing.
that seems a bit of a problem for every product not using git (which is still a lot, and whoever said git was the final solution anyway?)