

Gcc compile error with 2GB of data - p4bl0
http://stackoverflow.com/questions/6296837/gcc-compile-error-with-2gb-of-data

======
dexen
tl;dr:

the error is returned by the linker rather than compiler itself. It is not a
bug, just size limitation of the default memory model. Linux x86_64 provides
`large model' -- as pointed out by VJo, but it is not supported by GCC before
4.6, [http://stackoverflow.com/questions/6296837/gcc-compile-
error...](http://stackoverflow.com/questions/6296837/gcc-compile-error-
with-2gb-of-data/6297704#6297704)

------
joeyh
What a great technical problem, except for this bit:

> Btw, I didn't try to hide behind "this is scientific computing -- no way to
> optimize". It's just that the basis for this code is something that comes
> out of a "black box" where I have no real access to

Black box != science.

~~~
esrauch
A research assistantship I held was based on classified data; all of the
published work had to be approved by the DoD and the actual data we used
wasn't allowed to be published which made our results entirely unreproducable.

------
__rkaup__
Is including all that data in the object code really necessary?

~~~
dekayed
Generally not. My last project involved building a framework in which
scientists could run calculations for certain types of risk. A prototype built
by some of said scientists involved code like seen in the SO question. After
looking at the types of calculations being done, we figured out that most of
the equations that were used were similar and could be generalized. We also
moved all the coefficients and parameters for all the equations into
configuration files. I suspect that a thorough evaluation of this code would
reveal something similar as in my case.

------
juiceandjuice
Those expressions look a lot like an alternating series to me. You should be
able to generate an expression to produce them fairly easily.

------
bsiemon
This sorta reminds me of the silly things people do with JavaScript.

~~~
nddrylliog
Such as... an MP3 decoder? <https://github.com/nddrylliog/jsmad>

------
malkia
I suggested to try and use LuaJIT.

It has very good double precision floating point, and it looks like SSE/SIMD
were not used directly from the code.

LuaJIT should garbage collect unused code, if not it always regenerates it. It
should be able to compile and execute tons of code as he has (and it looks
like all his stuff is generated, but for C++).

~~~
premchai21
Last I checked, the LuaJIT allocator on AMD64 platforms uses only a small part
of the address space for the Lua heap, partly so that more efficient type-
punned representations can be used internally. I don't remember what the limit
is exactly, but it's only a few GB (and beyond that the GC starts having
trouble anyway). I don't know whether this applies to the machine-code JIT
output, or to external cdata arrays, but it's something to watch out for here.

