
Duetto: a C++ compiler for the Web going beyond Emscripten and Node.js - apignotti
http://allievi.sssup.it/techblog/?p=852
======
chrismorgan
That cnontetris example seemed to be having dire effects on Firefox's memory
usage. It was using 21% of my 4GB, but then when I started that it went
jumping up to over 80% and back down again and back and forth. Is it really
causing the allocation and freeing of over 2GB of RAM just like that every so
often?

Oh, by the way—the game was entirely unplayable for reasons likely to be
related to this memory thrashing.

~~~
apignotti
Our memory management approach is still not tuned and optimized. There are
quite a few low hanging fruits that should provide great improvements, but we
have not yet implemented them - Alessandro of Leaningtech

~~~
chrismorgan
Would you mind if I suggested that arbitrarily using a couple of gigabytes of
memory like that was a fairly severe problem? :-)

Seriously, I would like to see a warning put on links to that example to the
effect that at present it will make your browser use up to 2½ GB more RAM and
will thrash your memory. Because it did render my browser (and to a lesser
degree my system) unresponsive for a couple of minutes while things swapped.

~~~
apignotti
You are absolutely right, it is a severe problem and I apologize for this. We
are adding a warning on our examples page right now.

------
utopcell
I wonder how this compares to wt
([http://www.webtoolkit.eu/wt](http://www.webtoolkit.eu/wt)) in terms of
performance.

------
apignotti
As many have noticed the blog is being seriously crushed by the heavy traffic.
We are mirroring the content here:
[http://leaningtech.com/duetto/blog/2013/10/31/Duetto-
Release...](http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Released/)

~~~
angersock
_snip_

EDIT:

After some consideration, I've removed my post. Though the irony is great, the
fact is that this is a cool use of LLVM and a pretty interesting thought-
experiment made live. I know I wouldn't appreciate the kind of heckling I'd
just posted were our roles reversed.

Good work on the technical stuff folks.

~~~
mtinkerhess
Cut them some slack, jeez.

~~~
angersock
Done.

~~~
apignotti
I really appreciate it. We are doing our best with the limited capabilities of
three persons. Thanks a lot. - Alessandro of Leaningtech

~~~
angersock
You guys are doing great work! Please don't confuse my disdain for the concept
of C++ in the browser for a lack of appreciation for your work.

------
blueblob
Are there any tutorials on this so that we can compare it to emscripten?

~~~
multimillion
We will publish extensive tutorials soon, for now you can find some examples
on
[http://leaningtech.com/duetto/examples/](http://leaningtech.com/duetto/examples/)

In addition, the full source code of Nontetris (
[http://allievi.sssup.it/jacopone/cnontetris/](http://allievi.sssup.it/jacopone/cnontetris/)
) will be published in a couple of days, with a 'guide' on porting a C++ game
to Javascript with duetto.

Stefano of Leaningtech

~~~
flohofwoe
For games I think you need to show that duetto is just as fast as emscripten
when running. emscripten-generated code is very light on the garbage collector
(inside emscripten generated code no garbage is produced at all, only in some
wrapper code which needs to talk to JS libs), and having the "heap" in a plain
typed array usually also gives a performance advantage, since as in C++ you
have exact control of the memory layout of your data structures. Not having to
construct and destroy JS objects all the time, and keeping all data in typed
arrays is essential for performance I think. This is even true without diving
into asm.js.

~~~
apignotti
We have written a detailed post a few months ago comparing duetto to
emscripten. We also explain why we think it's not a good idea to use a pre-
allocated typed array heap:
[http://leaningtech.com/duetto/blog/2013/05/28/Comparing-
to-a...](http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-asm.js/)

~~~
flohofwoe
Disclaimer: speaking strictly from a game-devs perspective: I remember that
post and from a game dev's perspective the pre-allocated-memory vs. reclaimed-
through-garbage-collection argument is a bit weak IMHO. Games have predictable
memory usage, and game devs are used to see memory as a static, finite
resource anyway (since that's how it is on consoles). Grabbing a chunk of
memory upfront is quite common for games. And as for the (big) size of the
pre-allocated memory: either your game needs that much memory at a time or it
doesn't, if the user doesn't have that much free you're f*cked either way.

The perf comparison is missing a "asm.js + V8" bar, asm.js code is usually
faster then non-asm.js code, even if the JS engine isn't implementing AoT
compilation as FF does.

Edit: fixed "...uncommon -> common..." above

~~~
apignotti
I see your point. Still, I believe the browser environment is different from a
console since it is not supposed to monopolize system resources while being
used. Managing memory in smaller short lived chunks is more fair for the
system as a whole.

~~~
flohofwoe
Well, it's just about how much max memory you need at a time, no matter
whether that size is split into many small chunks or allocated as a single
block. True, an application which needs very little memory most of the time,
and spikes to huge sizes from time to time is not optimal with emscripten's
approach.

But I don't want to nitpick all day. The more options we have the better, keep
up the good work :)

------
nonchalance
> reducing the limitations and obstacles given by Javascript

What are these perceived limitations? There are fairly standard idioms and
pure JS libraries for getting around most problems people find

------
mercurial
I'm not sure why you would want something like that, unless you really have a
lot of existing code C++ business code you want to reuse, or you really love
C++.

~~~
kirab
I really love C++ and I bet with the currently developing standard (C++11,
C++14) more people will start loving C++ again :-)

~~~
betterunix
"I bet with the currently developing standard (C++11, C++14) more people will
start loving C++ again"

I doubt it. C++11 takes C++ even further from where it needs to be. Every
high-level feature is bogged down by low-level annoyances. Do you capture
variables by value or by reference? Do you need a weak pointer to break this
cycle, or can you use a weak pointer somewhere else (or maybe just not both
with smart pointers at all?)? Your destructor has an error to report -- now
what?

The reason nobody likes C++ is that it is overly complicated for low-level
programming and too low-level and annoying for high-level programming. At best
the only thing people can say about C++ is that it is "good enough" and
popular.

~~~
vinkelhake
Pray tell us, where does C++ need to be?

All C++ programmers who I have talked to welcome most things in C++11. It
simplifies their C++-code in many ways. As an example: specifying how to
capture variables was something they were already dealing with when writing
functors manually, now they have a shorthand syntax for it.

As for the reason why non-C++ programmers don't like C++, perhaps you are in a
better position to say.

~~~
betterunix
"Pray tell us, where does C++ need to be?"

In my honest opinion, C++ needs to separate low level concerns from high level
constructs. _How_ variables are captured in lexical closures is a very low-
level concern, but lexical closures are a high-level construct -- by mixing in
such low-level details, the overall utility of closures in C++ is reduced.
This is made even worse by the fact that the programmer must manually maintain
the lexical environment of the closure -- and the new "smart pointers" are
only marginally helpful, not nearly as robust as a modern garbage collector
(which can, oddly enough, _improve_ performance).

C++ could be a really great language if these things were better separated. I
write my code in Lisp, and when I have low-level needs I usually have to write
some C code and use an FFI. It gets the job done but it is _awful_ to debug
and there is a performance hit. C++ is in a much better position: you can
write low-level code, or you can write high-level code, or both, and no
language barriers need to be crossed (reducing costs in various places).

"As for the reason why non-C++ programmers don't like C++, perhaps you are in
a better position to say."

Well, aside from what I mentioned above, I can give you one more: extending
the language. You have a few narrow ways to do it, and then you are just
stuck. Operator overloading is great, but what if you want to add a new
operator? What if you want to change how your type checker works, so that you
can have a dependent type system? There are a few hacks that people can do
with templates, but on the whole adding a new feature basically means
rewriting your compiler (see e.g. what it took to get support for lexical
closures), and C++ compilers are pretty hard to write in the first place. This
is something of a Lisper gripe, though there are a few languages beyond Lisp
that allow programmers to add new syntax or semantics.

~~~
vinkelhake
Ownership is something that is central in C++. Programmers who come from
languages where they can rely on a garbage collector probably find it odd that
they would have to specify how variables are captured. For a C++ programmer,
it is natural to think about who owns something and in which scope it lives.

To generalize a bit: a lot of the things that might seem odd to non
C++-programmers are perfectly natural to C++-programmers.

It's not often that I hear that C++ is too narrow. Being able to add new
operators and changing the way the type checker works both seem like niche
things and something that is more fit for a research language.

------
nrser
i think you're out of memories brah:

    
    
        Fatal error: Out of memory (allocated 21757952) (tried to allocate 7680 bytes) in /var/www/techblog/wp-content/plugins/disqus-comment-system/lib/api/disqus/url.php on line 55

------
angersock
It's a slow morning, so this was good for a chuckle:

 _" Bring the robust­ness and proven scal­a­bil­ity of C++ pro­gram­ming to
the Web "_

Ha. Ha ha. Ha ha ha.

~~~
okamiueru
A bit unrelated, but does anyone know at what comment score one gets to
downvote non-constructive comments, or is that what the flagging is for?

~~~
angersock
Downvoting comes around 500 maybe? Not actually sure.

Anyways, lest you think my observation is just a "non-constructive comment":

The great thing about the browser is that it is free of a lot of garbage we
have to deal with in C++. The antics required to make C++ work in the browser,
while laudable, are absurd.

If you need cross-platform code, write in Java and avoid native extensions.

If you need cross-platform code that has to be _fast_ (i.e., you actually need
to have inline assembly), write in C or C++ and actually do your proper cross-
platform work (proper abstraction of I/O, proper defines, proper selection of
types, etc.).

If you need to draw shit in the browser, use the wonderful ecosystem of tools
that have evolved using the standard affordances of Javascript and CSS.

Here are the bits I take issue with:

 _" write web appli­ca­tions in C++, reusing exist­ing code and mak­ing
port­ing of whole appli­ca­tions and games to the browser plausible."_

Why? What type of code is worth reusing? What type of performance is this
going to get me?

Most C++ codebases are janky and sad--game engines being some of the worst
offenders.

 _" code both the fron­tend and the back­end of a web appli­ca­tion in the
same lan­guage and codebase "_

Why the _fuck_ would you ever want to use C++ as that language? It's terrible,
so much so in fact that it lost out to PHP and Perl.

Let that sink in for a second.

People said, "I would really like a language that's easier to work in!" and
then picked PHP and Perl. _PHP and Perl_.

Moreover, go look at their docs:

[http://www.leaningtech.com/duetto/examples/](http://www.leaningtech.com/duetto/examples/)

 _" Duetto does not attempt at emulating numerical types which are not
supported by JavaScript float is supported with 64-bit precision, like double.
long long is defined as a 32-bit integer. 64-bit integers are not supported at
all in JavaScript and duetto does not attempt to emulate them."_

Well thank Christ we never will need numerical types not in Javascript--I
mean, I've never used bit-twiddling in my engine code, or my decompression
code, or my audio code.

~

Look, it's a really cool use of LLVM, but we don't need C++ in the browser.

~~~
flohofwoe
Have you ever had to maintain a code base with more then a few hundred
thousand lines of code across Flash, Javascript, iOS, Android, PC, OSX, Xbox,
PS, Wii, Vita, NintendoDS, ...and whatever other gaming platform is popular at
the moment? The "popular platform of the day" changes a lot faster then being
able to write new code for it. You need to support that new platform within 3
months or less. Good luck rewriting your entire code base into another
language in that time ;)

~~~
tomkludy
Frankly, I've had to maintain a huge C++-only codebase, and that's why I hope
to never write another app in C++.

Yours is an argument for a high-level, typesafe, compiled-to-native and
compiled-to-javascript language. But IMO the argument falls down when applied
to C++, because it is not high-level, it is only marginally typesafe, and it
doesn't compile to javascript well at all (tradeoff of either massive
performance, or missing basic functionality like 64 bit ints).

It is very possible that no language will fit this role exactly for some time
to come. But, to me at least, it is very clear that C++ is NOT the language
for the job.

~~~
flohofwoe
Yes I see your point to a degree, in a way we're working in a sort of
"highlevel C++" most of the time at some performance cost: We have strict
coding conventions in place which forbid low-level C/C++ stuff in _high-level
code_ (no raw pointers, no C-style arrays, no pointer arithmetics, no C
library functions, etc...). Plus static code analysis and tons of runtime
asserts. I can't remember the last time we had a buffer overflow or pointer-
gone-wild. The performance hit of C++ compiled to JS is surprisingly small
(1.5x native performance in Firefox, a bit slower in Chrome, but the gap is
getting smaller, this is in the same ballpark as strongly typed bytecode
languages like C# or Java).

