
C++ – From goto to std::transform - jlemoine
https://github.com/Dobiasd/articles/blob/master/from_goto_to_std-transform.md
======
exDM69
What I find painful about writing pseudo-functional C++ using <algorithm> and
other tricks is that you have to allocate storage for intermediate values.
This makes C++ algorithms a lot less composable than similar things in other
languages.

So if I wanted to do something like (in Haskell):

    
    
        sumOfOddSquares xs = sum . map (^2) . filter odd $ xs
    

In C++, I would need to allocate temporary storage for the intermediate
values.

    
    
        std::vector odd_numbers;
        std::copy_if(xs.begin(), xs.end(), back_inserter(odd_numbers), isOdd);
        std::transform(odd_numbers.begin(), odd_numbers.end(), odd_numbers.begin(), square);
        std::accumulate(odd_numbers.begin(), odd_numbers.end(), 0, plus);
    

This becomes quite painful very quickly. You can use solutions like
boost::filter_iterator, but soon you're knee deep in C++ templates and the
long compile times and awful error messages that follow.

The example above is a bit contrived since it can be easily be done with a
single accumulate, but you get the idea...

    
    
        std::accumulate(xs.begin(), xs.end(), 0,
            [](int sum, int i) { return sum + ((i&1) ? i*i : 0); });
    

So while you can have lots of fun with <algorithm>, it's quite limited in what
it can do.

~~~
humanrebar
On the other hand, C++ has more flexibility. Since C++ allows you to specify
your allocation scheme, you can guarantee a piece of code will never trigger
an allocation, allocate in a special way to ensure cache coherence, etc.

For example, you could have written equivalent logic that allocates
odd_numbers on the stack or from a special memory pool.

Sure, application developers shouldn't care about that kind of fine-grained
control. However, they should care about having fast, small, safe, and high-
quality libraries available. For library authors and developers in special
domains (high-performance, safety-critical, etc.), these tools are essential.

~~~
exDM69
> On the other hand, C++ has more flexibility. Since C++ allows you to specify
> your allocation scheme, you can guarantee a piece of code will never trigger
> an allocation, allocate in a special way to ensure cache coherence, etc.

This is true but the way C++ standard containers work with allocators is
rather clumsy. So clumsy that often when custom allocation logic is required,
standard containers are not used at all and custom containers are built
instead. For example, EA has used a custom STL-esque container library to
avoid memory allocation pressure in console and PC games.

> For example, you could have written equivalent logic that allocates
> odd_numbers on the stack or from a special memory pool.

But this example does not require any allocation space at all. But there's no
way of doing this with standard C++ algorithms without allocating additional
space.

~~~
humanrebar
> But this example does not require any allocation space at all.

That's not possible if you're comparing algorithms apples-to-apples. If you
have an intermediate collection, it's in memory somewhere, even if the
language does the allocation for you transparently.

~~~
exDM69
> If you have an intermediate collection, it's in memory somewhere, even if
> the language does the allocation for you transparently.

This toy example algorithm, and many practical algorithms, do not need an
intermediate collection to be stored in memory.

Using some kind of laziness, you only need exactly one temporary element, not
the whole collection. You can make this happen not only in Haskell but also
using Python generators, Clojure lazy sequences or even in C++ using Folly.

The lack of means of composition, the inability to pipe the output of one
algorithm as the input of another, is in my opinion the greatest shortcoming
of C++ <algorithm>.

------
nly
<algorithm> is so 90s. In C++14 you'll be able to do this:

    
    
        auto square_copy = [](auto v) {
           for (auto& e: v) { e *= e; }
           return v;
        };
    

Another option:

    
    
        auto square = [](auto& v) { v *= v; };
    
        auto map = [](auto range, auto fun) {
            transform (begin(range), end(range), fun); 
            return range;
        };
    
        vector<int> vec = {1,2,3,4,5,6};
        auto squared = map (vec, square);
    

Note that, in both of these cases, if you use std::move() to gift a container
to these functions, they will not actually allocate any memory and will
effectively operate in-place.

~~~
stormbrew
Am I missing something? I don't see anything beyond C++11 in here.

~~~
nly
Parameters can't be auto in C++11. Polymorphic lambdas are a C++14 feature.

Incidentally, are you Stormhawk on Freenodes #C++ ?

~~~
stormbrew
Ah right. I have to admit I was so astounded this was omitted in C++11 (with
some really dumb rationale, iirc) that I keep forgetting it wasn't.

And nope, that's not me. I go by stormbrew on freenode as well and haven't
been in a C++ focused channel since the late 90s on efnet. :)

~~~
nly
Not quite as astounding as omitting make_unique :P No matter, C++14 is going
to be here _very_ soon. Both of the open source compilers are almost there,
and I've read the standard will be final once the Ts are crossed.

------
ajtulloch
Using Facebook's `folly::gen` library [1], you can get it down to:

    
    
        vector<int> squareVec3(const vector<int>& v) {
          return from(v) | mapped([](int i){ return i * i; }) | as<vector>();
        }
    

which is pretty slick. There are a bunch of the usual LINQ/functional
operators in `folly::gen` that are quite nice. As an example, to filter out
the odd elements of `v`, you can do

    
    
        vector<int> squareVecFiltered(const vector<int>& v) {
          return from(v) 
               | filter([](int i){ return i % 2 == 0; })
               | mapped([](int i){ return i * i; })  
               | as<vector>();
        }
    

[https://github.com/facebook/folly/tree/master/folly/gen](https://github.com/facebook/folly/tree/master/folly/gen)

~~~
humanrebar
That's cool, but I'd rather not bitwise-or my functional operators together.
Something like this would be better:

    
    
        vector<int> squareVecFiltered(const vector<int>& v) {
          return from(v
               filter([](int i){ return i % 2 == 0; }),
               mapped([](int i){ return i * i; }),
               as<vector>());
        }

~~~
stormbrew
That is actually really nice.

------
stormbrew
C++ is evolving at a kind of breakneck pace lately, and it's already a lot
different from the language it used to be. Auto, range loops, and lambdas are
making it much more pleasant to work in while still maintaining a lot of
performance advantages.

I know it's pretty popular to hate on C++, but I'm pretty excited about where
it's going, personally.

~~~
spoiler
I've recently started working in C++ (I've had some prior knowledge, but
mostly basics) and whenever I try to Google stuff up, a few of the results are
_Don 't use C++ because of X_

I read a few of them out of curiosity, and realised that they are mostly
religious reasons to hate C++, hardly any of them are proper, factual, or
empirical reasons. So, my hypothesis is the following: They were unable to
learn C++, and because of immaturity, decided it was stupid[0][1].

[0]: I'm no saint to this. I said on numerous occasions Erlang is shit because
I couldn't wrap my head around the syntax.

[1]: I believe it's a phenomenon associated with _Psychological Projection_ ,
but I might've confused it with something else.

~~~
TillE
C++ isn't the easiest language to work with, but what I find truly bizarre are
all the internet people who apparently hate C++ while also praising C as
wonderful and "elegant". I'm convinced that the vast majority of them have
never written anything significant in C, which is a painful experience
tolerated only by systems programmers who actually need something that's one
step above a portable assembly language.

C++ only fails if you try to write it like C, and don't take advantage of all
the lovely new tools it gives you. It's always been a more useful language for
nontrivial projects, and C++11 elevates it so much further.

~~~
GFK_of_xmaspast
I'm a 20 year C programmer finally making the transition to C++ and as far as
I can tell the main advantages C has left are: * Faster compilation *
Comprehensible error messages * No cout

~~~
stormbrew
I have more issue with the poor performance of iostreams than with the
interface, personally. Typesafe and type deducting string formatting is a
pretty nice thing to have.

~~~
nikbackm
Maybe because iostreams are synchronized with stdio by default? It's possible
to disable that for a big speed boost.

There's also the issue that iostreams might be implemented on top of stdio for
simplicity. Hard to make it faster in that case!

~~~
stormbrew
Even then, it's pretty much the only place in the stdlib that extensively uses
virtual calls, via std::locale. It's very complicated implementation-wise (I
was actually working on an implementation of it once and it's not fun at all)
and hard to find places to optimize it.

It's not that it's too heavy for a console app's output, really, but for
things like converting numbers to strings and vice-versa (a-la boost::format)
it is definitely a lot slower than atoi/sprintf.

------
jeorgun
I'm not really sure why references are being used here, if you're just making
a copy straight-away.

    
    
        vector<int> square(vector<int> vcopy)
        {
            for (int &i : vcopy)
                i *= i;
            return vcopy;
        }
    

is shorter, (IMO) much more readable, and on my tests, measurably faster than
the reference version. If you really want the std::accumulate version,

    
    
        vector<int> square(vector<int> vcopy)
        {
            accumulate(begin(vcopy), end(vcopy), begin(vcopy),
                       [](int i) { return i * i; });
            return vcopy;
        }
    

works just as well.

Still, I guess it's beside the point.

~~~
Dobiasd
Awesome! It is more readable and makes it faster. Thank you very much. I
changed my article accordingly.

------
pathikrit
After coding in languages in like Scala/Python for a while, it took me a while
to parse this. Scala equivalent: `def square(a: List[Int]) = a map {i => i*i}`

~~~
douglasheriot
After in coding in languages like C/C++/ObjC for a while, it took me a while
to parse your Scale equivalent.

Personally I find the C++ much clearer – I don’t see either as that much
better/worse.

------
Jare
Convenience overloads for common use cases (iterating over the full vector,
accumulating on a default-constructed accumulator, etc) would be great to have
as standard. Are there and I have missed them? If not, what's the rationale?
(this goes for the STL in general, not specifically these high order algos)

~~~
Negitivefrags
Here is a blog post on this very issue:

[http://herbsutter.com/2011/10/07/why-no-container-based-
algo...](http://herbsutter.com/2011/10/07/why-no-container-based-algorithms/)

~~~
humanrebar
That's a valid technical reason why there isn't a std::sort that takes
containers. It's not a valid reason why there can't be a std::range_sort or
std::range::sort that takes container arguments.

~~~
Jare
If I was doing heavy C++11 today, that kind of range namespace would be the
second thing I'd write; the first being a bunch of string functions for the
operations I actually want to do with strings (as opposed to the low level
"character buffer manipulation primitives" the STL offers).

I was ok with that low level, low semantics design in the STL until I heard
that stuff about integrating a graphics library. Now I'm just confused. :)

------
crawshaw
Readability improved from squareVec1 to squareVec4, then went downhill. Hiding
loops behind new names doesn't make programming better.

~~~
nly
At Going Native last year Sean Parent, from Adobe, gave a superb talk called
'C++ Seasoning' where his number one pro-tip for C++ was: no raw loops.

He made a good case that anywhere you have a non-trivial loop in C++, you're
almost always going to be better off folding the problem in to a composition
of algorithms. The code got more readable, more performant, and redundant code
can often be removed.

Please take an hour to watch it here in glorious high quality:

[http://channel9.msdn.com/Events/GoingNative/2013/Cpp-
Seasoni...](http://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning)

~~~
agumonkey
And if someone has nothing better to do, may I suggest to watch anything by
Sean Parent. Some old adobe talks might be in low-res, but they were still
very interesting, on the abstraction side of things.

~~~
nly
Where can we find these old talks?

~~~
agumonkey
Here's one :

A Possible Future of Software Development :
www.youtube.com/watch?v=4moyKUHApq4‎

------
vinkelhake
Just as a side note: if you're looking to write a convenience wrapper like the
author's transformVec, do not have it take the function as `const
function<T(T)>&`. Instead, do something like:

    
    
        template <typename T, typename F>
        vector<T> transformVec(const vector<T>& v, F op)
    

Using std::function in a case where you just want something callable just
makes the optimizer's job a lot harder. Using std::function is a good idea if
you actually need type erasure (for example if you want to pass something
callable to a function that is _not_ a template).

~~~
Dobiasd
Wow, awesome tip, thanks. This makes the wrapped version as fast as the
others. I updated my article accordingly.

------
Marat_Dukhan
The author probably thinks that all versions are equally fast, but in fact
they are just equally slow. Neither gcc 4.8 nor icc 14 can use vectorization
for this loop (likely because of aliasing).

~~~
pbsd
It's not because of aliasing, it's because of the push_back call inside the
inner loop, which may potentially have to allocate (in this case we know it
doesn't, but the compiler does not). If you replace squareVec6 with

    
    
        vector<int> squareVec6(const vector<int>& v)
        {
            vector<int> result(v.size()); // preallocate
            // result.reserve(v.size());
            transform(begin(v), end(v), begin(result), [](int i)
            {
                return i*i;
            });
            return result;
        }
    

the compiler will be happy to vectorize the loop.

------
heliumcraft

      def square(vector)
        vector.collect { |i| i**2 }
      end

~~~
milliams
I like the Python version of:

    
    
      def square(vector):
        return [v**2 for v in vector]

~~~
ptr
Haskell:

    
    
      let square = map (^ 2)
    

_Real_ partial application and currying is something I'm longing for in other
languages; it's easy to fall for the beauty of the Point Free Style.

~~~
icebraining
Well, in Python that would be:

    
    
      from functools import partial
      square = partial(map, lambda i: i**2)
    

EDIT: I guess it's still not the same, since the lambda isn't a partial
application. But I don't think we can just set the exponent in pow(), we need
to create a function that takes the arguments in the reverse order. Maybe:

    
    
      from functools import wraps, partial
      def flip(func):
          'Create a new function from the original with the arguments reversed'
          @wraps(func)
          def newfunc(*args):
              return func(*args[::-1])
          return newfunc
    
      _pow = flip(pow) #Take Y, then X
    
      # Then the equivalent would be
      square = partial(map, partial(_pow, 2)
    

Of course, flip is rather long-winded because it's a generic function, one
could just do:

    
    
      _pow = lambda y, x: pow(x, y)

------
u124556
`result = [i * i for i in v]` looks much more readable to me.

What if you need to square every other element? copy_if and transform then
don't look so pretty anymore. `result = [i * i for i in v if i % 2 == 0]` is
still more readable.

What if you need to do something else and you don't know if there is a
transform like function for it?

~~~
alecbenzer
> What if you need to do something else and you don't know if there is a
> transform like function for it?

What about the same thing in python? As far as I know the two examples you
gave exhausted list comprehensions' functionality.

------
geocar
It seems like it's the loop itself that is redundant. Why couldn't I just do:

    
    
        squaredVec = vec * vec
    

or

    
    
        squaredVec = vec ^ 2

~~~
bstamour
You can, but not with vector. Use valarray if you need to perform those kinds
of computations.

------
ww2
The example is too trivial. you could have 100 ways to write "hello world",
but it does not matter.

------
GFK_of_xmaspast
Why were all the examples using iterators, and not square bracket indexing?

------
mytummyhertz
am i the only one who thinks the std::transform version is LESS readable than
the for loop?

------
shultays

      but you still have to read the whole line
    

Oh the pain!

