Since then, Rakudo Perl 6 has become more than 100x as fast, at least for this test. And more than 250x as fast using a concurrent version of the test (which was as simple as putting a `.race` in the code).
This makes it still slower than the Perl 5 version, which uses XS (all parsing logic written in C). But the gap is closing with another round of optimizations to be merged later this week after an extensive refactor that took about 2 months. Of which you will most certainly see a report on https://6guts.wordpress.com , the blog of Jonathan Worthington.
PS: You might read this and be reasonably surprised that Rakudo Perl 6 is not, after all this, very fast yet. I have a - not entirely serious - explanation for that:
All problems in computer science can be solved with a layer of indirection.
Many layers of indirection make programs slow.
Perl 6 solves many computer science problems for you ;-)
In the future, we'll continue to solve those problems, just faster.
My guess would be that the strings had to be copied more often in the C/C++ code.
So it may be that the rate at which it gets faster has gotten faster.