In 2000, there was an attempt to settle some of the worst fighting among the porters, and there are suspicions that the launch of Perl 6 aka Raku at the same conference was not unrelated.
I stopped following the community so closely soon after that, but it looked like the fighting mostly moved over to the Raku side after that, but continued unabated in vehemence.
Maybe it's all those %^&% punctuation marks that Perl programmers are already habituated to cursing?
I think the fight was mostly between people who wanted the Perl 6 project dropped, and the ones working on the Perl 6 project.
In any case, since the rename to Raku, we have seen some disagreements in the community, but these have been solved (although in one case banning from the core has been necessary, unfortunately).
Up until we weren't, but that was basically the point where the P6 language designers said they wanted to write their own VM anyway.
It was not whether the people were talking, but whether the components grew in convergent directions. My impression was that for extended periods they did not. Am I wrong?
1. Parrot was initially intended to be the backend for Perl 6.
2. Perl 6 didn't materialize fast enough, so Parrot started focussing on hosting other scripting languages.
3. Parrot decided to become the backend for all scripting languages.
4. Parrot became less and less targeted towards Perl 6.
5. Parrot never really became a backend for any other scripting language.
6. Parrot became decicedly a poor fit for Perl 6.
7. Jonathan decided that maybe the JVM could also be a backend, and started porting
8. Building on that experience, Jonathan decided that they could write a dedicated backend for Perl 6, which became MoarVM
9. Parrot development dwindled to effectively 0.
10. During the Great List Refactor of 2015 (about 6 months before the first release) it became clear that porting the Parrot backend would be a lot of additional work, with no apparent upside
11. Parrot support was dropped before the first official release
That's why Dan and Guido had a bet that Parrot would run Python faster than CPython, and that's why Parrot attracted contributors like Sam Ruby.
It's definitely true that, as P6 dragged on and burned out a lot of contributors (including multiple project managers), many Parrot contributors thought that coupling Parrot's success to P6 was a mistake, but I can't see that as a change in focus because the multi-language focus was a goal from the start.
Granted, Parrot did wander in the wilderness for a while waiting for the P6 design process to proceed, and Parrot did evolve in some unfortunate directions because of that. The rest of this comment covers 2007 onwards though.
Your question about components is particularly interesting, because of what really happened.
A couple of P6 developers wrote PCT, a set of tools and libraries to make developing compilers and languages on Parrot easier. This was a good thing, and Parrot users and developers started migrating their language implementations to that toolkit.
The PCT developers refactored and revised this a couple of times, and to make that easier, eventually they took the code out of the Parrot tree and started syncing PCT into the Parrot tree on a semi-regular basis.
I was working with them weekly to fix Parrot bugs, improve Parrot performance, and add Parrot features to reduce the amount of code necessary in PCT. Thus it's difficult for me to believe that the Parrot developers were designing and implementing Parrot away from P6.
Toward the end of 2010, the P6/PCT developers said they were going to rewrite PCT to support multiple VM backends. I argued this would create a lot of churn in the language implementations actively using PCT (ironic, because the PCT developers spent a lot of time before and after this complaining bitterly that they couldn't keep up with Parrot churn) and would provide very little value for Parrot (PCT was out of tree already, and syncing in code designed explicitly to undercut Parrot seemed like a stupid idea). These discussions all took place in the public Parrot IRC planning sessions, which were logged.
I also had concerns about yet another rewrite undercutting P6, especially coming six months after the underwhelming Rakudo Star release, but it was clear which direction the winds were blowing, and that's when I stopped working on both P6 and Parrot.
Five years later, P6 had its first official release and Parrot was well buried.
I know this contradicts the official Raku hagiography, but I've gone back and read the primary sources a few times over the years as people have asked about it, and I'm confident the record backs up what I've said.
I just know second-hand that something broke the trust between the Perl 6 and Parrot developers around the first Rakudo Star release. And that strengthened the notion that another backend for Perl 6 was needed.
> I know this contradicts the official Raku hagiography
OOC, which official hagiography are you referring to?
Also, I find the use of "hagiography" in relation to Raku a bit strange: if anything, with the rename, the last saint has basically left the building?
I'm in roughly the same boat. I too dipped in and out starting in 2000, and had been reading contemporary discussions on and off from around 2009.
I want to draw your attention to some things I know due to publicly available records, things I think shed light on what really happened.
The first thing is that Rakudo began technically shifting to a multiple backend architecture in mid 2009:
* Following Jesse Vincent's summer 2009 discussion with Patrick Michaud about Perl 6 running on JVM and .NET, NQP was rewritten in the fall of 2009 to support multiple backends.
* jnthn edited his website to say his first interest in working on Perl 6 going forward was "Transforming Rakudo from a single backend compiler to one capable of targeting multiple backends".
> I just know second-hand that something broke the trust between the Perl 6 and Parrot developers around the first Rakudo Star release. And that strengthened the notion that another backend for Perl 6 was needed.
As explained above, Patrick had already focused his and jnthn's efforts on a multiple backend Rakudo in mid 2009, a year before Rakudo Star.
And, as chromatic has said, and the record shows, commits to Parrot "fell off a cliff" in early 2011, about 6 months after Rakudo Star, and a year before the dates of the first commit timestamps in MoarVM's git history, in Q1 2012.
At the time of writing this comment:
Use new nqp::time instead of nqp::time_(i|n)
20 days ago
[JVM] Add op 'nativecallinvoke'
15 days ago
Disallow explicity specifying op write registers
8 days ago
In fact, it's amazing how that fourth rewrite of NQP ended up not working at all for any of the backends Patrick showed on his slide. Or at least not working yet.
the record shows, commits to Parrot "fell off a cliff" in early 2011, about 6 months after Rakudo Star
How strange that that happened just after the developer summit where I said that the NQP strategy was going to leave Parrot developers in a dead end. Imagine that -- people don't want to stick around on a project that's continually undercut by bad technical decisions.
Thanks for finally confirming what I've been saying for years.
If Parrot had provided all that Perl 6 needed at that time, I don't think the multiple backend strategy would have been selected. Why would one?
> where I said that the NQP strategy was going to leave Parrot developers in a dead end
To me that feels like a self-fulfilling prophecy. Also, had Parrot been able to attract other languages to a significant extent, the abandoning of Parrot by Perl 6 would not have been an issue.
Personally, I think Parrot was not able to attract other languages because:
1. it didn't provide enough benefits to account for the extra effort needed by language developers. It definitely did not for Perl 6 at the time.
2. it didn't have a core team welcoming enough to keep anybody vaguely interested in Parrot
In the end, it is always about being able to create a community in open source. Without it, a project is indeed on a dead end.
Given that the multiple backend strategy only succeeded in replacing an existing backend with another backend several years later without achieving any of the interoperability goals of the JVM or CLR, I'm not even sure how to speculate on the reasoning behind that decision.
Also, had Parrot been able to attract other languages to a significant extent, the abandoning of Parrot by Perl 6 would not have been an issue.
You haven't addressed my point that NQP yanking the rug out from most other languages running on Parrot harmed Parrot. That seems like a material concern.
it didn't have a core team welcoming enough to keep anybody vaguely interested in Parrot
You haven't addressed my point that I personally spent multiple months offering to make Parrot better fit NQP/PCT for P6 and was repeatedly rebuffed.
If I remember correctly, the JVM port predates the existence of MoarVM.
Granted the JVM port has only ever been a second tier backend. Partially because it only supports Java's level of Unicode support.
> You haven't addressed my point that I personally spent multiple months offering to make Parrot better fit NQP/PCT for P6 and was repeatedly rebuffed.
Yeah, because that really wasn't the main problem.
The main problem with Parrot was that the intermediate NQP/PCT code supporting it needed a complete rewrite.
The intermediate code for supporting MoarVM, JVM, and JS are all very similar. They are also significantly different to how Parrot was handled.
There is relatively little backend specific code in Rakudo itself for the three current ones. There was a lot of backend specific code in Rakudo for Parrot. Instead the vagaries of the different backends is mainly handled by NQP.
That means that if you made Parrot work better the way Rakudo-on-NQP-on-Parrot was at the time, it would have been a wasted effort.
Imagine the effort you were talking about doing was like tunnelling through a mountain.
The problem is that Rakudo-on-NQP-on-Parrot was going around the mountain instead.
Sure you would have reduced the total length. The problem is that you would have to create a new tunnel in a different direction once Rakudo-on-NQP-on-Parrot instead started tunnelling straight through the mountain to meet you.
Pretty sure that's why they told you to not do that.
First, that wasn't my plan. We were actively working on the Lorito plan to reduce the total amount of C in Parrot and eliminate the need to write C to extend Parrot, just as you suggested needed to happen in another reply.
Second, it would have been nice to sit down and come up with a plan, rather than Jonathan and Patrick deciding all of this on their own, not telling Parrot developers, and then spending years complaining that Parrot developers weren't responsive to their needs even after their antics had chased away pretty much all Parrot developers.
Granted the JVM port has only ever been a second tier backend.
Ah, so frittering away a bunch of developer time and developers helped them halfway reach a goal years later.
I'd say, it turned out to be more difficult than expected?
I guess I don't understand how NQP would be yanking the rug out from other languages, technically speaking? That it could be seen as a vote of no-confidence: well such is life in the open source world.
Not having been around at that time, I don't think it is up to me to address that point. If you were rebuffed, I can only surmise that interpersonal and technical relationships had already soured enough not to trust the validity of your offer. Note: I'm not saying that the intent of your offer was questioned.
If I recall correctly, that's about the time Jonathan started complaining that the Parrot object model didn't work very well for P6 and wrote his own for PCT, which is still weird to me because he was the primary implementer of the Parrot object model in Parrot. (I don't remember the dates, but he also wrote an object model in C# and another in Java, which you mentioned in another comment.)
I tried for a long time to migrate the PCT object model back into Parrot, but both Jonathan and Patrick told me repeatedly it wasn't a priority, so I stopped offering.
I'm not the only person who had this experience; I remember someone else having a similar experience with Unicode support.
The one that says "Parrot dumped P6, so the P6 developers had no choice but to write their own VM".
I was there. That's not true at all.
Different eras with different dysfunctions, and different people involved, but for some reason, always personality clashes.
I don't think that I ever implied that. To me, in one sentence, it's more like "Parrot and Perl 6 diverged, so that Parrot was no longer a good fit for Perl 6, so the Perl 6 developers had no choice but to write their own VM"
To me this reads like the "mistakes were made" phrasing used when someone wants to be technically accurate but not say anything meaningful.
Regardless of whatever Raku developers and advocates say, I, as a lead developer of Parrot, went out of my way to make it work better for P6. Patrick asked me not to do certain things and so I didn't do those things.
Maybe that was my mistake.
But it's categorically untrue that Parrot wanted to get rid of P6 or considered P6 an afterthought or wasn't willing and able to be receptive to P6's needs, and I'm disappointed that that untruth continues to persist.
May I point out that basically nobody of the then Perl 6 team (except Jonathan) is currently involved in the development of Raku? So please don't transfer your mistrust to these people: they have no beef in this.
Well, maybe I don't have anything meaningful to say about this, as I have no firsthand experience in the matter. So the only thing I can do in this case, is trying to be technically correct.
I believe you had the best intentions towards Parrot and towards Perl 6 (which for some reason you keep mentioning as P6 in your replies). But, for whatever reason, it did not work out. We can keep discussing the past in that respect, but I'd rather work on the future instead.
If you really want to get the history straight, I'd suggest to write a book about it, similar to http://friendlyorangeglow.com
which taught me a lot about the people developing the system I grew up with. Good and bad.
The Parrot backend supporting code was significantly different.
So much so that to properly support Rakudo on Parrot would require a rewrite of it.
There are various reasons for that, but the main takeaway was that continuing to support Parrot made everything significantly more difficult.
Parrot made a lot of assumptions about how objects worked.
It knew simultaneously too much about how objects worked, and not enough about how objects worked. It basically assumed that the Meta object protocol would be written in C for every object type.
MoarVM doesn't know how objects work. It gets told how they work everytime it runs. (Necessary since Raku can mutate its own object system.)
Rakudo on Parrot was slow. If you happened to use a Unicode character in your source file it got dog-slow.
Worse yet, Unicode was an optional feature of Parrot! WTF!!
That's not good backend for a language that has excellent Unicode support.
Not only did the NQP intermediate code for supporting Parrot need to be rewritten from scratch. A large portion of Parrot also needed significant work.
The main idea of Parrot was that there would be integers, numbers, strings, and other. It turns out that other was really the main category. And again Parrot was opinionated about what other even was.
It was thought that if something like integer bytecode was similar to machine-code that it would be easier to optimize it.
The problem is that something as simple as calling a function involves taking a Signature object and asking it if it accepts a given Capture object. Since even just adding two values together is a function call; doing objects fast is more important than special casing integers.
MoarVM is basically the other category first and foremost.
By that I mean if you took out the special-casing of integers, numbers, and strings from Parrot. Then redesigned the PMC code to support what Rakudo really needed, you would end up with something that is very similar to MoarVM.
The reason you might take that special-casing out is that making bytecode similar to machine-code was completely and utterly pointless. At least as far as Raku is concerned.
It doesn't matter that much if integer operations are fast if you are only ever going to run one in a row.
The reason that MoarVM isn't extremely slow is that it knows enough about objects to be able to pull them apart to only run the parts that need to be run.
I think that Parrot as initially designed may have been a really good fit for a Perl5+, Ruby, Python, or Lua etc; but it turns out that it really wasn't a good fit for what Perl6 eventually became.
Let's do a hypothetical thought experiment.
Imagine if someone did all of the work to make Parrot work again the day after it was dropped. That includes rewriting the middleware and doing enough to make the PMC part usable. So then it would have continued to work over the years.
If the Unicode support of Parrot got better and faster I think it might be more likely to tied for 2nd.
For it to beat, or even meet MoarVM for first; it would end up needing so many changes that I'm not sure that it would even resemble what it once was.
I would have liked for Parrot to survive and be one of the backends that Rakudo runs on.
But dropping it was the correct move.
This is nonsense, and I say that as the person who spent months profiling Rakudo to figure out why it was slow.
Rakudo's parser was (maybe still is) slow because it can't optimize anything, even the <ws> token.
Adding NFG could have helped by allowing fixed-width access to normalized codepoints, but IIRC Patrick told the person who was going to implement NFG not to do it.
Seems like a pattern.
In which case I was right.
> So much so that to properly support Rakudo on Parrot would require a rewrite of [the middleware].
Which also means that most of the work you would have done to Parrot would needed to be reworked again afterwards.
There was `ifdefs` all over the NQP and Rakudo codebases to work around Parrot's differences. Which was annoying and error-prone.
The `ifdefs` are now mostly in NQP. And even those tend to be constrained a bit.
> Rakudo's parser was (maybe still is) slow because it can't optimize anything, even the <ws> token.
That is factually incorrect. There are several known optimizations that have not been implemented yet. One of which was even in STD.
Also since Raku treats regexes as code, optimizations to regular code paths can also apply to regexes. That includes optimizations to calling methods like using the <ws> token.
The main reason that they haven't been many is that the people that are competent enough and confident enough to do that work have been busy doing other things. Both their dayjobs and other optimizations or design work.
Really as far as I know, there have been next to no attempts to optimize specifically regexes and grammars since they first got to the current feature set. Certainly not in the several years where I was looking over every commit to Rakudo.
There were two options.
1. Create a new object system for Parrot.
2. Create a new object system and a VM around it.
Considering the object system needed is almost nothing like the one that was in Parrot, creating a new VM was probably considerably less work.
Then there is also the problem that some of the major contributors to Parrot didn't care about P6. They have said so publically. I don't think they would have taken too kindly to a completely new object system dumped on their doorsteps. (That's how they might feel about it anyway.)
Honestly thinking about it now, I think that if option 1 had been taken there would have been a fork of Parrot. I can almost guarantee it. And I'm not just talking about a fork of the codebase. There would have been a fork of the people behind it as well. It wouldn't have been a peaceful fork either.
I see all of the anger that you have towards what happened. I can only see it as being significantly worse and bigger if they had created a new object system for Parrot instead.
Creating a new object system, and a new VM to support it, was the correct move.
Especially considering that a number of the fundamental design decisions of Parrot which were completely unnecessary or plainly wrong.
I can only imagine the hell it would be to remove or replace any of the features.
You seem to be the only one still angry, still here, and still talking about it. Most of the other people who would have been angry regardless of which of those two decisions were made have either gone, or are quiet.
If the Parrot object system was replaced instead, a number of those people would still be angry, and still be here, and be loud. They would just be angry for different reasons.
Instead what happened is they got angry, but mostly drifted away with Parrot.
Come to think about it; things seemed to have gotten quieter after that event. Even though progress started going faster.
I would have preferred if they (you) didn't get angry about it. I would have very much preferred if there wasn't a reason to get angry.
The thing is that there would be some number of people would have gotten angry regardless. It's only a matter of who, and for what.
In the reality where Parrot got the new object system instead, I strongly suspect that one of the people who got angry would include jnthn. Since he is responsible for a lot of the progress in recent years, I think I prefer this reality.
I mean I miss the people from back then, but I think problems were piling up, and it was coming to a head at some point.