ExpressionEngine had a lot of great ideas in it, but its code was an absolute a mess, and its leaders had no understanding of the concept of technical debt. They just kept racking it up until, by the time I joined in 2012, any changes were expected to take months because the code was a nest of interdependent singletons. While I was there, we spent a lot of time basically responding to the moves of P&T.
If you weren't part of the ExpressionEngine community, then you might not know that P&T started out making paid add-ons for ExpressionEngine. I might be misremembering, but I think relationships and matrix were their main ones. One of the first projects I was given as a new engineer at EllisLab was essentially making an native relationship field for ExpressionEngine in response to P&T. If I remember correctly, we knew Craft was coming and were trying to undercut P&T even though they were one of EL's most popular plugin makers.
We ended up taking on a rewrite for EE3, because the techdebt was that bad and refactoring wouldn't have got us where we needed to go in time. In retrospect, I'm not sure if that was the right choice, even though I was one of the strongest voices in EL arguing for it at the time. I definitely thought it was the right choice at the time, but I was also a really inexperienced engineer. I don't know if there were any right choices though. Thinking back, I have a hard time imagining a path forward that didn't involve some level of rewrite. It's one of those experiences you can go round and round in your head about wondering whether you did/argued for the right things. In the end, I think EL's fate was already sealed by how much techdebt they'd racked up.
The rewrite took a solid two years, Craft was released first, and was better than EE2 in every way (and EE3 if you ask me), and thus began the death spiral. EllisLab is no more (though the rewritten ExpressionEngine lives on as an open source project maintained by one of the agencies that used it), and Craft is clearly still around.
My memory is that we didn't really know what those things might be and had no real way to find them out. But I could easily be misremembering.
It's one of the things I really appreciate about my current role - we have sales, marketing, product, and design functions responsible for figuring out what to build (which we offer input into), but largely, in engineering, we can trust they're doing a good job and just figure out how to build it. We definitely still have our techdebt, but we manage it very intentionally - where that sometimes means intentionally leaving it in place and adding to the ball of duct tape because we can't unwind it now. But it's never reached the point of becoming systematic. Our worst techdebt is tucked into corners where the unintended consequences caused by touching it can't reach too far.
I feel like it's a really good balance, take on techdebt intentionally, don't let it get systematic, and pay it off intentionally when you can.
With EllisLab, a more piecemeal approach very well may have been the right one. How do you think we should have approached it? EE was so tightly coupled through those singletons at the time - maybe singleton wrappers around subsystems, taking one at a time as features touched them? What features do you think we should have focused on? I'd be really interested in hearing more of your perspective on EllisLab before and post Craft. You lasted a lot longer than I did and probably have a lot more insight and thoughts!
Haha yeah this is basically what happened. I don't want to get too far into the weeds on a public forum, but in general, the suggestion of market research was initially met with resistance. I think they came around to the idea eventually but it was too late.
> I feel like it's a really good balance, take on techdebt intentionally, don't let it get systematic, and pay it off intentionally when you can.
That's a great summary. Stop the systems that create the debt by learning to separate concerns or discouraging other anti-patterns, and only take out debt if the trade-offs make sense and hopefully have a plan to pay it off.
> With EllisLab, a more piecemeal approach very well may have been the right one. How do you think we should have approached it?
I think were going in the right direction with the Relationships parser and later the new conditionals parser, they were very object-oriented and easily testable, and weren't too coupled with persistence or presentation concerns like the typical patterns of just putting everything in a library or controller, but not all of us were skilled enough to keep doing that. I feel like later on we naturally started using patterns like you see in Working Effectively with Legacy Code in order to keep newer code testable and easier to maintain, and then plugging those things into legacy where they were needed. These days, my bias is to fit things into Hexagonal Architecture so if it were me now, I would try to decompose the various major areas into loosely-coupled modules bound by well-defined interfaces, and the internals of those modules would be highly cohesive. You can kind of work towards this piecemeal as you work on various features but there's an awkward in between phase that can last for years on complex domains.
> EE was so tightly coupled through those singletons at the time - maybe singleton wrappers around subsystems, taking one at a time as features touched them?
Yeah maybe, we eventually made a solution for the `$db` singleton by making it so you could just ask for a new instance every time, that way no other code could mess with your in-progress query-building. I'd bet most of the classes that were singletons didn't have to be that way and we could just make new instances when we needed them. They were probably only singletons because it was convenient access and looked magic in CodeIgniter, and most of us were so green that we didn't know better.
> What features do you think we should have focused on?
Probably the things people asked for for years that Craft launched with and Packet Tide is now adding. We did come around to things like Live Preview and Fluid fields there towards the end but like I mentioned earlier, at a certain point it was too late. The market had gotten away from us and the community was fractured.
> I'd be really interested in hearing more of your perspective on EllisLab before and post Craft. You lasted a lot longer than I did and probably have a lot more insight and thoughts!
Sure feel free to reach out on Facebook if you want, I think we're still connected here. Yes I was there for too long.
From the outside, there were some weird vibes coming out of EL back then – they’d taken a bit of an adversarial tone with the community, which had us wondering whether it was a good idea to have all our eggs in the EE basket. Then an EL designer publicly ranted about P&T, calling us a “direct competitor”, on the EE Podcast, when Craft was just a twinkle in our eye. Which ironically, was the exact moment we decided to really commit to building a CMS. Self-fulfilling rant I guess :)
That podcast was either before my time or maybe just when I was starting, but yeah, that was definitely the mentality when I was there. There was also a bit of a siege mentality. My onboarding involved being instructed to join twitter, but warned that the community would be mean to me. Thing was, I found most of the criticism coming from the community to be warranted and valid and thought we should have been listening to it, instead of writing it off as a loud, mean, disgruntled, few.
Either way, you guys did a great job with Craft!