As Sean, my head SysAdmin wrote on a ticket to you, some of your pages were performing a sort on 188590 rows in memory each time. That just doesn't scale.
From a customer service perspective, this comment leaves a really bad taste in my mouth.
You guys say right on your home page that you're "infinitely scalable." And then if you click through to http://wpengine.com/scale-to-millions-of-hits-a-day-or-hour/ you read that WPEngine features "secret sauce" made of "one of the most scalable WordPress architectures on Earth" that makes you "faster and more scalable all the time." The (strong) implication being that any WordPress configuration will perform better on WPEngine than on an alternative hosting platform.
So if someone comes to you with a WordPress configuration that you can't make "faster and more scalable" by running it on your platform, isn't The Right Thing To Do to say "oops, our bad, sorry we misled you" rather than blaming the customer for having a bad WordPress configuration? Because that's what you're doing here -- blaming the customer.
In fact your comment basically boils down to three parts, only one of which is reassuring:
* Paragraphs 1-4 translate to "we're a growing startup and growing customer support is hard," which frankly if I were a customer I wouldn't care about in the least. Scaling your business is your problem, not my problem. We know it's hard, but it's like Tom Hanks said in A League of Their Own (http://www.youtube.com/watch?v=tPqYnC-SW5w) -- there's no crying in business.
* Paragraphs 5-7 translate to "your web site sucks, no wonder we couldn't scale it, nobody could," which is the blaming the customer part. I don't doubt that that is true (WP can be a real hog, especially if it's hacked up), but if you're advertising yourself as The People Who Can Scale WordPress, you shouldn't be surprised when people start coming to you with funky, hacked-up, hard-to-scale WordPress sites. Who else would someone with a site that can't scale go to but the scaling experts? When that happens it's OK to step back and say "we can't scale this," but it's not their fault for taking your marketing materials at their word.
* It's not till Paragraph 8 that we get to the "let's try to make this right" part, which is where this comment should have started.
I don't say this to be a pain or to make what I'm sure is already a difficult day worse. I'm saying it because I like what you guys are trying to do, and responses like this hurt you as much (if not more) as they help.
I've had to deal with situations like this myself, and no matter how tempting it is to blame the customer -- even when the customer is manifestly at fault -- it never, ever, ever improves the situation. It just makes things worse, first by giving the angry customer something new to be angry about, and second by deflecting you from the more important question of what you can do to prevent the situation from happening again. In this case, for instance, better screening of incoming customers to ensure their sites really can be scaled on the WPEngine platform would filter out problems like this before they become problems. (IIRC you've already started doing this, by blacklisting a small set of known-bad plugins. Maybe that list needs to be beefed up?) But time spent sniping at customers is time not spent making those improvements happen.
No, there's no inference or implication that any (ie regardless of how its built) WordPress configuration will scale on WP Engine. As I wrote in my prior comment, scalability is a partnership between infrastructure and code. That's the case irregardless of framework and PaaS provider.
We like car analogies at WP Engine - if I sell you a racing car and you're only a moderately good driver then you're going to crash when you approach the corner at 200mph. There's little I can do about that other than teach you to drive, and I'm in the business of building the cars. There are plenty of folks who can help you improve your race-craft out there.
To say that we've missled someone is an unfair assertion.
If I analyze the suggestions you're making in your comment here...
* Jacques complained about lack of 24/7 support, but I shouldn't have tried to reply and explain the reason for that because 'there's no crying in business'. ?????
* The WP Engine website shouldn't mention scaling WordPress because it's not technically possible to scale all WordPress code regardless of how it's written. (What should our site say? What should any scale-orientated PaaS website say given the same issues apply to anyone in this space)
* WP Engine should vet and screen it's customers so that they must prove their code can scale before we sell them an account? ?????
I'm sorry, but if that's the advice you're offering then, with respect, I don't agree.
That being said...
No, there's no inference or implication that any WordPress configuration will scale on WP Engine.
I would humbly suggest that your entire Web site is a refutation of this assertion. Even moving beyond the copy on the home page to the fine-print "Common Questions" section of http://wpengine.com/pricing/ , there's no disclaimer or other "are there WordPress configurations that WPEngine can't scale?" fine print.
My point here isn't that you shouldn't promise scaling, it's that it's not hard to see how a mismatch could arise between your expectations (scaling is hard, customer code can make it harder) and customers' expectations (scaling is easy, WPEngine has figured it out).
Jacques complained about lack of 24/7 support, but I shouldn't have tried to reply and explain the reason for that because 'there's no crying in business'. ?????
Yes. If a customer complains about a lack of 24/7 support, you say "I'm sorry, but we're not in a position to provide that level of support at this time." And nothing else. The reason why you're not in a position to provide it is irrelevant to the customer, and risks sidetracking the conversation into a blind alley about the reasons.
It also feels a little bit like pleading for understanding -- "cut us some slack, this stuff is hard!" -- which is off-putting. Of course it's hard, that's why people pay you to do it for them.
WP Engine should vet and screen it's customers so that they must prove their code can scale before we sell them an account? ?????
As I said, you already do some vetting by disabling plugins you know to be performance-killers. Look at this list: http://support.wpengine.com/disallowed-plugins/ , there's an entire category called "CPU/MySQL Thrashing Plugins."
I'm not suggesting that you need to give your customers a full CIA background check before bringing them on board, but you've already moved beyond just taking whatever code customers throw at you. If there's other things that need to be disallowed to ensure that WP sites work on your platform, the thing to do is to find out what those things are and add them to the blacklist.
Again, just my $0.02, IMHO, YMMV, IANAL, etc.
Use some common sense.
"infinite scaling" means _exactly_ that. If you don't want people to use accepted dictionary definitions of words like "infinite", you should not use them in your marketing.
A few telcos here in Australia have recently lost court cases about their use of the word "infinite" in their cellphone plans/marketing, and then cutting off or charging extra to customers who "used too much". The court, rightly in my opinion, said that the meaning of the word "infinite" is clear and well known, and any claim that it included "limited, of course, by common sense" is bogus. Be careful of letting your marketing team write too much hyperbole - occasionally you'll be held legally accountable for it.
I suspect if the amount in dispute had been an order of magnitude or two larger, Jacques may well have had a strong enough case to find a no-win/no-fee lawyer to go into bat for him... I know I'd prefer not to have to explain to a judge how those first two boxes in the first column on the WPEngine homepage didn't cover all of Jacques complaints.
I can create an infinitely scalable server and you can come in and drop a sledge hammer on it...does that mean I shouldn't use the word.
You have to understand the layperson won't know anything about tables etc and stating "infinitely scale" is a HUGE selling point for potential customers. If they wanted to be truthful they wouldn't say that, or they would say we CAN provide infinite scaling IF the following conditions are met.
The way I see it, you are not selling the car, but providing a driver so that I don't have to worry about how it should be driven.
I should sit in the back and relax while you make sure that my car is driving safely and comfortably.
Maybe your message isn't clear enough if you think that your product is the car. From your website:
QUIT WORRYING AND LET US RUN WORDPRESS FOR YOU
QUIT WORRYING AND LET US DRIVE YOU CAR FOR YOU
It seems to me wordpress.com is the "we supply the racing car and driver" solution provider. You don't have to worry about how it gets driven, but you also have only limited say in what car you get and which lines the driver takes through corners.
In my version, WPEngine are providing the race track - to which you bring your own car and driver. They go to a bit of effort to improve the safety, mostly by ensuring other track users don't do too much to affect your performance, but you're free to bring your own F1 car, MotoGP motorcycle, Nascar, or go kart; and run at whatever pace you think is appropriate.
Seems to me the OP's problem was that he wanted to run a top fuel drag rail. From his perspective, he made assumptions about what "a race track" meant that didn't include "decreasing radius downhill hairpins". Meanwhile WPEngine's website/marketing didn't anticipate that class of user, and didn't think they needed to point out that their track isn't an ANDRA approved 1440' long straight with a mile or two of slightly uphill runoff at the end.
With my sympathetic hat on, I can easily understand why both parties assumptions were sensible to them when they made them.
With my cynical hat on, it's pretty obvious that you could install a Wordpress theme or plugin on anybodies hosting platform with an O(m^n) or O(n!) function it it somewhere, and no magical cloud autoscaling pixiedust is going to solve your problem. At the same time, the expectation that a company claiming "Insanely Fast. Infinitely Scalable." and "At WP Engine, there’s no “first level” of support–our entire staff are WordPress experts, so you never hear “We don’t know how to do that.”" on their homepage should be able to do a better job keeping a Wordpress site up than some time-limited non-technical guy self-hosting on Linode.
Edit: and Wordpress is a monster. But that's why I wanted to pay you guys in the first place.
FWIW, it's clear that you _did_ mislead Jacques.
Whether the assumptions he made were valid or not (and frankly, based on the content of your homepage, it's hard not to have sympathy for his point of view), he clearly expected you guys would do a better job hosting Wordpress that he could do himself on his own Linode VPS. He also clearly expected that if there were problems, your "Expert Wordpress Support" would provide the solutions. Instead he found your support guys effectively saying "we don't know how to do that" and flicking him on to WebSiteMovers, who _also_ end up saying "we don't know how to do that" and charging him $500 for failing.
So I think you did let Jacques down, and I think he has every reason to feel missled.
You should probably acknowledge, at least prominently enough that anyone researching your service with a view to becoming a paying customer will find it, that there are possible pathological cases and configurations which WPEngine's technology/infrastructure/service will not solve. If you don't want to feature this on the homepage (and I can understand you not wanting too), I'd suggest a note near the top of the Disallowed Plugins page - and probably re-wording that page to make clear that avoiding everything on the "complete list" of disallowed plugins doesn't guarantee a successful deployment, and also that it's possible for themes with certain code patterns in them to cause a WPEngine deployment to fail to perform as advertised.
Secondly (and I suspect you've already got this one in hand), you need to train your support staff to recognise more quickly installations with pathological performance things going on, and apologise and extricate yourselves and inform the customer as early in the process as practical. I don't know what the tone/implication in Seans message about the ~188k row sort was, but from reading your defence here I can't help but think there was at least a bit of "you're doing it wrong!" - which even when it's true, it's not the wording you want to feed an unhappy customer.
(Note: this is written as a happy WPE user, with some curiosity about exactly what went wrong, and why some guy with a Linode VPS can, in some cases, keep a Wordpress site up better then WPE. I'm suspecting he's got define('WP_MEMORY_LIMIT', '2048M'); or something similar covering up some quickly hacked together "it works for me" not-very-well-though-out-code - which you can get away with on your own hardware/VPS, but not generally on shared hosting...)
You guys say right on your home page that you're "infinitely scalable." [...] The (strong) implication being that any WordPress configuration will perform better on WPEngine than on an alternative hosting platform."
I agree with your general point that, from the standpoint of the entrepreneur, "the customer is always right" should be words to live by. On the other hand, in the paas industry, it should be axiomatic, i.e., understood by all competent parties, that horizontal scaling (the "infinitely scalable" bit) manifestly will NOT address problematic architectures, n+1 queries, full table scans, poorly/non-indexed schemas, etc.
This, to me, is not blaming the customer, it's laying out in reasonable terms assumptions any competent developer or software entrepreneur should understand long before they employ a paas.
Your larger point, though, is valid: even after bracketing the given assumptions of what the responsibility of a paas truly is, better communication up front could temper expectations as to what "infinitely scalable" (a bold phrase, to be sure) really means to the individual customer. It doesn't mean a paas can fix a application that locks the datbase for a full second per query, etc. It does likely mean (I'm assuming here, I have no experience with WPEngine) spinning up new instances and handling increased loads should be Heroku-simple.
In this case, I think WPEngine is taking the hit for what they should (bad communication on the support side) and explaining in real terms what paas's can and can't do. There's a difference between "blaming the customer" and simply being honest about what the bottom-line issue is. NOT addressing this side of it may have long-term implications if another customer comes along with similar expectations. Being clear about what the client's responsibility is doesn't necessarily equate to "blame."
The problem with this is that the original post doesn't say "I have a site that was crashing and locking up all the time, so I took it to WPEngine." It says "I have a site that was running fine on a VPS at Linode, but I got tired of the hassle of managing the VPS, so I took it to WPEngine."
The original post is written by the complainant, of course, so it's possible it's subjective or slanted. But if it's correct on this point, I don't think it's unreasonable for a competent party -- in this case, a person with a WordPress site that runs acceptably on a commodity VPS -- to expect that it would run acceptably on WPEngine as well. Presumably performance killers like the ones you cite would be just as bad on the VPS -- if not worse, since it's unlikely that a VPS' operator will be able to tune MySQL, PHP, Apache, etc. to the degree that an expert at a place like WPEngine would.
There's a big difference between being infinitely scalable, and being infinitely foolproof. If your PaaS exposes enough power to the user, they'll find a way to shoot themselves in the foot occasionally.