Hacker News new | past | comments | ask | show | jobs | submit login
Optimizely to be acquired by Episerver (bloomberg.com)
158 points by scootklein 10 months ago | hide | past | favorite | 183 comments

I worked at Optimizely from before its series A in 2012 until the end of 2016, so I have a unique perspective on this. For most of the time when I worked at Optimizely, the company was all the rage. It appeared at the top of most "hot startup" lists, the Glassdoor reviews were 5/5, revenue was skyrocketing, and for a period in 2014 it became the fourth most valuable YCombinator company (after Stripe, AirBnB, and Dropbox). Of course, Optimizely's success was't guaranteed. In 2015, the company abandoned the self-serve market that had driven its original momentum and pivoted instead to vague and indefinite enterprise offerings that were (and are) hidden behind schizophrenic marketing, an impossible sales process, terrible customer service and a general approach of trying to extract the maximum amount of money from clients rather than providing them with value. From 2015 on, everything (including the internal culture) became mumbo-jumbo, a cloud of dishonesty. I used to be able to explain what Optimizely did to my grandmother; now I don't even really understand it myself.

Th Episerver acquisition is indeed a bad exit, and I think I will lose >$100k in stock I exercised (which is OK, I'll be fine). But I hope all readers will take from this saga a lesson in humility and the pitfalls of intellectual dishonesty and hubris. Just because your startup is skyrocketing isn't enough. Success is not guaranteed. Your company's leadership needs to be honest with itself, which Optimizely's leadership was not. They need to be humble and work hard, which Optimizely did not do.

I joined in 2015 and left after 2 months. I had left a fast growing late-stage startup, AppDynamics, where revenue was doubling every 12 months, from $75M to about $150M when I left. The director of engineering at Optimizely who hired me said their revenue was doubling too, from $40M to $80M. At the next all hands a few weeks later, the CEO said revenue had been declining slightly for 2 straight quarters. At lunch, after the all hands, no one seemed to care. Someone literally complained that their friends at AirBnb got duck for lunch and we didn’t. The director either lied to me or didn’t know. People were nice and smart, but it was clearly going off a cliff. It seems for many B2B startups the march up the value chain to enterprise is challenging, especially with regards to pricing. I remember the president of sales at AppDyanmics would not cut prices to compete against New Relic in the self-serve market. At Optimizely, I recall enterprises were pissed at being charged more just for better SLAs. From other comments it sounds like they finally figured out the enterprise pricing model by jettisoning self-service. Not sure if that was the right call or they could have found better success by avoiding the pricing consultant fiasco. Though I vaguely recall they were losing too much money per customer so that needed fixing. In any event, it’s hard enough to make money in startups as an employee, but declining revenue at a growth stage startup is a death sentence for your equity. I liked the people at Optimizely, the transparent culture was great, and I wished it would have done better, but the writing was on the wall. Sure enough, not long after leaving the first round of layoffs came, private equity invested, and the announcement was revenues doubled over the prior 18 months, minus the detail of the last 6 being flat to down. Sorry to hear about the impact on you. This is a problem with the current state of startups staying private longer and why I likely won’t work for one that doesn’t have an extended exercise window [1].

[1] https://zachholman.com/posts/fuck-your-90-day-exercise-windo...

The pricing consultant they brought in was Simon Kutcher...

I have a sort of general question about this narrative, which seems to apply to lots of startups that begin as self-service, developer-focused projects and end in enterprise hell.

Is it not the case that these startups begin developer-facing, get market traction, are lavishly funded, and then discover that the self-service offering they've built simply can't satisfy the projections they've made to justify their valuation?

Which is to say: would Optimizely be doing much better if they hadn't pivoted into enterprise hell? Or would they be a much smaller company?

I see why customers would have a strong preference! But it's less clear to me what the right decision for the business is. But I'm just asking!

It's possible to do both. At Twilio we started as a self-service, developer-focused company. Today we have many large enterprise customers (and spend a lot of effort to court those kinds of customers), but we still have a huge number of small developers, and one-person hobby shops can and do still easily set themselves up to use our platform. We have added some enterprise-only features, which I sometimes have mixed feelings about, but everything that's traditionally been available to non-enterprises is still available to anyone with a credit card.

I completely agree with you on (at least in terms of Twilio's experience):

> would Optimizely be doing much better if they hadn't pivoted into enterprise hell? Or would they be a much smaller company?

We were definitely going to hit a ceiling if we didn't add enterprise features and work on getting all the various certifications that large companies will require to even start talking to you, and build a sales force that knew how to sell to larger companies. If we hadn't done all that, we'd be a much smaller company today, and likely a competitor would have done it instead and eaten our lunch.

> It's possible to do both.

I think the more interesting question is: can you always do both?

I agree in a general sense, as there are a lot of excellent examples in the wild at this point (of companies consciously maintaining their developer base), such as Cloudflare, Fastly, Stripe, DigitalOcean, Twilio, GitHub and so on.

It may be a case where it's much easier for some types of services than for others.

It's probably true that you can't always do both, but I'm not seeing how it wouldn't be possible with something like Optimizely.

I worked at Optimizely for 4 years and can tell you that pivoting to Enterprise has been one of the best company decisions.

One of the important distinctions between Optimizely and most developer-first platforms is that experimentation is a hard practice to pick up. Most companies have difficulties getting their programs off the ground and keeping them funded, let alone grow or scale them. Small digital businesses struggle more for several reasons: 1) they have few resources, so teams are understaffed and resources are pulled easily, 2) they have little money, so the percentage uplifts are rarely motivating, 3) they have little traffic, so it is harder to get a statistically significant measure in their experiments

Because of these issues, Optimizely always had really poor retention in the SMB space. Nonetheless, the SMB customers helped Optimizely build up a brand name, build up legions of practitioners, and get the skills and experience to go after the Enterprise market. When Optimizely started acquiring enterprise customers, retention improved substantially.

This isn't to say that there aren't lots of problems with Enterprise sales and that Optimizely didn't make tons of cultural mistakes in that pivot. But on the core financials, Enterprise kept Optimizely afloat. The problem wasn't the pivot to enterprise, but the trade-offs that were mismanaged along the way. The path to enterprise was inevitable and correct.

First off, love the username :-).

You argue that pivoting from self-serve to enterprise was one of the company's best decisions. If that were true, Optimizely would be considerably more valuable now than it was four years ago, which is manifestly not the case.

Moreover, if the self serve model was indeed fatally flawed, then it would not be possible for anyone to build a profitable business serving this segment. However, a competitor, VWO, showed that it is possible to build a large, profitable business using this model.

Finally, let's say my prior two arguments are wrong, and the unit economics of the self serve business were fundamentally flawed--an assertion I challenge--the self serve business would still have been useful as a source for attracting future enterprise customers. When Optimizely cut the self serve plans, it also killed its largest source of enterprise deals.

Are you suggesting that revenues fell after bailing on SMB. That is hard to imagine, but I guess possible. Given that the core of the industry is about asking counterfactual questions, I would think the appropriate question would be 'would they be more valuable now if they had not gone to the enterprise - would they even have the deal they did wind up getting?'rather than are they more valuable now then they were 4 years ago. Hard to know, but my guess is that they wouldn't. A simple web editor with a random number generator isn't going to be of interest to anyone looking to buy. The larger problem is that statistical inference is hard - it just is. And, unlike analytics, where you are just placing sensors into an existing system, here you need to also place actuators, so implementation is much more complicated. That means that at any scale, both the marketing team, and the dev/IT teams need to be involved for anyone to get value and not wind up breaking systems all the time. Software like theirs, and ours, isn't magic, and without good editorial and hard work by the client, the entire exercise is more statistical theater than science. And that material fact was always in conflict with the rhetoric that they make AB Testing easy for everyone. It lends itself to a particular type of solutionism that VCs and the larger industry are prone to be seduced by. Self service to SMB clients might be a profitable biz, but perhaps not enough to service such a large amount of venture funding. FWIW VWO also tries compete at the enterprise.

Richard, I appreciate your theoretical arguments. Consider me the applied physicist who has actually seen the data. :)

On the valuation. Optimizely's pre-exit valuation rose in every VC round since changing the focus of the business from SMB (Series B) to Mid-Market (Series C) to Enterprise (Series D). The valuation of the exit took a hit for reasons that were not the enterprise focus. I could say more here, but consciously choose not to.

The self-serve model at Optimizely was certainly flawed. If VWO was able to build it profitably, that's likely because their labor costs, being based in India, are substantially lower than Optimizely's. Keeping a legion of engineers funded in SF has very different unit economics than keeping engineers funded in India.

Now on your last argument, that self-serve customers could acquire enterprise accounts, I certainly agree. If that strategy were better executed, it could have created a valuable pipeline. But the self-serve to enterprise funnel was sorely lacking, and when self-serve was cancelled, the business benefited financially and migrated everyone to the new pricing model for substantial success.

Its easy to look at Optimizely's final fortunes and link that to the enterprise approach, and I won't argue they're completely unrelated, but there were very different factors that caused Optimizely to fold into EPi's acquisition spree.

Thanks for making these points. They seem less hyperbolic and more grounded than your earlier claim ("it was one of optimizely's best decisions") and I appreciate that change in tone.

You characterize my arguments as theoretical, but they are empirical. Optimizely's valuation didn't increase after the decision to kill self serve was made, and VWO was able to build a successful business without killing self serve.

Also, as you now agree, optimizely would have had a better source of enterprise deals with self serve intact. That's important.

Stepping back, every SaaS business has issues converting self serve users to enterprise. Fixing those issues takes hard work, which optimizely was not willing to do. Similarly, every business has higher churn in monthly self serve plans than annual enterprise plans. Fixing this takes hard work too. Things were not unsolvable, optimizely simply had no appetite for solving them.

To address your other points: 1. I donbt that labor costs would have made a difference, but say they would have. Was anything stopping optimizely from reducing costs to have overseas development in india? 3. Your comments about the various foci at different rounds of funding is irrelevant. I'm not arguing that building an enterprise business is bad, I'm arguing that killing self serve was bad.

Ultimately, as Dan predicted in 2013, Optimizely died of indigestion rather than starvation; the market was always there, and still is.

Hope this helps clarify my position, even if we still disagree. If you send me an email, I'd be happy to help squash your skepticism about the $100k too.

PS. I don't think that killing self serve was the only thing that led to optimizely's demise, but I do think it was a factor.

"Optimizely died of indigestion rather than starvation" while it might be true, not sure if Dan ever admitted anything close to that in public.

I think your argument that VWO is a profitable business, is unknown. VWO is also still private. It could be going through the same issues. Additionally, I'd say that valuations are inherently made up. Optimizely was perceived as having more value previously, it doesn't mean it actually was. When it was serving the SMB space, it had a massive churn in that space.

>I think your argument that VWO is a profitable business, is unknown

VWO has been profitable since its first month of inception

(throwaway account + I am a VWO employee)

The big missed opportunity was to build a bounty-based marketplace to serve the SMBs... charge for results vs. the product.

Would the bounties be large enough to cover the work involved? I'm doubtful.

Never dump your prosumer/developer/enthusiast base, no matter what any business expert tells you. Keep them going in parallel.

CROs and VP of Sales types don't like the idea because they can't control the messaging and manage the sales funnel. But if you have people who genuinely love your product and will champion it internally, ignore these fools. Just keep delivering a quality product with value and if there is a market, the revenue will follow.

GitHub, Twilio, and OpenDNS all built their businesses off people taking tools to work.

This. Not least because some of us have invested effort in singing your praises, selling you to teams, and used our reputation to promote your business, indirectly benefiting because clients are happy.

I've been burned twice doing this; once with Fastly [0] and the second time with AppNeta APM, which they then dumped as it didn't fit with their "enterprise" portfolio.

0. https://news.ycombinator.com/item?id=22073520

I think that this is a great question that isn't asked nearly enough. There are all sorts of stories about companies moving up market successfully, but I'd love to see more written about the cases where it's attempted and failed. The DNA of enterprise vs SMB SaaS is really different.

For what it's worth, I would guess that had Optimizely not pivoted to the enterprise they would indeed be smaller, but more importantly would have had a slower growth rate at least at that point in time. In an industry obsessed with high growth rates that's the kiss of death and I imagine the reason behind their pivot. But that's just my guess.

> then discover that the self-service offering they've built simply can't satisfy the projections they've made to justify their valuation?

I suspect a lot of startups sell their investors on enterprise from the get-go. Self-serve can be seen as a foot in the door, a way to prospect potential enterprise opportunities. They watch self-serve sign-ups for a bigcorp.com domain and then hand it over to account sales and swing for the fences.

I think a big problem is differentiating your self-serve from your enterprise offering. You don't want bigcorp.com to feel happy enough with your $20/month foot in the door offering. And at the same time enterprises aren't stupid money fountains and they don't just sign $100k/year contracts unless they see major value. I think this creates a volatile business where a dozen or so enterprises make up the lions share of revenue for a startup and the thousands of self-serve customers are just kind of there like background noise.

SMB and grow into enterprise is the easiest path bc can iterate and grow revenue. In theory, similar effort for more revenue initially and more revenue as seats grow.

Problems are often:

-- Beyond just VC, it becomes an issue ongoing to having full-time dedicated sales: quota, qualified leads, etc, it's a beast. Internal culture and priority shift to start/maintain/grow.

-- if not naturally pulled here and no obvious tier separatation, above can easily destroy the SMB side, instead of using as part of your moat + growth. The marketing+sales org will kill it, unless you split those as well or otherwise solve

-- VC money just means artificially faster deadlines and expectations for all of the above, so even harder to do 2 things, even if long term natural and better to habdle

I agree with the point you're making.

I think the problem is to become "lavishly funded" means that you need to have a pitch that creates a narrative of how you will reach a lavish level of revenue that is believable.

For B2B SaaS, usually that means moving upmarket and raising prices.

I don't think we can say that Optimizely was necessary wrong in doing what they did with knowing what they knew at the time. There are many examples of B2B SaaS companies successfully starting with SMB, then going enterprise. First company that comes to mind is New Relic ($3.6 billion market cap)

To piggyback on this question, I am also curious - would Optimizely choose to go this Enterprise sales route if it weren't for their sky high valuation back in the day. How much of this change was driven by the customers' need vs. a perceived opportunity to grow the company?

You know the answer to this question. Rationalizations may have included lines like “pivoting to enterprise will help us help even more customers” but it’s all about growth and TAM.

Funny - I worked for two SaaS businesses that at one point used Optimizely and loved (and I mean LOVED) it.

But I distinctly remember at the second job, that incredibly rough transition to that vague enterprise pricing structure along with the software getting clunkier and clunkier, which led us to abandon them and never look back.

I think at one point they were trying to get us to go from paying $99 a month (!!!) to like $3,000 a month for virtually the same service??

I'm kinda curious about your >$100k loss on your stock. Your strike price from pre series A options was more than the gains on a near $600 million sale price? Maybe I am misunderstanding, but how is that possible?

Probably the op spent $100k to exercise the options. It's likely this deal will wipe out all common shareholders and only the VCs will get anything. That's a $100k loss. The op will be spending the next decade writing this loss off against capital gains and earned income.

I've been there.

Can you write off just the exercise price or the fair market value at time of exercise?

Confirm this with your tax advisor, but FMV at exercise is your basis in later years (generally because the difference in FMV and strike will have caused you a taxable event).

c.f. https://www.untracht.com/news-insights/vantagepoint-newslett...

Edit: Consider the case where this were not treated as true. You're clearly not going to pay $2 strike, exercise at $5 ($3 of employment income due to the bargain element), sell at $10 in a later year and pay on $8 of capital gains, right?

I'll note for conceptual understanding (by people who aren't patio11, who I'm sure understands this all far better than me) the general point that a gain or loss on the sale of a capital asset is how much money you got from selling it minus how much you "paid". More technically, how much you "paid" is called the basis, and you can do things that adjust the basis as patio11 noted above.

For general information on this topic see https://www.irs.gov/newsroom/capital-gains-and-losses-10-hel...

That's a really good question. I'm not certain of the answer.

When I had this happen to me the FMV was slightly higher than my exercise price, but not enough to trigger AMT. I know I only wrote off the actual cash I lost, and I used a CPA to help me make sure I did it correctly.

I recently asked a tax advisor a similar question: if I do a cashless exercise sale for $5 when the FMV is $6 and my strike price is $2, can I pay income tax only on the $5 sale price? The answer was no: I would owe income at the $6 price and then immediately accrue capital losses on the $1 spread between sale and FMV.

I guess that's a longwinded way of saying "I doubt it". Tax law around employee options is brutal. :/

Yes, capped at $3,000/year.

If the OP is telling the truth, there is no way he lost money on the stock he bought. The early shares are all up.

Not necessarily. There is liquidation preference for the preferred shares.

Yep. They are listed as having taken money from Insight Partners, a growth PE firm. You know they negotiated some preferred terms.

For the record, I think op is probably speculating a bit. That said, I'm not sure how you can be so sure op's not losing money unless you have intimate knowledge of the cap table and the deal.

The acquisition share price is usually told to employees. They probably got the information from current employees. Since they are a shareholder, they'll eventually find out anyway.

I was involved in an acquisition as a shareholder. I was nigh impossible to get anything out of the executives. It was an acquihire so maybe things were different.

That...sounds illegal. You should at least have gotten a document in the mail with the buyout details, after everything was finalized and signed. It may take a few months in some cases but they have to send it out.

You're certain you were a shareholder and not an option holder? That is, you had exercised some of your options?

Another possibility, since you said it was an acquihire, is the company actually shut down operations and sold off its assets to the buyer. Maybe in that case there would be no share price to report since no shares were bought? I'm not an expert.

I was definitely a shareholder. I had early exercised some options, and reached my one year cliff. I believe your conclusion is correct. They shut down the company and sold off assets. The acquihire probably returned some money to the investors.

I spoke with (paid) two attorneys (employment and business) at the time and they said there wasn't much I could do. In the grand scheme of things it was small potatoes. I was pissed at the time though.

I worked there in 2013. I very specifically remember getting a speech about how if we didn't believe that the company would be worth more than google, we should quit. Seems like that may have been a better option for a lot of folks.

Wow, I was interviewing there in 2013. I was interviewing for a fairly senior engineering position.

When they sent me the details for the on-site they sent me a role two levels lower. When I asked about it I was told the Director of Engineering felt that was a suitable role.

I withdrew from the process at that point. I was pretty bummed as I was super high on Optimizely back then. But I knew enough about myself to know that wasn't a good way to start a job (assuming I got an offer).

Was that the one where Dan told people that they were traitors for wanting more cash instead of stock, not super long after some of them cashed out some of their equity?

Surely you're joking, Mr. Feynman. (Sorry, I've kind of always wanted to be able to say that in casual conversation and not have it be a total non sequitor.)

But, seriously, your description does sound exactly like a company on the decline. I couldn't blame you for bailing. What do you think the first signs that things were terminal there?

Speaking as a customer, it feels like this is exactly what is happening to Mode Analytics right now.

As a customer of Mode, why not give Taplytics a shot? :)

That's the phrase I was looking for. Their sales process is impossible.

They appear to have become a great example of why the small businesses I run typically just walk away from any service we were potentially interested in using if we see a pricing page containing the word "call" but no actual pricing. If you're going to aim for high-touch enterprise sales, that's your choice, it's your business. However, the chances that you will then provide either acceptable quality of service or good value to anyone on the smaller end of the scale tends to zero IME, so it saves everyone time if we look elsewhere immediately. The problems start if it turns out that these companies aren't actually generating enough value to justify the enterprise-level costs either.

For example, say you're running a tool that allows people to quickly experiment with multiple versions of their web site, measure some quantifiable success rate for each version, and perform some basic statistical analysis to guide future changes and improve conversion rates. A basic but useful version of this tool can be implemented in a few days by one competent developer and one competent statistician; I suspect quite a few people reading this discussion have done exactly that. Polishing the tool might take longer and improve its utility somewhat, but it's not as though it's using some secret technique that no normal business can implement for themselves in-house.

At the mostly-self-service end of the spectrum, it might still be worth customers spending a bit of money on the pre-existing tool you make to do that job for them, because you're really competing on immediacy and convenience as much as technical capabilities. At the enterprise level, your competition could instead be some in-house team or some freelancer or agency being brought in from outside just to develop a tool directly for your customer. If they're potentially doing that at a cost less than just the first year of annual fees you're demanding up-front, and according to the customer's exact requirements, both of which seem quite plausible in a case like Optimizely's based on information in other comments here, what exactly is your sales pitch?

When I used to work for one of the UK’s largest social networks, I spent a long time researching optimisation platforms. I’d used Optimizely before and liked it, but when I saw the pricing table had been replaced by the dreaded ‘Call’ CTA I knew we were in for a bad time.

When I finally negotiated the rigmarole of callbacks and salespeople, the quote we received was so ludicrously high that there was not a chance that it was ever going to be worthwhile. Especially considering their old self-serve model was pretty good value and the horde of functionally identical and much cheaper competitors.

That's really interesting. I work along side Optimizely in the analytics field and I remember that moment - I used to email into support@ to ask about stuff to help improve data across our platforms, and always had no problem. Almost over night the email support was shut off (in support of optiverse + partners, a route I totally get when you're trying to move upmarket) but it felt really forced, really fast and I think they could have done a better job ramping up enterprise without slamming all the self-serve and free customers.

My company is going through a similar phase of trying to move more up market, and I'm glad we've tried to keep the free/self-serve tiers explicitly because we think they build good mindshare.

My impression is Optimizely never crossed the chasm. Huge following with VC funded startups and early stage startups but never crossed into "Google Analytics" territory where every company with a website was using it.

Slowing down users meant higher prices to sustain VC promised growth. Higher prices meant enterprise sales. And it went down from there.

I'd say in the early days they were on the way to getting that mindshare. I had SME clients using them, where VWO was cheaper but they didn't like using VWO.

When Optimizely hiked their prices and lost the smallest plan, all these clients went elsewhere. I'm sure their contribution was noise to Optimizely's turnover, but the awareness of what was possible with A/B testing and the way they'd tell their friends about Optimizely and what they'd just done on their website - you couldn't buy that exposure.

Yeah, I agree. I think the use cases weren't substantial enough so people just didn't use it enough to justify subscriptions in most cases. This means high customer churn, especially if they were selling to customers based on strength of their name (which I think they were, to digital marketers who conned themselves into believe it was a product they needed). They received funding from a PE firm (Insight Partners) who were probably eager to get a return.

If you can share, what was the dishonesty you're writing about? I can't tell if you're saying it was fraudulent dishonesty or self-delusion (where you think you're in a better position than you really are).

And do you know if the self-sever market was drying up for Optimizely in 2015? I assume they wouldn't abandon it if it was growing at a decent rate.

> The Episerver acquisition is indeed a bad exit, and I think I will lose >$100k in stock I exercised

Based on what information? So far, all that's been publicly confirmed is that the sale price was below $600M, which presumably leaves opportunity for your shares to be worth something.

GP probably got information about the share price offered from current employees and did the math. Once the acquisition is finalized, they'll receive more details since they are a shareholder.

Employees don't know share or exit price yet. Won't be announced for a few more months.

However, if the OP is speaking the truth about the timeframe he got shares from, he is guaranteed to be up. There are definitely people who lost money on this deal, but definitely not Series A people.

Worth noting that Optimizely is the A/B testing tool of choice for donaldjtrump.com.

This is unfair. That contract was sold to the Republican National Committee 3 years ago. They also sold to most of the democrat campaigns. I'd be careful about any business with over 1,000 customers, chances are they have a few shady clients in their portfolio.

To be fair, there was a pretty vigorous internal debate about what to do about that a couple months back. There's a huge segment of the employee base who were advocating for canceling the contract with the RNC (the Trump campaign piggybacked off that).

This description absolutely nails Foursquare as well.

Wow. I interviewed there in 2015 and got turned down. Wasn't a great interview experience either. Bullet dodged, I supposed.

Me too - super arrogant. While I clearly wasn't what they were looking for, I had a lousy taste in my mouth after that experience. I still remember an awkward moment when I answered a question and the interviewer circled back to remind me I hadn't fully answered his question (he was right, but it just felt unnecessarily didactic).

I wonder how Google Optimize factors into this. Provided its a free service, it seems like Optimizely may have had trouble offering something unique to the self-serve market over something that is completely free. Similar to how it seems Dropbox has less to differentiate itself from Google Drive and other similar services these days.

I think Google is less of a threat than many believe because Google products have to address a Google-sized market which leaves tremendous room for competitors. If you're going to go straight up against Google then, sure, you're going to get clobbered.

I initially misread the title as "optimized to be acquired [..]"

Maybe my serendipitous reading explain your observations of what happened to the company.

Love these insights

Why Stripe will likely win: https://news.ycombinator.com/item?id=24067211

Continuing self-service while expanding "up market" into the enterprise. If AWS can be self-service, pretty much any service can be.

Stripe may expand "up market" at some point, but at the moment they're pretty enterprise unfriendly and don't seem interested in becoming enterprise friendly. You can't pay for phone support, their PCI compliance can cause some enterprise customers to blink and even if you're funneling millions through them they won't negotiate on price like other payment gateways. Also many of their more advanced features, just aren't that well implemented or documented (think connected accounts, etc) and since those cases aren't as heavily used we've had to side-channel to a C-level to get an issue expedited after bouncing around support for a show-stopper bug.

Additionally as a sidenote, that market is quite crowded with a LOT of choices. This is certainly not a winner take all industry, there are literally 20+ choices that will be reliable. Typically most large enterprises will end up with a vendor that will give them a good deal on rates, which isn't Stripe.

Appreciate the feedback. You can pay for phone support (indeed, you get it for free, though you can pay extra for a premium support package if you like). We think that our PCI compliance functionality is best-in-class. And we do negotiate price for larger accounts, as indicated on stripe.com/pricing.

More broadly, Stripe now works with a long list of businesses that are processing more than $1B/year or more, and that list is growing quite quickly. Indeed, there are more enterprises using Stripe than Adyen, which is often cited as an ostensibly enterprise-focused competitor. Larger companies using Stripe include Amazon, Shopify, Instacart, and Peloton. (There's a longer list at https://stripe.com/customers). That's all to say: we're very invested in this enterprise thing.

If any enterprises you work with have had a bad experience, would welcome any details. patrick@stripe.com

As someone who recently oversaw the transition from Zuora + Authorize.net to Stripe for a business in the eight figure a year range, can confirm.

Stripe worked with us on pricing discounts (over $150k per year in savings by switching from our previous set up) & the enterprise support is absolutely top notch. I literally could not be happier with the service.

This is different, both Stripe and AWS users are developers.

I don't know how you can be so sure that Stripe will win, Adyen is already on the enterprise space, and they are a bigger company than Stripe.

> Continuing self-service while expanding "up market" into the enterprise.

Things are more complex than a sentence, the whole company needs to operate differently https://blog.luap.info/why-most-saas-companies-cant-be-succe...

Stripe users are also anyone who wants to run a website on Wix or Squarespace or WordPress/WooCommerce and want to sell things. I have personally seen people that can’t spell “HTML” successfully set up basic eCommerce sites and make money off them. There is zero reason to pigeonhole Stripe as being for developers only any more than Square, etc.

I interviewed for a marketing role at Optimizely back in 2013...I passed all the interviews with the team and then had a final, short interview with the CEO. He asked me a few basic questions and then asked 'if you only had 3 years to live, would you work at Optimizely?'. I responded honestly and said no. Said that I'd love to work here to help and grow the business, learn, and further my own career but if I had only had 3 years to live I'd spend my time differently. The hiring manager called the next day and said I would not receive an offer and when I asked him if it was because the answer to that question he said yes. That made it obvious they had a strange and not particularly healthy culture...lucky for me as I ended up at a much more successful early stage startup where I accomplished what I wanted to accomplish.

Over the years I A/B tested each of my interview questions in order to better measure what I was looking for. It sounds like you might have been asked an early version of one of the questions I used to ask so sorry you didn't get the "optimized" version. (there was a big difference in responses between only 3 years and 10 years to live)

Also, to clarify, I used to end my interviews with TWO questions: (1) if you had 10 years to live, what would you do? [wait for answer] (2) if you had 10 years to live, would you take this job?

The things I was assessing in these questions were candor, intellectual honesty, and passion. Sure, it would be great if people authentically were passionate about taking the job if they had ten years to live. That was a tiny minority of responses.

The only "wrong" answer to these question was when someone would answer YES to the second question after clearly answering something completely different to the first one. For example, if someone would say travel the world to #1 and yes to #2.

The reason why this measured candor was because if someone could tell me to my face during an interview they wouldn't take this job, then I knew they would tell me to my face when something was broken in the company after I hired them. I was looking for the exact opposite of what this thread implies I was looking for. I didn't want ass kissers. I wanted truth tellers.

Reading the op's post and your reply, I can only come up with two possible conclusions:

1. The op is lying about his experience.

He states he answered NO to question 2, which is what you claim to be looking for. Unless his answer to question 1 was "Work for Optimizely" (which, I guess, someone might say, maybe) then I don't see how you get a contradiction. By your logic, he would have been a hire.

2. The recruiter lied to him about why he was rejected.

Maybe everyone got a short sit down with the CEO back then and he took that to mean he passed all the other interviews? Maybe something else didn't check out and that was an easy way to let him down?

Anyway, if you're A/B testing, wouldn't you hire people regardless of their answers and then assess their performance over a longer timeframe to determine the efficacy of the questions and answers?

Not sure exactly what happened but I did personally interview the first hundred employees so that means I probably interviewed him or her and would have done so regardless of how the other interviewers felt about the candidate. My interview feedback was treated like the rest and I rarely if ever vetoed a hire if everyone else was unanimously in support.

And you are right, I did hire people regardless of their answers to some questions and I would then assess how they turned out to figure out if my questions were any good. I basically gave everyone I interviewed a pass on at least one question even if they bombed it abysmally. Some of those folks turned out to be our best employees so I learned to never hold one bad answer against someone. It also gave me a dataset to improve my questions for the next wave of candidates.

Of all the questions I've been asked in interviews over the course of my career, this is the only one I can actually still remember to this day. Partially because it's extremely unique. And partially because I was fairly certain at the time, judging by the look on your face, that I'd completely bombed it. (Also reading your explanation above, I now KNOW that I bombed it.)

Anyway, I'll attest to the fact that Dan at least didn't veto all candidates purely on the basis of their response to this question (unless I was the result of a massive clerical error).

Outcomes aside, I'm very thankful for my experience at Optimizely. My time there was a major inflection point for me both personally and professionally. I wouldn't be as effective in the workplace as I am today—nor would I have some of my most meaningful friendships—had I not cut my teeth at 631 Howard — thanks Dan (and Pete, if you're reading this too).

EDIT: Also considering the timeline and the role described in the OP's message, there's a nonzero chance we were interviewing for the same role. Small world!

Surely another possibility is that dsiroker has lied about his experience?

Yeah, that's a possibility too. Thinking about this more (really, why am I still thinking about this more?), I regret positing that OP was lying without including the possibility that dsiroker was lying.

Additionally, I should have considered the possibility that both parties are mis-remembering the interaction. Seems I owe @worldsoup an apology.

@worldsoup I apologize for suggesting you were lying about your experience.

Anyway, if I had to guess as to what really happened, I expect the recruiter told a small, white lie when pressed on why OP didn't get the job.

Probably OP though they'd passed the interview because they got a meeting with the CEO. Or OP thought passing the interview meant they had the job, when in reality there were multiple qualified candidates and Optimizely chose another one.

There's other information in this thread to support this conclusion. I won't rehash it here.

In conclusion: the most charitable explanation is both parties are telling the truth to the best of their memories, and maybe a third party was a bit dishonest in a way meant to spare someone's feelings.

Either way, pretty stupid questions to ask

It always amazes me that people who achieve so much or hold a high ranking position within the company can ask such ill conceived questions.

> The only "wrong" answer to these question was when someone would answer YES to the second question after clearly answering something completely different to the first one. For example, if someone would say travel the world to #1 and yes to #2.

That is a strange assumption.

My answers to #1 and #2 would have been contradictory to you, but with 10 years to live I would see working at Optimizely for 3-5, then doing what I wanted to do for 7-5 years a great tradeoff.

Would I have said that in answer to #1? No. I'd have said "Travel and more time with friends and family", while knowing full-well I had to work, or I would starve.

Would I have said it in answer to #2? Perhaps, but unlikely.

What was the success rate with these two questions?

This is something I wanted to say in my response but left it out to avoid taking away from my other thoughts.

Circa 2013, had I been asked this series of questions, the answers almost certainly would have been:

1. Spend as much time with my friends and family as possible. Do a bit of traveling. Try to finish off seeing all the MLB parks.

2. Yes. I want to leave a legacy for my children. I figure there are a few ways to do that. Join a very promising mid-stage startup, e.g. Optimizely, Pinterest. Join a very promising late-stage startup, e.g. AirBnB, Dropbox, Stripe. Join a FAANG.

I was super hot on Optimizely back then, so I would have jumped at the opportunity to get in relatively early. With ten years to live, and a family to think of, you can be damn sure I would have given all I had for 3-5 years.

With three years to live, I _might_ have taken a moonshot on a mid-stage like Optimizely. In that case my plan would have been to negotiate a high-equity package, early exercise the first year, then quit and live the next two years of my life however. But, I never would have said that in an interview.

Huh, I don't know if I know one person who would honestly answer that question with a yes.

that was a test for a particular type of person that OP failed to pass. He could not lie to a simple question.

So I guess lying was one of the skill he really wanted in a marketing guy?

Ah, I hadn't considered that the interviewer was testing the applicant's ability to lie. Interesting, if true.

Someone who needs or wants the money to provide for their family might.

he didn't say which was his answer :-)

That question is psychotic.

Just kicked the tires on Optimizely for a site with less than a million MAU. They wanted $50K upfront for one year. No monthly or quarterly billing available. Went with Google Optimize instead, works fine for free. In the face of that, very surprised Optimizely doesn't do month to month to get folks started.

It's worse. They essentially kicked-out their existing self-serve customers in the process. We were on the (at the time) silver plan, but when we were ready to upgrade to gold, there was no gold, no silver, nothing... Just some super-expensive and vague enterprise plan.

But there's a silver lining: we created and open-sourced Alephbet[0] - a simple A/B testing platform together with a couple of backend options with AWS Lambda/redis[1] and couldn't be happier :)

[0] https://github.com/alephbet/alephbet

[1] https://github.com/Alephbet/lamed

Thanks for this, also I love the name :p

Actually it's a great idea to name your projects with Hebrew letters. "Shinbet" sounds cool ;)

Optimizely used to, but their sales strategy changed to deliberately reposition themselves in the market.

They've priced out self-service and smaller users, and repositioned their sales model for larger companies with immature internal capabilities. They lock you in with that annual pricing, and include enough margin to throw a massive amount of support resources at you to ensure you get everything fully off the ground and deeply embedded into your internal workflows.

As an early self-service user, it was really irritating when I tried to bring them into a new company I started at and realized they made that change. But after working for a major marketing agency for a while, I've realized that it makes sense for them (even if it sucks for my purposes). In the world of large scale brand marketing companies (such as CPG companies), even a rudimentary informational/branding/brochure-ware website tends to be a $500k+ abomination, involving a super complex IAT[1] consisting of 3-6 external agencies and internal teams. In that world, the single greatest cost for anything is the man-hours required for account management, since even the tiniest of thing involves so much coordination (both logistically and politically). Optimizely's absurd looking price bakes in the cost of providing that level of account management support as well as initial implementation/usage technical support. Without those, it's entirely likely that the brand could purchase Optimizely and it'll sit unused because the agency scopes don't account for it and no one is willing to eat the unscoped hours required to implement/support/use it.

[1] https://isl.co/2018/10/agile-iat-four-principles-for-better-...

I have been thinking about how Saas has been the golden product but how as some smaller Saas companies grow they no longer appear to be selling Software as a service but rather Service via software.

They used to, but they ended it a few years ago. They consciously moved higher market, higher touch, higher cost.

Ultimately, i believe they got squeezed between smaller companies using free or cheaper offerings and larger companies probably building it themselves.

I worked there for 5 years. This is the correct take.

Interestingly, many other companies are starting to take this path, and I think it sucks. Not that they're focusing on enterprise customers, but that they're essentially pulling a bait-and-switch: get traction and word-of-mouth with smaller self-service customers, and then once they get enough street cred, go tell those smaller customers who were crucial to their early success to fuck themselves.

Full Story has gone this route. They used to have a plan that would work for smaller companies, now they have nothing between their free tier (1000 sessions a month) and their lowest level paid tier which is "5 figures annually" (they won't even announce any pricing on their website, though not sure if they ever did).

(co-founder of Optimizely here)

HUGE thank you to YC and the entire HN community for all their support over the years. It was almost ten years ago that we launched here on HN. [1]

[1] https://news.ycombinator.com/item?id=1788634

Still remember first time used original Optimizely product for my own startup. Was amazing

no, thank you Dan! You've helped us and countless other startups find their way. Distinctly remember coming to your office and doing a 2 hour product feedback session on Streak. Super generous!

thanks for letting me shadow the sales team. learned a lot from watching your guys at work!

It was great working with you, Dan.

thanks for an amazing culture and experience at Optimizely, Dan.


Dan, Paras from VWO here.

Congratulations for the acquisition. We've enjoyed competing with Optimizely over last the last 10 years, and certainly learned a lot in the process. Hopefully, that will continue even after the acquisition.

The way we look at things, experimentation as a market is certainly in an early phase. The more complex the world becomes, the more necessary experimentation becomes to understand what customers really want.

As an aside, it is true though that for experimentation to work, a company needs to be ready for it. Their culture needs to support being proven wrong and having patience to build momentum of wins over the long term. Any org with a short term horizon will likely see experimentation as a cost without corresponding returns. But companies that really see long term - think Amazon - ground themselves in experimentation.

Of course, not every company can be Amazon but our belief is that more companies will start realizing that there's no alternative to developing a culture of experimentation. This is why we're excited about the market. For us, at VWO, it still seems day 1 :)

This sounds like a bad exit. From what I can tell the original Optimizely space has been slowly becoming a more discrete area of progressive delivery, rather than an industry on its own.

Many players have jumped into this space with their own A/B testing and Feature Flags solutions as part of their total offering, many of those offerings being free, open source or cheaper. Also potentially better in the concrete tasks they enable. I doubt that Optimizely's feature flag offering is superior to something more specialized like LaunchDarkly.

Also there are a couple of strong incumbents' solutions like Google Optimize and Adobe Target and it's always hard to go against incumbents specially when the incumbents are coming after you and not the other way around.

One clear problem for Optimizely in this space is that experience optimization became a function of marketing departments through out the years but for a while they were positioning themselves as a developer tool. This go-to-market strategy opened a lot of opportunities for other players who saw a bigger market when selling the same type of solution to Marketing Departments.

Maybe I'm wrong but it seems that they just stopped growing and have been experiencing a lot of customer churn since this is likely an expensive product with a hard to calculate ROI. They're probably still selling a lot but nowhere near to the original investor expectations / close to becoming profitable.

I was always very impressed by their office on the end of New Montgomery Street in the heart of Downtown San Francisco. You could see some very swaggy kitchens and open office space through their floor to ceiling windows on the ground floor. I was jealous for some of the employees who worked there.

Maybe this is ad hominem, but it seems to me they must've raised a lot of money to prioritize that kind of setting, likely in the guise of recruiting. Crunchbase lists them as having raised $251.2M and their last round being debt financing.

If this is a down-round acquisition with most of the employees gaining very little I wonder if this is a lesson to founders to be more cost-effective and raise less money.

There were a bunch of startups right there. From the fact that we rented commercial space there (like a minute's walk away), I know that if they leased before 2012 it was going to be a sweet deal. I don't remember any of the numbers now but I recall that if I could get that deal today I'd take it in a heartbeat, even if just CPI adjusted. I think we had just under 10k sq. ft.

> One clear problem for Optimizely in this space is that experience optimization became a function of marketing departments through out the years but for a while they were positioning themselves as a developer tool. This go-to-market strategy opened a lot of opportunities for other players who saw a bigger market when selling the same type of solution to Marketing Departments.

I can't speak for all industries, but for the ones I'm familiar with, marketing is always the product owner of websites. With two practical implications being

1. The entire website costs (development + software vendors) ultimately get booked against their budget, so they have the true purchasing authority for pretty much all the website tech that isn't centrally mandated/controlled

2. Website projects (including budget and requirements planning) tend to start in marketing loooonnnggg before a tech resource gets brought in. So developer awareness/familiarity ends up moot, since there's too much incremental effort and cost involved for it to be easily get buy-in and added to the plan at this stage.

It makes a ton of sense to target your solution at them instead of developers. They may not be able to use your product well or do the implementation, but they're the ones with the purchasing authority and ability to ensure the budget accounts for it. And can add it into the project requirements far earlier in the planning stage than when tech resources get involved.

Google Optimize & VWO hurt them. Free, or low price, low friction won at scale with smaller teams running 1-2 experiments or low level personalization. Everyone I know who's going to high volume testing is either on a hosted CMS that has this baked into their offering or JAMStack.

No one I know has deployed Optimizely since 15/16.

Thats the issue. They start at $50k. No startup can afford that, so we build our own or use google optimize. I've been at multiple startups that have scaled to millions of users and Optimizely could have been a player if they had a self service budget option. Need to get in early like other saas/cloud providers because the cost of switching becomes to high as the business scales. Imagine if AWS didn't have self service option and you had to go through a sales process with min commits. It would have failed.

Optimizely's pricing makes no sense for small teams but VWO and Optimize cannot compete on features that start to become very useful as your experimentation and personalization efforts increase.

Any roll-your-own experimentation platforms take considerable resources to make accessible to those in the organization interested in using it (product, marketing, etc.)

In my experience, I agree the core functionality (variant management, remote config, etc.) is relatively small amount of effort compared to the interfaces to make it accessible to those non-technical orgs, like you mention.

However, we found that those interfaces only allow very limited, shallow tests and you very quickly outgrow them as an organization. In other words, once you reach diminishing returns on button color and header text optimizations, you start wanting to test deeper UI experiences and complicated user flows. At that point, you have to involve engineering anyway.

When an organization has engineers who are motivated by business metrics, they have no problem implementing shallow tests (like button colors) while working on tests of the deeper UI experiences as well. And at that point, the non-technical interfaces have little value.

I wasn't really talking towards the WYSIWYG editors. Those are trash on all platforms and fall apart quickly for all but the most simple tests.

Metric + page management, results analysis, segmentation, etc. all work better in Optimizely than they do in other platforms.

Optimize is not an incumbent but a challenger. Google Optimize came out years after Optimizely and is less featured and built by a smaller team. The main benefit to Optimize is that it's cheap.

I didn't mean to say that Optimize as a tool is an incumbent. What I meant is that Google and Adobe as companies are incumbents.

A few years ago, we were a monthly customer of Optimizely for a few hundred dollars a month. Reasonable for a startup.

Then they went to the annual cost of $30K+ upfront and ended all monthly options. They had to move to high cost, high touch to compete with the free/cheap offerings to stay in business. This acquisition suggests that didn't work.

We ended up building randomization, remote config, and logging ourselves, and did the analysis with our existing stuff.

I used to work two desks over from Dan at the Obama campaign in Chicago in 2007, when he was developing and testing an early proof of concept of what would become Optimizely. He was able to increase fundraising performance significantly by doing (what would now be considered simple) A/B tests. He's sharp and tenacious, identified a market ahead of its time and invented a new product category.

Whether or not this is a "good" exit, it's a great accomplishment for Dan and the team and can't wait to see what they do next!

Didn't they work on an educational game first? Or something about carrots and sticks? I liked their story when I read about them ages ago.

I remember meeting Dan, the founding CEO of Optimizely, at winter 2010 YC demo day. I had just grabbed my name tag and was walking toward the building when he stopped me outside to give me the pitch. He immediately struck me as a smart, capable guy, but as a former software engineer I couldn't understand why companies wouldn't just build A/B testing themselves. What did Optimizely add? How was it defensible? Dan didn't answer the questions to my satisfaction so I thanked him and moved on. As the years went by and I saw them raise round after round, get great press, put up billboards, and build out a beautiful office that I walked by at least once a week, I felt terrible for missing out on the angel investment. I was a rookie investor (I think that was my first YC) and chalked it up to my inexperience. I even tried to extract "lessons learned" and apply them to similar investment opportunities. It feels bittersweet to see things end this way, with probably no return for the common and a haircut for investors. I had built them up and expected them to succeed but I guess it's good to know I was right. Lessons: It ain't over 'til it's over and vanity traction like press & billboards mean nothing. Don't build a big company on a bad idea, it's a waste of time and money for everyone, especially the founders & employees.

BTW, I'm not sure if you would have expected them to get to $100m in ARR either when you passed on them because you were skeptical of the market. There's a lot between $1m to $100m ARR that can go wrong. In this case, I don't think it was the market opportunity.

Did they reach $100m ARR? If so that's impressive, a lot more than I would have guessed.

In 2014 I wrote an article on why Optimizely's approach to AB testing was statistically flawed [1]. I was working at a competitor so I needed to be a bit circumspect.

It was discussed here: https://news.ycombinator.com/item?id=7287665

I always wondered how they got away with it for so long.

[1] http://www.datascienceassn.org/sites/default/files/Most%20Wi...

It's a bit breathtaking how basic the statistics knowledge is to make this critique (not complaining about the critique, mind you). I'm startled that this was coming as news to anyone.

Have you ever given thought to generalizability in A/B tests? I'm surprised there isn't more of a developing science of constructs that generally work...

Optimizely switched to a Frequentist statistics model in 2015, which changed how pretty much all testing companies do stats. Your article was valid, for a full year.

Completely unsurprised.

Experimentation-done-right is too expensive and too ambiguous to sell as a product. Every product experiment requires a complex set up, a lengthy running period across a huge base of users, and then heavy analysis in order to achieve statistical confidence over a specific feature's impact on a business metric. That "confidence" is often represented by a subpercentage point that may or may not be statistically significant. Fun problem for the data scientist, plain hell for the PM.

In my company which uses experimentation for everything, each A/B test requires two weeks before the Product Manager can even see the results. Two weeks of waiting for a confusing, contradictory dashboard that can't be taken at face value, that needs careful, human analysis before it can be called a "win".

That slowness is fine for high-traffic, high-risk & high-value lines of business. But it's not fine when you're releasing feature that aren't just optimizations.

Competitors like LaunchDarkly and Split.io have recognized that critical difference, I think. They know that causality is expensive, and are particularly aware that the fine line between feature release and metrics impact is tied too heavily with a company's politics, i.e. it chafes against the intuition of executives.

Instead, they offer experimentation as an add-on to their developer tools. You can experiment if you need to, but it doesn't obligate you to do so.

That goal is much more realistic than the Optimizely's current goal: "helping our customers win in a digital-first world".

Agree. Plus I'd also like to highlight how experimentation platforms/tools are sold as 'conversion uplifter'. All orgs are looking for solutions to take their conversion graphs up and to the right. And they are ready to pay hefty sums for any tool that speaks to this motivation.

But then they are asked to wait! Wait for the concept to mature within the org. Time being of critical importance most orgs end up getting frustrated as what they initially started with doesn't hold true now. And when you already have lots of money, it's easy to switch.

I count 18 sentences of fluff/framing before they say who is acquiring them. Talk about burying the lede!

18 seems like an outlier, but for press releases I’ve read in the “we’ve been acquired” category, I’d guess that the median is > 10. Does anyone have first-hand knowledge about why these statements are released in this teasing way?

I'd say it's just a PR tactic to make such things seem like "big news," when, in fact, such acquisitions are a fairly normal thing in the tech world. It's hard to even buy that kind of publicity, so one has to take advantage of it when one can.

That is not a successful acquisition. It's definitely a means of burying the lede. If anything, it's to draw attention to the PE firm.

The problem I had with Optimizely when it was the go-to solution, was that it had a truly problematic impact on front-end performance due to the blocking way its script was loaded and page variants introduced. In some cases page loads were blocked by up around 5 seconds.

For obvious reasons it was tricky to run an A/B test just for testing the impact of Optimizely's script itself. But the key issue is that all the similar tools at the time (Optimizely not being the only culprit here) were determined to not required developer effort, which led to poor overall performance.

Then React et al came along and took ownership of the DOM, which meant adding tools which also manipulated the DOM became even more problematic.

Fortunately tools like Launch Darkly and Split solve this problem in a better way (high performance full-stack feature flags), even if it does mean developer effort to add tests. Optimizely did eventually launch their own version of this, but never really won back the developer mindshare.

Ultimately, it seems Optimizely enjoyed a few years of success, but a combination of developers getting more concerned with performance and the front-end world moving on to different architectures, seemed to lead to its decline.

Back in 2015 I was pitching Optimizely internally. We already had an in house barebones A/B testing tool. Once I got over that hurdle our CTO let fly how his only experience using Optimizely had been a time when Optimizely went down and took his site down with it. Optimizely eventually brought out their head of product, and an army of senior engineers to convince him things were good now. We signed an annual contract, but Optimizely was never added to any page that was revenue critical/dependent on speed. Optimizely got their huge contract, and in the end marketing and design got their lightweight A/B testing tool, but we never realized the white paper level metrics jumps because it was never used on any page that would actually have material impact.

Congrats Optimizely from you friends at Convert.com (A/B testing tool that keeps the prices online)

Are there any good open source alternatives to Optimizely that can be self hosted?

I'm aware of Wasabi[1], but I believe it's abandonware at this point.

[1] https://github.com/intuit/wasabi

Another proof point that once the founder-CEO is replaced by a professional CEO, it's game over more often than not. This happened in 2017 at Optimizely.

Happening right now at UserTesting. As the CMO would say: "yay!"


UpGrade [1] is an open-source A/B testing platform for education software. We want to make it easy for education software companies to pilot new materials and measure efficacy. We hope this can help optimize student outcomes and help advance the science of learning. Would love any feedback -- we just launched!

[1] https://upgrade-platform.org/

I would not want to be Optimizely and have to compete against a strong free offering like Google Optimize.

I use Google Optimize. I wouldn't call the free offering "strong."

There is a limit of about six experiments at one time. Each experiment can have up to 8 variants. And if you want more, then be ready to shell out thousands per month (I forget the exact number, but that's the ballpark).

Overall, if I were to redo things, I'd probably be better off just using Google analytics events.

Edit: that said, I'd never use a piece of bloatware like Optimizely for just the split-testing feature. If your product would benefit from split testing, then include it as a feature. It's not a "hard" feature/problem to implement into just about any existing product that would benefit from it.

Optimize can run five experiments at a time and it only lets you measure 3 metrics. Any other metrics need to be analyzed in GA and will almost certainly end up sampled and unreliable.

Congrats, Dan! :)

Hello, Jean-Noel here, from Kameleoon (also a competitor to Optimizely, but mostly serving European markets so far). First off, congrats to Optimizely / Dan and co for the acquisition. For us Optimizely has always been a role model, especially from the technical side of things (I'm the founder of Kameleoon, but currently serving as CTO, so obviously I have a tech background). Optimizely was (and still is, in many ways) a great platform. And it did create (or at least, heavily helped develop) a whole market. However I agree with Feynman's comment, the main problem with this company (and even looking from aside, it was quite obvious for us) was that they raised way too much money and made promises to their investors that were completely impossible to fulfill. Basically, it looked like they wanted to be compared to AirBNB, but their potential was honestly never the same, and this was quite clear from someone with a little insight into the business. Because of that, they started making mistakes after mistakes, burned way too much money and it all went downhill.

They're not the only ones with that problem - almost all competitors / actors in the field raised too much for the side of the market, from my opinion. Dynamic Yield is probably the one that did the best at that game, with a great exit when the hype was at its peak. But all the other ones are clearly in trouble (even Optimizely exit, I am sure, is clearly not a success for their investors, etc). The only actor that has a reasonable overall strategy is VWO (somebody asked if they're were profitable - of course they are, since they did not receive any funding, they have to. While Optimizely clearly never was), because it stayed lean and avoided the pitfalls of over investments (going to all continents on the planet, having 20 offices all over the world, etc). Hats off to Paras for that. We try to stick to the same strategy at Kameleoon, even if we had to raise a bit of capital. But you don't need to raise $200 millions to have a great CRO platform - we proved this for a fact.

About the switch to Enterprise - anyone in this industry will tell you it was absolutely necessary. Impossible to sell experimentation to SMB and expect to be profitable, you need larger customers. Because contrarily to Mailchimp for instance, a SMB customer paying $50 (or even $500) per month for a CRO platform won't be able to operate it on its own and will churn. The problem with Optimizely pivot to Enterprise is not the pivot in itself, it's how they did it (I heard some horrors stories from people inside Optimizely, not sure if they are true, but one thing is clear - it did not go well).

Anyway, it's still an exciting field, from the technical side of things, there are still many innovations to be done and we hope to spearhead that at Kameleoon :-)

I hope the founder exited well. I remember working with him at Google when he was the PM for Google Optimizer. Nice guy.

I don't. The founders pushed people who trusted them to work hard and value the company over their own needs. While the founders and their friends were able to pull out equity, every one else was told that they had to wait their turn, which never came.

If Dan and Pete were nice guys, they would have take care of the employees that built their company. The employees wouldn't have been millionaires, but they might be able to scrape together a down payment for a house. And not like the mansion that Dan has, just something that would move them out of the fear of rents increasing and losing their jobs.

They used to be nice guys. I don't wish ill towards them, but they do not deserve a good exit with this.

Wasn't Dan or Pete forced to step down on account of gross incompetence? I recall there being some internal drama about their departure.

Haha, certainly no drama and no one forced us or even asked us to step down. I wrote a blog post about it: https://link.medium.com/mHgtiQ3bu9


If you don't mind answering, what's the strategy behind a hiring tactic like the one in the link above? It seems cartoonish and not grounded in reality....

Sure, I just responded because you asked so nicely. :)


Haha sorry for the snark but I really appreciated your follow up! We’re thinking about hiring at our company and your framing was useful to read.

Worked at a company that considered Optimizely for one of their possible A/B solutions. We went with another vendor, but I then ended up writing an in-house solution that took about a month [w/o analytics].

It is very easy to implement this in-house so long as you own the systems and don't outsource [too much].

I used their paid product once and failed to buy it two times after that.

They tried to move up-market and did it in an unreasonably difficult way in my opinion. It was easier to do business with ORACLE and then Google. The sales folks didn't listen, their proposals ignored our requirements, it was a mess.

I hope the staff got something.

Maybe I haven't worked in tech long enough, but am I the only one who has never heard of Episerver?

It's an old-school enterprise CMS. And a crummy one. It's done a pretty remarkable job of staying relevant all things considered. The current market leader in this space is Adobe who own AEM and bought Omniture years ago so they can offer analytics and A/B testing in one bill of sale. Episerver must be working to position themselves the same way.

Lot of big enterprises buy these kind of systems and pay through the nose for them.

Episerver for a while was one of the main competitors of Sitecore in the IIS/.Net Enterprise CMS space. Both were European (Swedish and Danish respectively)

I built sites with both back in the late '00 and while Sitecore was marginally better I don't miss working with either.

Adobe also bought Magento, while not as bad as Omniture, has it's own levels of hell.

It's one of the crappier CMS startups from the early 90s that somehow survived. Mostly known in Scandinavia.

I/we competed against these guys 21-23 years ago. We had real tech, they had brilliant powerpoint engineers.

I have no idea why they're buying Optimizely.

I only know of them because they acquired Ektron CMS. Which to this day is the worst CMS I have ever had the displeasure of working with.

As someone not too familiar with the ins and outs of acquisitions or IPOs... is it unusual to get acquired after laying off a big chunk of your staff? Is that an indicator that they probably accepted a lower valuation than they would have before that layoff?

Not a rule of thumb but undisclosed acquisition prices are many times an indication of a poor exit.

I worked for a startup that basically let 90% of its engineering team quit due to low morale over the course of a year without making any effort to (a) stop the exodus, or (b) replace them. When they exited about 6 months later, it became pretty clear that it was intentional.

Sorry for my daftness, but what is the incentive behind intentionally losing a bunch of staff before 'exiting'? Just to reduce your cash burn?

Mostly to get better profitability numbers to show to potential acquirers

Exactly. A small number of us were offered packages to incentivise us to stay (ironically we still left), but the vast majority weren't. Which led us to conclude that the plan was to just keep everything afloat long enough to sell.

It can depend. Valuation is based on SaaS business metrics like Cost to Acquire Customer, Average Contract Size, Average Lifetime, and Margin.

Reducing headcount or doing a hiring freeze is a way to improve your metrics ahead before you go shopping yourself for an acquisition.

I worked at Optimizely for 2.5 years and left as recently as February. I'll be grateful for the experience of selling experimentation into the European market, even if Optimizely was on a flat trajectory. It is one of the most difficult things to sell. The problem with experimentation is you are selling hard work. Everyone says they want to be data-driven, but no one wants to do hard work. No one wants to tell an executive he/she is wrong and their idea sucks. No one wants to do the work of building a culture. It is akin to selling religion.

I'll be most proud of the companies I sold to that then started doing internal webinars on how to do hypothesis-driven product development and really engaging in bottoms-up approach to experimentation. Those deals took 12-36 months to close, but you felt like you were changing how they ran their business.

I see a lot of folks throwing shade on Optimizely here. I think that is ridiculous. Not many companies can grow to 300 employees and over $75 million in annual revenues. Even fewer companies get to IPO. Everyone's thoughts about how Optimizely used to be worth more are BS. Valuations are perceptions of product/market strength rather than the reality. We shouldn't be looking at this as a failure. Rather we should look at this as a natural course of a business finding its product/market fit.

Form a market perspective, Optimizely was hemmed in by big players like Adobe and Oracle who bought competitors to round out their marketing suites. Adobe and Oracle do not care about their experimentation products and will give it away for pennies, so it isn't like they are driving innovation. On the low-end, Optimizely created a bunch of fast followers. VWO, ABTasty and Kameleoon, just to name a few copied not only the Optimizely tech but also their marketing in some cases and then just sold for cheaper. Ultimately, the market decided that Experimentation technology just wasn't super valuable (you still need to generate the test idea yourself). The technology could be copied easily and many people didn't care enough about it to change their cultures.

Outside of the market forces, Optimizely definitely made some mistakes. You could make the argument that pricing and product development had issues in the past few years.

I think those were not the core issues. I think the basic mistakes they made were in execution. It is less sexy to talk about but ultimately if you have a great product and a need in the market, you still can't win if you don't execute properly.

Marketing execution was poor, Sales enablement was haphazard, Partnerships with other tech firms or agencies changed almost daily, and culturally, the organisation struggled to maintain the original culture it had around transparency. It failed to realise that competitors had caught up to a bunch of core functionality it thought was unique. It failed to market the new features that were differentiable. It is easy to blame pricing or moving into "enterprise" as the issue because that is what people see on the outside. I think that is unfair. Many companies have done the same thing and it worked. It is all down to execution.

Ultimately, I think Optimizely built a real market and culture in the tech world that experimentation matters and data matters. The fact that you have a ton of copycat companies prove that others see it as valuable as well (I wonder how they'll fair in the future). Regardless of where it goes as part of Episerver, it will have brought loads of people into the world of experimentation, and I think that is a good thing for the world.

Agree with Paras there, VWO was not a follower, they started at the same time. These things are actually funny - I even started Kameleoon before Optimizely (early 2008), but we developed our visual editor first and did not apply it to AB testing at the beginning. We made a (successful) pivot at the end of 2011 and have been focusing on CRO since, becoming competitors to Optimizely & others.

Calling Optimizely's competitors simple followers is clearly an over-simplification and a bias from the VC/SF crow indeed. As Paras pointed out, every actor in the field had their firsts in the market (some of Kameleoon's examples: our anti-flickering technology, our aggregation features in our reports / data engine, our automatic cross-device history reconciliation or our full ITP support). In many ways, Optimizely tech is now trailing behind today.

However it's true that Optimizely (and to a lesser measure, VWO) did a lot from the marketing side to develop the market, and we benefited quite a bit from that. We're definitely grateful to them :)

Just to be clear, VWO wasn't and isn't a follower.

We started in late 2009, exactly the same time when Optimizely started and we had visual editor on day 1 and have many firsts in the market to our credit (integrating heatmaps into A/B tests, asynchronous code, Bayesian statistics and so on).

It's a perception among VC/SF crowd that Optimizely was the only innovator in the A/B testing market but that's not the truth if you actually put effort into researching. Many players, including VWO and of course Optimizely too, pushed the market in terms of innovation.

-Paras (founder of VWO)

Any alternatives to Optimizely?

https://vwo.com/ is closest and better in a lot of aspects, check more here : https://vwo.com/compare/optimizely/. Full disclosure - I work for VWO

Kameleoon (disclosure: I am the CTO & founder). Close to Optimizely in the sense that it has enterprise level features. Our main focus at Kameleoon is performance / speed, at every level (loading of the JS engine, DOM manipulation, tracking, server side testing and/or feature flagging). But there are lots of other alternatives as well. As always, it depends on what exactly you need :-)


Dynamic Yield

Adobe Target

Shot themself in the foot. RIP.

Sounds like they're having financial trouble and needed to sell. I'm hearing this will likely result in nothing for employees.

If you're going to downvote please explain. Replies after mine with inside information agreed with me.

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact