Plus, bootstrappers should support other bootstrappers, eh?
As for a comparison, Optimizely uses Akamai which has 100,000 nodes in 75 countries. VWO uses Cloudfront and does two round-trips. Cloudfront has 32 points of presence in 12 countries.
Optimizely response time: Slowest: 289ms, Average: 65ms, Slowest: 19ms
VWO response time: Slowest 1983ms, Average: 276ms, 171ms
Unique features to Optimizely that VWO does not have:
- Mobile web support & editor
- Dynamic text insertion
- Staging server support
- Human-readable results
- Documented API
- Advanced developer tools
- Cross-browser testing for Chrome, Safari, Firefox, IE, iOS, and Android
- One-click integrations with Google Analytics, Omniture SiteCatalyst, KISSmetrics, MixPanel
- Implementation is 1-line of code vs 21 lines of code
- Pricing starts at $19/mon vs $49/mon
- Optimizely comes with a 100% money back satisfaction guarantee with 99.999% service level agreement
- Optimizely support offers web, phone, email, and a dedicated account manager, as well as the CEO's direct cell for Platinum customers
In terms of adoption by the top 10,000 websites on the internet Optimizely is 6x bigger than VWO.
Note: I co-founded Optimizely and used Google Website Optimizer, Omniture Test&Target, and VWO but by no means an expert in their products.
I am a bit confused.
- 100% money back satisfaction guarantee
Your Terms say (https://www.optimizely.com/terms)
- 4.B. No Refunds. You may cancel your Member account at any time; however, there are no refunds for cancellation. In the event that Optimizely suspends or terminates your account or this Agreement, you understand and agree that you shall receive no refund or exchange for any Optimizely Property, any unused time on a subscription, any license or subscription fees for any portion of the Service, any content or data associated with your account, or for anything else.
Also couldn't locate the SLA on your site, can we have the link?
All plans come with a 30 day FREE trial and a 100% Satisfaction Money Back Guarantee.
Our SLA is part of the Mutual Services Agreement contract we have with customers.
Is this SLA done individually with each customer after they signup?
Plus, you are wrong about almost all of your points:
- We do have a well documented API. Many customers use it on a daily basis to integrate VWO into their systems. http://visualwebsiteoptimizer.com/split-testing-blog/announc...
- Our 21 lines of code is unique as it provides an asynchronous fail safe method. With 1-line of code, websites still have a theoretical single point a failure where code is fetched from your servers/CDN. If your code isn't able to get fetched for any reason, standard timeout of browser (30 seconds) applies and since it is synchronous page load stalls. With our asynchronous approach, we completely decouple the dependence (even if it doesn't look elegant, it certainly is much preferable to our customers). Here are the details: http://visualwebsiteoptimizer.com/split-testing-blog/asynchr...
- I'm not sure what you mean by Human-readable results. Certainly we have some great looking reports that our customers love. 50+ case studies on our websites is a testimony to that http://visualwebsiteoptimizer.com/case-studies.php In fact, one independent website that tracks UI from color blindness perspective says we have one of the best charts out there http://wearecolorblind.com/example/visual-website-optimizer/
- Our tests run on mobiles as well, a lot of our enterprise customers regularly test their mobile landing pages
- We have one-click integration for Google Analytics, and have plugins for KISSMetrics, SiteCatalyst, etc.
- ALL employees of Wingify (including me the CEO and our CTO) do support for all levels of customers (not just enterprise ones). In fact, support is bed rock of our offering. My number is there on our about us page http://visualwebsiteoptimizer.com/about-us.php
According to best of my knowledge, apart from cross-browser previews I do not think there is any feature that VWO lacks. You may want to cross-check your facts. Our backend delivery infrastructure prioritizes on speed and fall-back rather than elegance of how the code looks.
However, you did not mention the features VWO has that Optimizely currently lacks:
- Usability testing. Our users can get quantitative feedback and improvement ideas on what they should be A/B testing http://visualwebsiteoptimizer.com/split-testing-blog/geo-beh...
- Heatmaps and Clickmaps to show where people are clicking on their pages. Our customers LOVE This feature and some solely use VWO for its heatmap functionality http://visualwebsiteoptimizer.com/split-testing-blog/ab-test...
- Enterprise security options such as whitelisting office IPs for login and realtime activity notifications to administrator http://visualwebsiteoptimizer.com/split-testing-blog/hallowe...
- Free 30 day trial without requiring a credit card.
- Numerous other small features that I won't go into (test labels, test organization, frequently used goals, roles/permissions, account management, reports exportable to CSV/PDF, etc.)
I'm glad to see you are focusing on improving your response time. Unfortunately hitting live servers serving dynamic content in 5 locations is never going to be able to beat 100,000 edge-cached nodes in 75 countries serving static content when it comes to response time and reliability.
According to the link you provided it looks like Pingdom reports VWO response times as:
VWO Slowest Average: 352ms
VWO Overall Average: 207ms
VWO Fastest Average: 45ms
According to Pingdom, Optimizely response times are:
Optimizely Slowest Average: 95ms
Optimizely Average: 65ms
Optimizely Fastest Average: 19ms
Here is a head-to-head comparison of Optimizely vs. VWO including response time and feature differentiation: http://i.imgur.com/BRTxC.png
To clarify some of the points you might have misunderstood:
- Here is a demo showing the Optimizely visual mobile editor: http://www.youtube.com/watch?v=HucXw_egPRs
- Documentation of Optimizely one-click integrations: http://support.optimizely.com/customer/portal/articles/58188...
- The Optimizely API is publicly available at: https://www.optimizely.com/docs/api
You are right we don't currently offer ancillary features like heatmaps and usability testing. We've been focused on offering the best possible experience for A/B and multivariate testing.
@paras: Your response times do not look good. Why are you using a homegrown pseudo CDN instead of a real one like akamai or limelight ? Also why not make your API docs public ?
Reason API doc is not public is because API is on demand right now, but anyone can use it. They just have to request for a key from us.