

Ask HN: Review my new A/B testing and analytics startup - oni0n56

Hey everyone, I've spent the last year designing and developing a tool for running A/B tests and keeping tabs on site traffic. It's just launched and live at http://getcharmed.com/<p>I’d love to get your feedback and help improving it and making it as awesome as possible.<p>Some of the things I’m most excited about:<p>• Sharing. I’m hoping it does for analytics/testing what github did with code. Here is what the permissions look like for one of my sites http://cl.ly/0I1N3P0f3g451b3v0W2L Let me know if you want me to add you to it as well!<p>• Goals. It has the basic “conversions” like visiting a page, clicking link. but also magnitude-based measures like time, pages viewed, scroll depth, which can all be tested at the same time to get a real accurate picture of how changes to the site affect user behavior.<p>• The traffic panel. Instead of having every possible measurement under the sun, my goal here was to have a very easily digestible representation of what is happening on the site. This means things like using the favicon and page thumbnails for easy recognition, parsing the URLs easy readability, and having a visual representation of the weight of all the numbers. Here is what it looks like one of my sites http://cl.ly/1D1f0J3N2T0A2X0l111V<p>This is just the tip of the iceberg, there are a lot of features planned or even just commented out for launch, but I really want to nail the basics first and make sure they are as useful and practical as possible.
======
jeffmould
Looks great! Congrats on your launch. A few things that stick out for me:

1\. I like the free option, and would most likely start there to get a feel
for everything. But...if I wanted to start with a premium plan instead, there
is not enough information on the website to tell me how this product is
different (besides look/feel) from products such as Google Optimizer, Google
Analytics, or Optimizely. Sell me on the benefits of using your product over
these other products (in particular Google since it is free for all levels).

2\. This is a little pet peeve on my part so you'll have to excuse me, but fix
the transparency on the signup button background. Right now it stands out and
looks thrown together compared to the rest of the site.

Overall though, the layout of your site looks nice and the screenshots you
provided look great as well. Looks very user friendly. Can I suggest maybe
creating a video for the website that walks people through the system or even
adding more screen shots?

Good job and look forward to hearing/seeing more.

~~~
oni0n56
Thanks! I think I just fixed #2, I am guessing it was a Chrome/Win bug with
rounded corners. Though I can't see it myself.. is the problem still there?

~~~
jeffmould
Yep, all fixed now. Looks good I'll give it a try when I get my site launched
and let you know.

------
whalesalad
Here is a link for those interested :) <http://getcharmed.com/>

------
iampims
The site looks great. When I tried to add a site, the link for the js to be
included was missing the scheme (http|https):

    
    
        <script src="//usecharm.com/xx.js" type="text/javascript"></script>
    

If we could have a restricted live demo, or a video to get a feel of how
things work before needing to get some fair amount of data, that’d be even
better.

Great job and good luck with it.

edit: apparently // is the way to go, see comments below.

~~~
aonic
Not including the http|https tells the browser to use the current page to
figure out if the file should be loaded over http or https. Loading http
content if you're on a https page will trigger non-secure content warnings
with browsers

~~~
iampims
Thanks for the tip. I assumed we’d have to manually add the scheme to the url.

------
tlack
I think this is a great business to be in right now.

As others have said, I'd include screenshots and details about the
_optimization process works_ , not the monitoring bits and referral tracking -
after all, most people already have a very involved process they use to do
that.

Design wise it looks good, but the signup button transparency is distracting.

------
klochner
You're wasting space on the homepage in explaining the benefits of a/b
testing, you would be better off devoting more content to how your product
works.

I want to know what's involved in setting up and running a test, but came away
from your site without really knowing what the user experience is like.

~~~
bravura
I assume he's actually A/B testing whether explaining about A/B testing helps
his conversion. If not, the irony is killing me.

------
javery
The first thing I thought of when I got to the site is that this looks like a
bought template. I don't know if it is or not, but that's not what is really
important. It feels that way because it looks like a case of the "lorem ipsum"
curse - when you have to fill out an already designed page and just had to
keep adding stuff. The Clarke quote adds to that impression, if you don't have
a quote about your product does that mean you don't have any beta users? Or
were none of them willing to be quoted? Then on your signup page you have
Voldemort and Dumbedore providing quotes.

Also, your twitter feed isn't very re-assuring on the infrastructure side.

Overall it looks like an interesting product though and something I might
consider using.

------
animeshjain
why three domains - getcharmed.com, charminc.com and usecharm.com. seems a
little unusual and might even be confusing.

edit: looks pretty neat on the UI

------
dflock
Okay, signed up and went a bit deeper into the site:

* You've got the same rounded-corners-not-transparent issue at the bottom of every page

* Only when you get to the 'copy this bit of JS into your site' stage, do you find out that it _also_ requires jQuery. I know a lot of people will already be using jQuery (including me) bit the site I wanted to test with isn't using jQuery, so this has moved me from 'Go', to 'Stop and think'. I wish I'd known this earlier on.

------
SupremumLimit
I like the landing page and the screenshots look nice, but I would appreciate
an explanation of how A/B testing works from the technical standpoint. How do
I set up a test? What if multiple pages are involved? Will returning users get
the same page consistently?

It's definitely an interesting service though, and I'll be happy to pay for it
if it saves me time compared to setting up my own A/B testing framework.

------
thenduks
Looks very nice, design-wise -- unfortunately I can't actually try it out
because I'm not willing to put your javascript directly on my production site
without experimenting first. I get that your app needs access to do it's
thing, so I'm not sure how you'd fix this... but it's a real stopper for me.

~~~
oni0n56
Well you could install it on a development site or something first. You can't
currently add in a domain that's not live (since it verifies its existence by
pinging)— but contact me and I can add it in manually and you could try
running it locally or on a private site.

~~~
thenduks
Yea the ping-ing thing is what stopped me. I need to test it as un-live as it
gets, on my local development instance, before I start pitching this stuff to
my boss.

I'll shoot you an email. Thanks. _edit_ : Or, I would, but you don't seem to
have it listed in your profile.

~~~
oni0n56
Try support@charminc.com — Will update my profile now.

------
mmelin
I like how your support options progress from "Online" to "CEO's iPhone" :)

I'd like to read more about how it works, maybe see an integration guide, but
it seems you only have the one (beautiful) page right now?

~~~
oni0n56
Thanks. Yes it is only one page right now. I had originally planned a whole
site with different sections and pages but it seemed a bit wasteful since the
current integration guide is: • have jquery on your site • add a line of code
linking to your JS file

But behind the scenes how it works is that JS file (ie. usecharm.com/mmelin.js
or something) returns different code depending on the cookie of the visitor
and the configuration of the project. So if you launch a test, the goals and
changes will come out in that file and it is also based on what page it is
requested from, so the change could be different when requested from the
homepage of your site vs a content page. By default it just has basic
analytics stuff like referrer-detection and measuring demographics.

Do you think I should add a "behind the scenes" or similar section that
explains all the technical stuff going on?

~~~
dflock
Yes. Currently it looks great, but nowhere does it actually explain what it
does - in specific technical terms.

------
gsiener
This looks great. Just signed up, I'm very interested in trying it out. Shoot
me an email, I'm happy to talk through your roadmap and give feedback.

------
oni0n56
Design was done by Dragon Interactive ( <http://dragoninteractive.com> )

------
aonic
Love the user interface screenshots. What did you use for the graphs? Custom
library?

~~~
oni0n56
It's pretty much all CSS-based at this point. Javascript is used for tiny
things like the centering of the confidence intervals.

We were looking into some of the canvas libraries out there but that really
only becomes necessary when doing pie-charts and other non-rectangular shapes.

------
parasctr
jQuery requirement doesn't suit me because I am currently using YUI. Are you
planning on making YUI version available? If not it would be nice to have a
version with no framework dependencies.

------
petewailes
I'll deploy next week on a site and let you know my thoughts

------
kqueue
off topic, I like the design. Can I get in touch with the designer?

------
ninjastar99
This looks GREAT!

------
lenary
I _love_ free wood!

