

37signals report card - wlll
http://37signals.com/reportcard

======
kyro
The specifics of the data are not important here. 37s has a real knack for
creating a continuous stream of positive buzz using status updates like these
in addition to their blog posts. It's less about their average customer rating
or whatever increasing by 3% this year, and more about knowing the company is
still alive and doing well with a good product and happy customers. What's
particularly smart is that, excluding their books, their articles and reports
aren't long or comprehensive, but have the minimal amount of information to
offer enough value to keep them relevant and in a positive light -- like when
you kick your feet periodically in water to stay afloat. It's not easy by any
means. I think it's quite clever.

~~~
chaz
It's also a marketing moat because it raises questions for customers when
competitors don't have similar report cards -- what are they hiding? The low-
pressure copy and honest voice are a big help, too. Very clever indeed.

~~~
nemesisj
I think transparency is a huge competitive advantage. And like you said, it
works great as a marketing tool AND as a tool for accountability. There are
several ways you can do this but probably the easiest is to just sign up for a
service like Pingdom (which is great b/c it injects a 3rd party measurement
into the equation) then use their status page tool. We did ours using a
twitter widget, pingdom's status page, and some CSS, and I think it turned out
OK: <http://www.getadministrate.com/status/>

------
smg
I am not sure if average page load times are a good way of measuring site
performance. I wish they published numbers like median, 75th percentile and 95
percentile load times.

Please see here for why average is not a good metric.
<http://zedshaw.com/essays/programmer_stats.html>

It is especially dangerous for 37signals to do this because a large percentage
of web developers look up to them for best practices.

~~~
beat
More numbers == harder to read. The point is not to give an accurate picture
(as someone else pointed out, they certainly have more detailed internal
metrics). The point is to make potential customers feel that their apps are
fast and responsive. Hard numbers make that sound plausible. More detailed
numbers would make the point diffuse and confusing.

~~~
smg
I am not sure if either of us can accurately say what is the point "they" are
trying to make. The point I was trying to make is that a large number of
developers who look at 37signals for technical leadership could be dangerously
misled into using average as a performance metric.

Leaving aside the technical issues even from a customer point of view using
average to measure performance is pernicious. There are scenarios where the
average remains the same while almost half their customers are experiencing
load times that have doubled. What should the report card say in that case?

------
ruswick
" _92% gave us the highest Customer Service rating._ "

This seems somewhat disingenuous, considering that the rating scale was only
out of three, and the "highest" level is used to show any satisfaction
whatsoever. Their copy implies some sort of rigor to their rating process, and
suggests that the "highest rating" is somehow significant and indicative of
inordinate quality, as would be implied by a five or ten out of ten. A three
out of three when the bottom two ratings indicate dissatisfaction or
displeasure isn't really saying much. I'd think that it's fair to assume that
the majority of most customer bases like the product in some way. A more
realistic description would be to say that "92% of our customers were pleased
with our service."

It's like saying that 92% of students in a pass/fail class earned the highest
grade. It might technically be correct, but it doesn't mean anything.

------
ishansharma
While there has not been much downtime, I have noticed a big spike in their
Twitter account apologising for some bug or other.

In fact, for last month or so, every second and third day, I see a tweet from
them in my timeline saying "Sorry for disruption in ________service of
<product>"!

------
thecosas
They continue to be a great example of a company that tries to communicate
when something is an issue (or is going great).

It seems like the critics of which metrics are highlighted are missing the
point... the average person who uses their products or plans to sees a company
who is willing to communicate how they are doing.

If you need to dig into more detailed stats, they're not doing a good job with
how their service performs. The whole point of their services is so that you
don't need to worry about those things :o)

------
lucasjans
I think one useful metric to track would be similar to the stock market's
"52-week high." As an end user, comparing to the last month isn't very useful.
But knowing, is this the top mark they've had in the last 12 months? That
provides richer context.

------
olegp
At <https://starthq.com> we are working on producing similar report cards for
all the web apps out there. For the web app developers out there: is there any
other information that you would like listed?

------
ffffruit
Good to see a mix of quantitative and qualitative measurements in there. While
uptime and customer satisfaction are probably correlated, there are probably
more factors influencing the latter.

------
adem
They're making a bigger deal about their company than their products. Am I
missing something?

------
crunchcaptain
I thought this might be a survey or request for users to fill-out a report
card, not an informational page. We tried Basecamp once. It seemed like an
expensive implementation of something we could easily setup ourselves. Not
sure why everyone is so 'ga-ga' for 37signals...

~~~
timjahn
It's much easier to run your own Basecamp than pay $20/mo for it.

All you have to do is:

\- maintain your own server

\- design the interface

\- write the code

\- test the code

\- fix bugs on a regular basis

And that all easily costs less than $20/mo.

------
sdfjkl
Which NMS is tracking these numbers?

------
uforgot
That paperclip annoys me too much to read the page.

~~~
andyhmltn
That's true. Looks so much better without that background.

~~~
uforgot
Can't they remove it? Then we can all pretend it's a postit note.

