We've seen this with our site. The difference between getting a personalized feedback message and not is even more pronounced than the 62% cited in the article. And similar to the findings reported, the more a person has contributed the more likely they are to contribute into the future. Though for us, there is a dip in the middle that we're trying to resolve.
We actually came up with what we call an "engagement" curve, which is the PERCENTAGE of people that will contribute at least X+1 times given they've contributed X times to the site.
Here's the rough outline: http://s5.tinypic.com/2db9vyw.jpg
X-axis is number of contributions. Y-Axis is percentage of people that have contributed at least X+1 times for the given X.
So we can segment that out with various tests to see how it affects the engagement curve. For example, how does the curve shift when a user gets a personalized message? or a generic e-mail? Or does contribution type A, etc etc.
Very interesting post. I've actually been following that same exact idea - I check over the error logs to see if any users encountered an error, email them an apology and let them know I'm working on it right now, and then email them once it's worked again thanking them for their patience.
Also, for questions which have been posted, I email the author (if they've given us their email address) and ask them if they're satisfied with the answer. If not, we'll spend some more time researching it and contact some special authority.
We've done that for one week, but the results have been mixed. I still think it's critical, but it's only one factor - you need the initial userbase just to get the new users to post.
EDIT: It also occurred to me (thinking about to pg's "If you measure it you'll improve it" quote") that I'll need some substantial reporting as well. To that end, I'll be writing some rake tasks today so I can plot the data over time.
Post your 'mixed' results. Does your site offer the possibility to test on peers (see my post below on the main thread.) I'm interested in graphing the result of this.
Our web site sends registration information directly from my cofounder's email address. (We don't like private messaging systems.) The plan is to start really personal conversations with all the people who sign up, until that gets too overwhelming.
Just as a test, do this: Pick 10 random users who sign-up and start a co-founder conversation. Pick another 10 random users and send an email saying "A member has replied to your message" or "A message was sent to you from Michael". See which is more effective.
You are discounting a very important factor - authority. Your co-founder is an authority figure, he's selling a site. He belongs in another group as the signed-up user. Another user sending a message is a peer. He tells the user that there are other people there, and that they are interested in what he is doing.
There is a big social difference in the two. Don't do one and never test the other.
We actually came up with what we call an "engagement" curve, which is the PERCENTAGE of people that will contribute at least X+1 times given they've contributed X times to the site.