
Show HN: Proofville, A Competitive Editing Game – Crowdsource Your Proofreading - lmeyerov
http://proofist.com/party.html
======
gojomo
Related: the Soylent Word Processor ("It's got humans inside!") from the MIT
CS & AI Laboratory:

[http://projects.csail.mit.edu/soylent/](http://projects.csail.mit.edu/soylent/)

~~~
lmeyerov
Definitely! Also, there are a couple of wonderful followup projects that think
about how to make crowdsourcing affordable (each task still costs money):
[http://www.cs.berkeley.edu/~kuangc/](http://www.cs.berkeley.edu/~kuangc/)

------
ivan_ah
This is very interesting. Proofreading as a service could be used by so many
content companies out there.

I guess for something long like a book it would make more sense to have the
_same_ person proofread the entire thing, but for blog posts and other text,
parallelism could be very powerful.

I'm not sure how the pricing will work out though:

    
    
       $0.004/w  = $1.4/page = $350 for my book
    

this is 1/6 of what I paid to my copy editor. Who will work for such a low
wage?

Last but not least, does it all have to be through-the-web. I find I catch a
lot more mistakes on paper.

Either way, I wish you good luck with your business and I will be following
the developments closely.

~~~
lsb
Sometimes copyeditors fix really subtle problems, sometimes copyeditors fix
really obvious problems.

This should help fix the obvious problems, and make their time better-spent!

~~~
lmeyerov
Right, looking at crowdsourcing (the main proofist site) was born out of my
frustration that _most_ copyediting comments are for simple mistakes that
anyone could have caught. I want to make ours cheap enough that you can get
10-20 pairs of eyes per piece of text and without the usual wait.

This game is an experiment to make it free, and if we get mass, can get the
desired level of matching. (In just one hour, we're already a hit with
educators in math/science!)

------
vitovito
I've been working on my own mturk-based proofreading system for OCR scans,
modeled after Ryan Tate's typingpool. This is very close to a web-based
version.

Your illustrations suggest you chunk content to mturk at the paragraph level,
is that correct? Did you try it at other levels of granularity, by the
sentence, word, or N number of words? If so, what were the results? If not,
why not?

How did you settle on 4¢/word?

Are you considering a mobile app/FB version of Proofville?

What's the back-end? Is it your own system, or is it an existing crowdsourcing
infrastructure like PyBossa?

How many reviewers compare a passage? 1? 2? Best of N until there's
consistency?

~~~
lmeyerov
A housemate (linked below) runs
[http://captricity.com/](http://captricity.com/) \-- crowdsourcing OCR is a
_really_ good idea.

The collaborator for the main site used to work @ cap, and we've been evolving
a REST API to simplify calling into mturk.

For quality, we don't use arbitrary mturkers. Getting good results on mturk
takes a lot of experimentation: I now think about it is as a combination of HR
agency, traffic conductor, and street vendor. The challenges of building a
useful & sustainable app on it are a bit different from what's addressed by
crowdsourcing research papers, though that's not so surprising in retrospect.

Sounds like this deserves a blogpost. It's been a fun summer project.
Basically, nodejs with roll-our-own crowdsourcing algorithms, APIs, and UI
widgets. A lot more to do, but it's finally good enough to address my
proofreading needs, and I'd like to help others who also need to get some
writing done asap :)

Btw, we're $0.004 / word, not $0.04 ;-) That means we already cost 60% less
than shops with comparable quality and slower turnarounds.

~~~
vitovito
Yes, please do! I can only begin imagine some of the issues, having fed a
couple thousand minutes through typingpool, but there isn't a lot of writing
about using it at scale. I'll look forward to it.

------
cdcarter
Is there a way to proofread something without having to upload your own text?
Now I'm in the mood to edit some copy!

~~~
lmeyerov
There is for the main site. I like the idea of different matchmaking modes for
the game. A player suggested allowing more than two reviewers working together
in a session, which may be one fun way to do it!

------
svetha
I absolutely love this. This is about way more than spell and grammar check.
It is not just proofreading. Now the crowd can be a test audience to help make
content more readable and digestible. I hope this works out!

------
jnbiche
If this takes off, an API would be great, allowing people to integrate this
into WordPress plugins, etc.

------
ams6110
Spell and grammar check have been part of any decent word processor for quite
a while now.

~~~
thatswrong0
But they can't deeply check style, clarity, or other such qualitative aspects.
Word won't tell you when your sentence is difficult to read or confusing.

~~~
lmeyerov
We've caught a lot that word processors haven't, should make a gallery :)

------
relate
Tried it out, works well!

