

Kids these days: the quality of new Wikipedia editors over time - vgnet
http://blog.wikimedia.org/2012/03/27/analysis-of-the-quality-of-newcomers-in-wikipedia-over-time/

======
redthrowaway
It's not even a new user issue. I've been an editor for going on 5 years, and
I still get templated and reverted on inconsequential edits.

Case in point: I recently edited this picture:
[http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.pn...](http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.png)

It's a screenshot of the 6 octillionth generation of a Turing Machine built in
Conway's Game of Life, using the GPL'd program Golly. The previous image
contained Windows GUI elements (title bar, etc) which are non-free and cannot
be used on WP when a free alternative exists. So I cropped it and uploaded the
edited image.

Within minutes, I had received a template on my (seldom-visited) commons talk
page informing me that I had uploaded a file without specifying its license,
and that it would be deleted. Take a look at the "Permission" field on that
image: "See LICENSE.TXT distributed with Golly for GPLv2 license"

Despite this (incredibly clear) assertion that the image was GPL'd, I received
a warning that it would be deleted. Why? Because I hadn't included the "This
image covered by the GPL" template that a) I didn't know existed, b) there was
no mention of on the upload page, and c) is a wordier version of what I wrote
in the license field.

As an experienced editor, I'm used to these stupid quibbles and time-wasting
fights. I'll still contribute, although they are a large part of why I don't
contribute more. As a new editor seeing this, however? I would have told them
to fuck off, got banned for incivility, and never gone back.

There's an attitude among the regulars that Wikipedia is a treasured resource
that must be defended against innumerable vandals, trolls, and spammers by a
select cadre of noble volunteers. To an extent, they're right. But when you
have such badges as "The Defender of the Wiki Barnstar" [1] being held up as
the height of achievement for veteran editors, it engenders a culture that is
exclusionary, if not actively hostile, towards new editors.

[1] <http://en.wikipedia.org/wiki/File:WikiDefender_Barnstar.png>

~~~
simonbrown
Are you sure showing title bars isn't allowed? Almost all screenshots on
Wikipedia include them, even for open source software. e.g.:

<https://en.wikipedia.org/wiki/File:Firefox11.png>

<https://en.wikipedia.org/wiki/File:Winscp_screenshot.png>

~~~
redthrowaway
I've had images with them included deleted before, which is why I modified the
golly screenshot. To be perfectly honest, I'm not sure _what's_ allowed, and I
doubt many people who are not active commons contributors do, either.

~~~
flomo
Some editors went through and deleted a massive number of useful Mac/Windows
screenshots several years ago, but I don't think there's ever been an actual
policy stating these screenshots were a legal issue.

Also, the GPL only covers code, not program output (under normal
circumstances), so I don't see how it applies to screenshots anyway.

~~~
redthrowaway
>the GPL only covers code, not program output (under normal circumstances), so
I don't see how it applies to screenshots anyway.

It's an interesting question. The drawing of the window is, I would argue,
indistinguishable from the code. It's like saying the HTML, CSS, and images on
a site are GPL'd, but screenshots aren't. The rest of the program output is
merely algorithmic, and thus not subject to copyright.

~~~
flomo
This discussion is wandering off into la-la land. The GPL isn't designed to
cover things like websites or bitmapped images. There's other 'free' content-
oriented licenses which should be used instead.

------
jonnathanson
I'm glad that they're finally coming around to the realization that Wikipedia
has become increasingly closed to new contributions, and that they've stopped
touting the (patently absurd) hypothesis that new users just don't "get it."
(The fact that they'd even think, let alone think _first_ , to blame the users
is just a giant head-scratcher).

As a simple UX experiment, I would ask new users this: try to contribute
substantively to any article on Wikipedia. Just try it. Make a good-faith,
high-quality edit to a page, and see how long the edit is allowed to stand.
More likely than not, the contribution will be automatically reverted, within
milliseconds, by a bot. If it's not, it'll be hand-reverted by a hardcore
Wikipedia editor -- part of the statistically small, but disproportionately
powerful cadre of self-appointed content cops, who seem to see their jobs as
being bulwarks against change. In its zeal for the trappings of due process --
attributions, anti-"vandalism" policework, source checks, guidelines, and so
forth -- this clique has lost sight of the net effect it's had on the site,
which is to calcify and close off the free exchange of information that was so
crucial to Wikipedia's early growth.

IMO, Wikipedia has faced a fundamental challenge in recent years: namely, that
content-quality efforts have threatened new content volume. I don't envy this
strategic predicament, being forced -- quite literally -- to choose between
quantity and quality. It's not an easy balance to strike, and, given the
circumstances, Wikipedia's historic track record is quite admirable. Recently,
however, the balance has tipped too far in the direction of quality-policing.
And now it's starting to undermine the core tenets of the project. I remain
optimistic that Wikipedia (and/or the Wikimedia Foundation) can right the
ship. But it'll have to mean a substantial uprooting of some bad seeds that
have been allowed to take hold for years now.

~~~
stcredzero
_(The fact that they'd even think, let alone think first, to blame the users
is just a giant head-scratcher)_

Logically, I can see this. However, when you survey the world, you see a whole
lot of blame directed at users. (With certain tech/business subcultures as
notable exceptions.) I'll note that when our brains evolved, there was no
software nor was there UX, and the only suitable targets for blame were people
and critters. I'd bet this is a human cognitive bias.

~~~
jonnathanson
Largely agreed, but if you'll permit me to get cute, I'd suggest that's
there's _always_ been UX. Whether in city design, the evolution of tool usage,
invention, product design, etc. UX is at the very heart of why we, as humans,
always seek to improve upon our lot.

The difference with software is that, for the first time in our history, we're
able to measure, isolate, quantify, and control the elements of UX better than
we've ever been able to. (It helps that software is often experienced in
isolation from its environment, so UI can be more closely correlated with UX
than it is for other domains). UX was a fuzzy, ethereal, probably subconscious
concept that only recently became a serious discipline. But it's always been
important. And our brains _have_ evolved to conceptualize it, albeit
intangibly until now.

~~~
stcredzero
_Largely agreed, but if you'll permit me to get cute, I'd suggest that's
there's always been UX. Whether in city design, the evolution of tool usage,
invention, product design, etc._

Permission denied. :) Everything but tool usage is in the realm of cultural
evolution. For the simplest tools, most all of the error is user error.
There's simply not so much functionality in a stick that isn't mostly
dependent on user actuation.

 _our brains have evolved to conceptualize it, albeit intangibly until now._

That's my point. UX flaws that are independent of user error have been largely
intangible until now.

~~~
jonnathanson
Define "cultural evolution," though. I'm not sure what you mean there. Not
trying to be difficult, because I rather enjoy this conversation. Just unclear
about the distinction you're drawing.

I'd argue that UX design -- even if it didn't have that exact name -- has been
a distinct discipline long before software. Just ask anyone in the food
service industries, the retail industry (department stores were basically
innovations in UX in retail; so was IKEA), the casino gaming industry, the
amusement park industry, and so forth.

Casinos, in particular, are fascinating UX case studies. The person who first
thought of modern casino layout, comping free drinks at table games,
oxygenating the gambling floor, removing clocks from the walls, comping rooms
and other amenities for big spenders and regulars, which games to place
adjacent to which others, etc., was a UX designer in spirit if not in title.
And those decisions were pretty rigorously tested and quantified. These things
may not meet the technical definition of UX as we commonly speak of it on HN,
but they certainly hold with the spirit of the discipline Don Norman would
later come to articulate as "UX."

~~~
stcredzero
_Define "cultural evolution," though._

As for a definition, I'm talking about the evolution of individual behaviors
as transmitted through culture. If one somehow rendered the entire human race
sterile, but we continued to propagate ourselves for the next 2000 years
through cloning, you'd still have "cultural evolution."

 _I'd argue that UX design -- even if it didn't have that exact name -- has
been a distinct discipline long before software._

Again, I don't disagree. That you bring this up indicates to me you've missed
my point.

 _food service industries, the retail industry..., the casino gaming industry,
the amusement park industry..._

All of these predate most of the evolution of the human brain's structure and
capabilities. It's somewhat true that there were "UX errors" before the stone
age. I say "somewhat" because it's really hard to delineate these as entities
without a certain degree of technology. When all you have are sticks, what is
the error of the designer and what is the error of the user? Maybe the user's
just "holding it wrong?"

It seems to me, that we're likely to assign blame to sentient and animate
entities. And even if the stick wasn't "whittled correctly" according to Thag,
maybe it's just fine to Ookla? It's just hard to talk about "UX errors" as
quantifiable entities until we get standardized production and large sample
sizes.

~~~
jonnathanson
If I understand your original point correctly, it's this: it is human nature
to attribute most things to user error, ergo, my assertion that Wikipedia's
blaming its users was a "head-scratcher" was off the mark. Our evolved
inclination, which predates the discipline of even _considering_ UX as a
tangible -- and, more important, a controllable -- concept, is first to start
with the hypothesis that the user is in error. (And, furthermore, that such a
hypothesis is not necessarily unjustified by historical frequency).

I actually agree with you here, but I think this point and mine are not so
much at odds, as they are orthogonal. My point is that, human nature or not,
Wikipedia came about in the modern era. Even if our cognitive bias/inclination
is toward blaming the user, we have tools and analytic frameworks at our
disposal which exist precisely to allow a necessary check against our brains'
heuristics. Those checks should have been run by the Wikimedia elite. While
I'll admit that "head-scratcher" is an unfair description, rendered mostly for
rhetorical effect, I believe my point still stands. We have modern tools at
our disposal, precisely because we are now -- uniquely, in our history --
aware of our brains' strengths and weaknesses in pattern recognition and
situational assessment.

If this is not an accurate summation of your position, then I'll freely admit
that I'm missing your point.

~~~
stcredzero
No, that's it. Strangely enough, we were basically agreeing the whole time.

------
tokenadult
Like a lot of people here, I have sporadically read Wikipedia for years. I
came on board as a Wikipedia editor in May 2010 after meeting the project's
co-founder and his family in person the year before. I've since seen some of
those immediate family members on another lengthy occasion. My children
regularly interact with that family in an online education community. Through
a web of mutual friendships, I thought I had some sense of what the community
norms would be like on Wikipedia before I started. Moreover, I began editing
only after reading the several published books about Wikipedia available at
that time (as disclosed on my Wikipedia user page), and came on board as
someone who has actually had both academic journal editing positions and paid
journalism editing positions before Wikipedia even existed.

Even at that, I get a lot of well sourced edits reverted by ideologically
motivated drive-by I.P. editors as part of an ongoing process of edit-warring
on articles that I happen to know sources for.

[http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/...](http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/Case/Race_and_intelligence)

For some controversial topics, no amount of good-faith editing by editors who
actually know how to look up reliable sources and how to have civil
discussions of controversial issues can overcome the flood of point-of-view
pushers (both I.P. editors and registered editors who are sock puppets or meat
puppets of previously banned editors) who want to drag down the project to
below the level of a partisan blog. There simply isn't any incentive in
today's atmosphere on Wikipedia for readers who actually know what
encyclopedias look like and who have actually engaged in careful research on
controversial topics to devote any of their time and effort to Wikipedia.

My number of edits per month has plummeted, and mostly I wikignome to clean up
copyediting mistakes on miscellaneous articles written by young people or
foreign nationals who didn't write grammatical English in the last revision of
the article. The way to increase participation by productive, knowledgeable,
literate editors is to drive away the ideologues and enforce some reasonable
behavioral norms on article talk pages and user talk pages. I see no sign of
that happening over at Wikipedia, and until I do, I will heartily support
anyone's effort to build a competing resource, either limited to a specialized
topic or a direct attempt to build a better quality general online
encyclopedia.

I think the "Lamest Edit Wars" page in project space sums up much of what is
amiss about Wikipedia.

<http://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars>

------
blahedo
It's good to see real data to address this question, rather than the
neverending stream of basically anecdotal information about the problem that
we've been slinging around. It's especially interesting to see the percent of
not just "good faith" (flawed but well-intended) edits that are reverted but
also the percent of "golden" (actually contributing) edits that are reverted;
this sort of hostile drive-by is discouraging even for experienced editors.

One thing I see in the graphs is that the "survival rate" of editors who made
a good-faith first edit was already in relatively steep decline by 2005, but
the same rate for editors whose first edit was golden remained on a high
plateau through 2006 and then just categorically dropped off a cliff. What
happened then?

~~~
unreal37
In 2006, bots were introduced to do various clean up tasks automatically.
There are currently over 800 registered bots on Wikipedia. They are generally
made to enforce the rules. But if you need 800 different bots controlled by
800 different people to enforce the rules, that means you have a lot of
obscure rules.

For instance, there is a list of domain names that wikipedia sees often as
spam. So if an innocent user goes to edit a page, and wants to add a new piece
of information referencing a domain on that list, a bot will come along and
mark it as spam (and revert it) even though its a good faith edit.

It can be frustrating and confusing for new users. The bot is often quite
terse and rude, and so there is no incentive to return to wikipedia to make
edits. 9/10 new edits are rejected, so no wonder they can't keep new editors.

------
nowarninglabel
"What this means is that while just as many productive contributors enter the
project today as in 2006, they are entering an environment that is
increasingly challenging, critical, and/or hostile to their work. These latter
findings have also been confirmed through previous research."

This confirms to me what I argued on HN last year:
<http://news.ycombinator.com/item?id=3272926> and
<http://news.ycombinator.com/item?id=3273204>

------
Klinky
NPR's Talk of the Nation recently had a segment devoted to the bureaucratic
run-around that can happen on Wikipedia.

[http://www.npr.org/2012/02/22/147261659/gauging-the-
reliabil...](http://www.npr.org/2012/02/22/147261659/gauging-the-reliability-
of-facts-on-wikipedia)

~~~
bane
This is a great example of my complaints elsewhere in this thread, thanks for
the link.

So listening to this show, there's an expert guest, a couple call-ins
(including another expert guest) and even a caller who talked about how
important the crowd sourcing is, who all more or less say the same thing I'm
saying, trying to contribute to wikipedia is a pain because of the overzealous
editor's arbitrary and capricious application of largely nonsenical rules --
the foundation representatives? Regurgitation of the guidelines and
nonanswers. Then lots of lip service to the importance of crowd sourcing an
encyclopedia without acknowledging that the crowd is turning into a very small
gang and "let's make new contributors feel more welcome" without any
particular ideas presented of how to do it.

------
Alex3917
This ignores the fact that the standard for what is considered a good edit has
risen dramatically over time. Most of the featured articles from the early
years of the site no longer even meet that standard. There are 3,500 featured
articles, and almost 1,000 that have become unfeatured as standards have
risen.

~~~
jessriedel
No, they used the same method to judge edits over the entire sample space.

> To test the hypothesis ... we randomly sampled the first edits of newcomers
> to the English Wikipedia from the earliest days of the project to the
> present. With the help of some experienced Wikipedians, we hand-categorized
> the edits of 2,100 new users according to a four point quality scale

~~~
Alex3917
That only shows that the edits have been roughly the same quality of time,
what I'm saying is that edits of the same quality are now judged much more
harshly than they used to be. For example six or seven years ago if you wrote
something that was true but unsourced, or else the citation was formatted
incorrectly, the text would generally be left up and a higher level editor
would get around to hunting down the proper citation eventually. But today
these kind of edits would get instantly reverted, even though they create
value. The issue is that because so much of Wikipedia is already properly
cited, adding new text that is correct but not properly cited temporarily
lowers the overall quality of the encyclopedia even though it would ultimately
create value if left up and fixed, so essentially what we have now is one of
those game theory failures.

This really isn't that hard to figure out and all they'd have to do is
slightly tweak the design to fix it, so I don't really see why they're having
so much trouble on this one.

------
nwj
Is it possible that the opportunities to make quality edits has decreased as
wikipedia has matured?

What I have in mind is the possibility that topics that haven't been well
written out are like low-hanging fruit and are easy to positively contribute
to. As wikipedia has expanded, that low hanging fruit has disappeared to some
extent, and thus new editors have fewer opportunities to actually provide
quality edits.

This explanation does not place blame on either newbies or pre-existing users
who are "content cops". Instead, it proposes that the difficulty of providing
a "golden" edit increases as we move forward in time. If that's a problem
(debatable, since it means wikipedia as a product is better than it was
previously) then it strikes me as a difficult one to solve.

~~~
Symmetry
Those were basically my thoughts before reading this article. However, while
it might be that its more difficult to create a good edit, that still
shouldn't lead to more of the good edits that are made being reverted as the
data shows is happening.

------
jes5199
I was surprised to see that “Assume good faith” was a principle of Wikipedia -
it doesn't match my experience with the community at all.

------
RobertKohr
There is a set of knowledge in the world. It is vast, but much of it has
already been contributed to wikipedia. The knowledge of the world continues to
grow, but I don't think at the rate that it is contributed to wikipedia.

Contributors are really mining the collective knowledge of the world, and so
they are running out. So similar to mining an nonrenewable resource, the cost
of extraction goes up with time:
[http://desmond.imageshack.us/Himg521/scaled.php?server=521&#...</a><p>With
difficulty in making significant contributions dropping off, you would expect
to see a drop off of new users (they really don't get rewarded enough for
their efforts). Back in the day, you could contribute something meaningful
that you know to wikipedia. Most likely now, someone has already put it in.

------
bane
Thanks wikimedia for once again providing a detailed analysis of something
that's been absurdly obvious for the last 5 or 6 years - "However, the rate of
rejection of all good-faith new editors’ first contributions has been rising
steadily, and, accordingly, retention rates have fallen."

And so the conclusion? "At the Foundation level, this includes major software
changes like the creation of a visual editor to lower the technical barrier to
entry..."

Which completely doesn't address the human issue of entrenched editors who
reject perfectly good edits by new users and delete perfectly good content.
_Nothing_ about that problem is addressed by making the editing GUI simpler.

Useless.

------
marshray
What a nice bit of research and summary presentation.

------
cnspcs-cmplr
Oh, the good old days. I went and inserted some Irish Evil into Wikipedia in
commemoration.

