
Have you pondered the moral implications of your startup? - ecommercematt
I'm working on a start-up that, if successful, has potential to be a major reference of human knowledge (think wikipedia in terms of vastness and impact, although it isn't particularly similar to wikipedia). We won't be able to control who uses our resource, and as it is community-driven, the specific direction our site will take is unknowable.<p>Call us naive optimists, but we're convinced that, for the most part, our startup will be used for good purposes. We're strongly against censorship, and we don't think censorship really works on the web anyway, so we'll be leaving this up to chance.<p>An example of a startup that might be slipping to the darkside is Reddit. I know Reddit was backed by YC, so anything perceived as a bash might not be well-received here. Nonetheless, I'm certainly not alone in thinking that the Reddit community has taken a turn for the worse. My biggest problem with Reddit today is its lameness, however, Reddit is not just lame, it is increasingly used to propagate toxic misinformation and hate speech.  I wouldn't be so troubled if I didn't see so many members of the community at Reddit embracing those sentiments.<p>I'm pretty sure that the Reddit team doesn't condone hate speech, and I'm not proposing they censor the site (it wouldn't really be possible, anyway), but it makes me wonder:<p>What do they think about this segment of the community they built? 
Do they wish they'd done anything differently? 
Is there anything my team can or should do to minimize our site's use for what we consider to be immoral purposes?
Has anybody here spent much time thinking about this issue? Have you come to any conclusions?
======
pg
I'm surprised how often this comes up in YC startups, actually. I'm not sure
if this is true of founders in general (it might be) but practically everyone
we've funded not only wants to not be evil, but actively wants to make the
world better.

This was certainly true of the Reddits. And in fact I think they succeeded. If
you make a site where everyone can say what they want, some people are going
to say things other people don't like. But isn't this a net improvement over
the preceding model, where there were a few narrow channels for the
distribution of news, and the companies that controlled them controlled the
news? I'm not sure what you mean by "hate speech," and I doubt you are either,
but I think we're net ahead if we have a world in which its harder for the
powerful to suppress news, even if a few people take advantage of this new
openness to say things that offend others. In fact, some of the best ideas
started out that way.

~~~
ecommercematt
It doesn't surprise me that this is a common concern in YC startups. It also
doesn't surprise me that the Reddit guys gave the issue consideration.

I agree that limiting the ability to suppress news, even if some people take
advantage of increasing openness to advance harmful agendas, is a positive
trend. I used to regularly derive enjoyment from Reddit, and occasionally, I
still do. Nonetheless, I think Reddit's struggle with various forms of abuse
(by idiots, racists, spammers, etc.), is relevant to those of us working on
community driven web-based projects (even if "community-driven," and "web-
based" are the only traits in common with Reddit, as is the case with my
startup). If I were one of Reddit's founders, I'd be troubled by its current
state, and I want to do whatever I can to prevent similar problems from
happening to our site.

As to what I meant by hate speech, I'll lean on Justice Potter Stewart's words
regarding the definition of pornography:

"I shall not today attempt further to define the kinds of material I
understand to be embraced within that shorthand description; and perhaps I
could never succeed in intelligibly doing so. But I know it when I see it..."

<http://en.wikipedia.org/wiki/I_know_it_when_I_see_it>

There is plenty of hate speech on Reddit, and I know it when I see it.

The first time I discovered your (pg's) existence was when I received a link
to your essay "What You Can't Say."

<http://paulgraham.com/say.html>

You said in that essay "like every other era in history, our moral map almost
certainly contains a few mistakes." I agree with you. Nonetheless, I'm sure
that it isn't a mistake to consider white supremacist propaganda, for example,
to be harmful to society.

I'd be surprised if you haven't encountered hate speech on Reddit, but perhaps
you haven't. I imagine it wouldn't take you long to find, if you tried, as I
have stumbled across it quite regularly by accident. Regardless, the
particulars of my critique of Reddit aren't relevant. Figuring out how to make
a community good, and keep it good, is my goal. The consensus here appears to
be that an ounce of bad sprinkled in with tons of good is still worth it. I
agree, but I'm still going to think about how to reduce that ounce to a gram,
or less. Any ideas would be appreciated.

~~~
pg
_Nonetheless, I'm sure that it isn't a mistake..._

If you read the essay carefully, you'd understand that in every era "right
thinking people" are _sure_ that the things they want to ban are bad. In 1700
after explaining how broad-minded you were you'd be saying "Nevertheless, I'm
sure it isn't a mistake to consider atheistic propaganda to be harmful to
society." And you'd be wrong.

The whole point of the essay is that you have to step out of yourself to have
any hope of seeing beyond the prejudices of your time, and that this is
extremely hard. Your casual use of blanket labels for forbidden ideas is a
sign you don't appreciate the difficulty of the problem here.

You'll notice I have never said what kinds of speech I think should be banned.
That's because I've seen enough to know that that that second clause following
"I'm pretty open-minded, but..." is very likely to be mistaken. Like someone
saying that some open mathematical problem will never be solved, you're
setting yourself up to look like a fool to future generations.

So your use of "sure" to me is very convincing evidence that your filters will
generate a lot of false positives. I spent a whole month thinking about this
problem. WYCS took the longest of any essay I've written. And I would be very
reluctant to use that word "sure" in this kind of situation. So either you
understand this stuff so much better than me that you've passed through
uncertainty and back into certainty, or you simply have the confidence in your
opinions that everyone is born with.

~~~
Eliezer
You're up against hindsight bias here, PG.
(<http://www.overcomingbias.com/2007/08/hindsight-deval.html>) People don't
realize how absurd the future looks when you have to predict it in advance.
(<http://www.overcomingbias.com/2007/09/stranger-than-h.html>) 500 BCE seems
much stranger than 2008 CE, which seems very normal by comparison - so people
look back and see a steady progression toward normality, things getting less
absurd over time, and they expect this trend to continue.
(<http://www.overcomingbias.com/2007/09/why-is-the-futu.html>)

People don't realize how counterintuitive moral changes look when you have no
advance idea of where you're heading.
(<http://www.overcomingbias.com/2007/03/archimedess_chr.html>) So they don't
use the kind of cognitive strategies that would have been necessary for, say,
Archimedes of Syracuse to question slavery.
(<http://www.overcomingbias.com/2007/03/chronophone_mot.html>)

------
cperciva
Yes, I have pondered the moral implications of my startup.

Cryptography comes with a price: The bad guys get to use it too. Where my
startup is concerned, this means that organized crime would have access to
TLA-quality secure backups (and by "backing up" data from one system and
"restoring" it to another, secure communication, too).

In the end I decided that since (a) criminals would need to pay for the
service (which makes tracking them down easier), and (b) I'm going to be
logging IP addresses, the extent to which this would assist criminals is
fairly limited; and that the legitimate needs people have for secure backups
far outweigh the potential for abuse.

~~~
randallsquared
Er, not to mention which, anyone who wants secure commo can just use GnuPG and
friends.

------
mrtron
You can't make people good.

You can make yourself a better person and lead by example, but you can't stop
people from doing something like spreading misinformation and hate speech. It
has been going on since the beginning of civilization. If your startup has the
right moral and socially beneficial intentions, then that is really the most
you can do. If someone comes along and uses your tool for evil, that doesn't
mean you shouldn't have created your tool.

In scientific and technological research, there are often amazing uses that
benefit humanity. For example, splitting the atom resulted in nuclear power
which is arguably a great invention for mankind. However, due to human nature,
it created the greatest weapon mankind has seen. You cannot blame the
researcher for facilitating this, you can only blame human nature for abusing
the technology.

~~~
imsteve
> created the greatest weapon mankind has seen

Stretching it a bit... I think the guy who made machine guns happen slept
worse at night. You must realize though, that both were obvious inventions
that were going to happen no matter what.

~~~
r7000
indeed: <http://en.wikipedia.org/wiki/Repeating_crossbow>

------
edw519
Stop worrying. Now.

Every nanosecond you worry about this is a resource permanently lost from
where it belongs: your startup.

What if Henry Ford hesitated because he worried about drunk drivers? Or Edison
and Bell hesitated because they worried about drug dealers and criminals? We'd
still have automobiles, lights, and phones. But you would have never heard of
any of them.

You have little or no control over this. Have a little faith in others;
stepping forward to confront evil is their thing. In the meantime, do yours.

~~~
boredguy8
The hottest fires in hell are reserved for those who remain neutral in times
of moral crisis.

~~~
edw519
How is building something people can use (or possibly misuse) the same as
remaining neutral?

~~~
aston
It's a moot point anyway because most of the time when you make a moral stand
in your product you end up making the experience for its audience better. The
tricky part is knowing which types of people you'll lose or gain based on
those moral stands.

------
cousin_it
Long time lurker, finally registered to respond to this.

On controversial matters like race, religion or abortion, no matter what your
position is and how sure you are about it, there almost certainly exists
someone on the other side who's better educated/informed and just as sure as
you. My ideal community site would allow the most extreme opinions, but not
let any group jam the signal for others. For example, the Wikipedia page "Race
and intelligence" would be much improved if it were split into two pages side
by side, each editable by only one of the factions.

~~~
ecommercematt
That's an interesting perspective. The problem that initially jumps out at me
is what to do when there aren't two clear sides to an issue. Any ideas on
handling complicated, muddled, and contentious issues with multiple sides?

~~~
cousin_it
Not sure. Google gives me this: [http://unqualified-
reservations.blogspot.com/2007/08/uberfac...](http://unqualified-
reservations.blogspot.com/2007/08/uberfact-ultimate-social-verifier.html)

------
ntoshev
Yes, there are things you can do to drive the community in the way you want,
without resorting to censorship.

There are softer measures you can take - make the votes of trusted editors
(and people who vote like them) count for more in the ranking algo in the
Reddit's case, for example. Is this ethical? Well, you have to decide for
yourself.

A few examples to think of might help:

\- Western civilization generally spreads its values along with its
technology. Is this Ok?

\- Is it ethical to support democracy in cultures that don't grow it
themselves?

\- If you are the government of some country and have a tribe practicing
female genital mutilation and considering it normal (including the women),
what is the ethical thing to do?

\- Is Scientology ethical? Is it ethical to constraint them?

------
mynameishere
_I'm not proposing they censor the site (it wouldn't really be possible,
anyway_

Sure it is. The problem is that censors are always idiots: They define things
as "good" and "evil" rather than "true" and "false". This tends to leave them
and their compadres reading nothing but falsehood.

~~~
brianr
Censorship usually is not associated with filtering true vs. false
information. From wikipedia:

 _Censorship is the suppression or deletion of material, which may be
considered objectionable, harmful or sensitive, as determined by a censor._

"Censorship" implies a value judgement, not just filtering out content that is
false (although even then, it can be hard to determine what is unquestionably
untrue). That's why it's so subjective and difficult.

------
papersmith
Just curious, can you give some examples of hate speeches on reddit? I'm
thinking of the recent James Watson controversies, but that would seem pretty
fuzzy. Any clarifications are appreciated.

------
MisterMerkin
You can't throw out accusations of hate speech at the reddit community without
some specifics which I've yet to see you mention in this post.

------
maxwell
That which lasts is self-correcting.

------
jsnx
If I understand it correctly, the idea with Reddit is that the community can
decide what news it wants to see, by posting links and then filtering them. If
the community wants hate speach (or more of those Ron Paul links) and Reddit
serves up hate speach, then isn't the algorithm working?

I suppose you could actively ban users who skew the community in that
direction -- though you'd have an endless calvacade of sock puppets to deal
with.

------
cmars232
My current toy idea is a real-time captcha trading market. On the one hand,
spammers might pay to use it. On the other hand, it will inspire websites to
come up with something better than captchas.

In any case, making a real-time market is too fun to worry about implications,
don't you think? :)

------
edu
"Things happen, what the hell!" \- Terry Pratchet, The Hogfather.

------
sammyo
Bah, humbug. ;-)

