
Fear of sentient robots - iamwil
http://www.allartburns.org/2009/07/29/fear-of-sentient-robots/
======
grellas
I remember my college days in the early 1970s at a liberal California
university when computers were continually portrayed as soulless monsters
having no redeeming value. That was before the personal computer. Same
functionality - same destructive potential as before, but now seen as friendly
toward human advancement. It is all a matter of perspective - computers
themselves are a tool, one that can be used for good or bad.

The corporation is impersonal by design. It is a structure designed to
facilitate enterprise.

It does so: 1) by giving an enterprise perpetual existence; 2) by allowing
investors to put up risk capital without having to risk their personal assets
in the venture; 3) by separating management from ownership such that the
operations of an enterprise can be flexibly handled; 4) by creating discrete
units of ownership that can be used to create special incentives for founders,
investors, employees and others; 5) by creating a vehicle for widespread
public ownership of the enterprise.

Is this good or bad? It is all a matter of perspective.

Consider that, without the corporation, we could not have the modern startup.
Try to gather a serious team together without being able to promise them
limited liability should the venture fail. Try to raise funds for your sole
proprietorship from strangers. Try to offer equity incentives to employees
when all you have to work with is a family business or a partnering-type
vehicle. You will not get far.

Robotic or not, a corporation is a legal tool that serves an important
purpose. Yes, it can be used for good or for bad. Just as with the computer,
it all depends on the people behind it.

~~~
fredBuddemeyer
a corporation is so much more than a legal tool because of the effects that
emerge from it. a simple example is your number 3 - a description of the
conflict of interest inherent in corporate structure (and the source of much
controversy lately).

while a corporation is a collection of people if it is successful it buries
every one of them, until the end of time. the behavior that emerges from the
simple act of self-preservation over time is complex and very different from
entity to entity (see evolution).

more personal, accountable structures do not require feudalism or regress.
what they require is communications technology like that we are utilizing
right now (and real names help).

~~~
grellas
Fascinating idea of having more personal, accountable structures - is there a
write-up somewhere of what legal form these might take? Or are you talking
about LLCs and the like? I have been doing startups for a long time now and
cannot imagine them (nor most other forms of enterprise) without _some_ form
of limited liability vehicle.

Or is it the idea that communications technology will somehow transcend the
form of the structure and lead to better results simply because people relate
to one another differently?

I am asking this sincerely. I guess my imagination does not easily comprehend
the potential alternatives. If you can give me some pointers to discussions on
this, I would appreciate it.

I have often seen corporations criticized but have never heard of any formal
structure along the lines you describe (again, unless you mean LLCs or the
like).

~~~
fredBuddemeyer
fair enough q but the legal structure of organizing people is a reactionary
force and well behind the changes taking place in society (for instance web
businesses with distributed operations still need a shopkeepers style business
permit in some jurisdictions - "the business has to be somewhere where do they
get their mail?").

in the case of corporations and their modern alternatives tax-engineering is
the driver more than the things we are talking about.

w/r/t liability the corporation is an extremely inefficient mechanism for the
elimination (or lately society's assumption) of the liability. the underlying
risk of the entity has no bearing on the price of this risk (a break even bomb
factory pays less than a profitable pillow store). the increasing importance
of director's insurance shows the market has a way of addressing this and may
provide the harbinger (if not the "write-up") you seek.

~~~
grellas
I am referring to the risk that investors face of losing all their assets
(beyond what they invested in a company) in case the company fails. I don't
understand why people would put up money for investments in such cases. Are
you saying that insurance would cover them?

Not to belabor this but the idea of investors being shielded from losses
beyond their risk capital was what gave the corporation its huge expansion
over the years. That is how public companies came to be formed, for example.

Still not understanding how investors would be motivated to invest in most
companies without such protection.

~~~
fredBuddemeyer
its pretty simple, corporate ownership/government fiat isn't the only way to
protect investors that aren't willing to assume risk. you've done a good job
of deconstructing most of this now just look at the pieces and see they are
separate.

------
fredBuddemeyer
the most fearsome self-sustaining entities are also self regulating, they are
called governments. legacy interfaces like representative legislatures and
voting have limited influence on their path towards omniscience and
omnipresence.

------
lallysingh
Pretty dissapointing. I was hoping for a paranoia-filled full-on psychosis,
serialized to HTML, and all I find is a bait-and-switch.

Look, if you want to rant about corporations, make scary movies about them.
And no, the corporation from Alien doesn't count, the scary part was the
Alien, not the corporation.

In the mean time, I'm going to stay right here, work for a corporation, and
keep winding more copper for my EMP.

------
geuis
This is a thread that's reflected extremely well in Charles Stross's
Accelerando. It took me a couple of readings of the novel to really make the
connection between intelligent corporations in the story and what's actually
happening in the real world.

~~~
jacquesm
Accelerando is one of the strangest books I ever read.

The 'rapture of the nerds' is a theme worth exploring (the singularity), but
I've never given it much credit. After reading accelerando I'm not so sure
anymore.

It certainly is food for thought. Given the pace of progress in technology in
the last 30 years or so I'm pretty ambivalent about how fast it goes. Sure,
there is lots of movement but I don't see anything that is really
mindblowingly new. Networks existed, the internet is already decades old.

We're able to communicate faster and better, the cost of a bit has decreased
further and further.

But we are not much closer to for instance real AI than we were 30 years ago
(only then we were much more optimistic that we'd have it in 30 years).

~~~
endtime
>But we are not much closer to for instance real AI than we were 30 years ago

This is very hard to evaluate. It seems unlikely that progress towards strong
AI will be linear - and I'm not basing that on Kurzweil or claiming it will be
exponential, I just don't see how the notion of making steady progress towards
strong AI even makes sense.

~~~
jacquesm
That's a good point.

Ok, how about this then, if IQ would be some measure would you accept that
even lower animals have an IQ of sorts and they can solve simple problems on
their own ?

And that we still can't get a computer to solve anything on its own without
very precise instructions ?

A first step towards AI would then be to get a computer to be goal driven
enough that it could solve a simple problem without it being foreseen (and the
solution spelled out) by the programmer.

It could still be years before that would lead to a true AI in the sense that
it could do something that we could not but it would be a step along the way.

~~~
marcus
I think genetic algorithms have been able to surpass that standard a long time
ago.

While the developer has to define the playing field and the rules, the
solutions the computer comes up with are way better than randomly searching
through the solution space. check out
[http://www.popsci.com/scitech/article/2006-04/john-koza-
has-...](http://www.popsci.com/scitech/article/2006-04/john-koza-has-built-
invention-machine)

Of course that is no way near strong AI, it is still a step in the direction.

And we are working on simulations of the cortex of simple mammals. (read a bit
on Blue Brain).

I am not saying we're close to strong AI, but we are making small steps in the
right direction, and if philosophically speaking you are a materialist like me
(believe that consciousness can be explained in theory using nothing but
physical/chemical terms) than it is just a matter of time until it happens.

~~~
jacquesm
True, genetic algorithms are definitely a step in the right direction. There
is the one famous example of the filter circuit where even the designers of
the software couldn't figure out how the damn thing worked, but work it did.
Parts connected in the weirdest ways and if you took them out the circuit
stopped to function. Apparently the genetic algorithm had in its feeback loop
discovered some none-trivial parasitic effects of some components. No sane
designer would have done it that way, but that's beside the point because the
filter had less parts than any designed by humans directly.

I'm very strongly a believer in the material consciousness,

The emperors new mind was an interesting read but I found it to be a
tremendous buildup to an almost magical reliance on quantum mechanics to
provide us with our consciousness.

I don't have much with magic so I'll stick with regular physics and chemistry
until it has been proven otherwise.

There are too many brain-computer analogies to avoid the strong suggestion
that the brain computes. It doesn't do it in a way that we can hook up a
debugger to just yet but it does seem to have a lot in common. It's more of an
architecture/interface problem.

But strong AI probably won't come from being able to interface with the brain
or from a sudden leap of insight in how it all works. I think strong AI will
follow from a seed AI which in turn will follow from some relatively simple
breakthrough or insight that we still haven't gotten around to.

The simulation of the cortex of simple mammals is another such step in
between, do you have a reference for that ?

~~~
marcus
I wasn't talking about brain computer interfaces, although that is an
interesting field. Note to self: try our algorithms on EEG data...

My point regarding the materialism was that if you can simulate the
physical/chemical processes that happens in a brain, and I am talking about
the molecular/quantum level, the result will be indistinguishable from the
original.

The reference you asked for: project Blue Brain by IBM
<http://online.wsj.com/article/SB124751881557234725.html>

~~~
jacquesm
I meant that to figure out how the brain works we will have to figure out it's
architecture first, then we will be able to hook up to it to refine that
knowledge.

Thank you for the link, I'll read it today. The rate at which hacker news
produces good stuff to read is getting higher than I can read fast!

------
asdlfj2sd33
In addition to corporations we also have humans. Breaking news, 100% of
murderers are human!

