Hacker News new | past | comments | ask | show | jobs | submit login
Not OK, Google (techcrunch.com)
651 points by CapitalistCartr on Oct 5, 2016 | hide | past | favorite | 575 comments

Google Now has done the same for me, told me how long it will take to get to a bar I frequent. My reaction was quite exactly "Oh that's neat, thanks!" and I went and had a great burger that night.

Totally OK for me Google. I respect that people have different privacy thresholds, but I think the fact that it's different for everyone is being lost in articles like this.

One tiny caveat being that, Google (and others as well, to be fair) will be able to indirectly collect data even on the privacy-aware part of the population who don't use Google much. The simplest example being, even if you don't use GMail, still some part of your emails inevitably end up in GMail inboxes. Now also consider this: just being a guest in a Google-stuffed house means you are under surveillance.

So no, it is not just my problem or your problem, it's everyone's.

Sure, and to illustrate your point, I have an email very similar to someone else's. I very frequently get their emails (invoices, church events, travel itineraries, purchase receipts). Google thinks they're _my_ trips, and updates me about flight times.

I think this example serves both our points. To your point, it's totally leaked this other person's info into my "google world" because I'm on gmail. On the other hand, that person is leaking his information directly to me just because of typos when he fills out online forms. Perfect privacy requires a lot of vigilance in a digital world, with or without google/gmail/hotmail/yahoo/etc.

> Sure, and to illustrate your point, I have an email very similar to someone else's. I very frequently get their emails (invoices, church events, travel itineraries, purchase receipts). Google thinks they're _my_ trips, and updates me about flight times.

Fun story there. Because Google's internal privacy safeguards are so strict, the people working on features like that can't look for example emails to train their ML models with.

They can only look at emails that were explicitly sent to them in order to improve the feature (and almost no one forwards along positive nor negative examples). What they can do across the email corpus is run jobs that return aggregate stats, where each stat must be coarse enough that it is infeasible to trace back to original users (often 100k+ users per data point).

So, AFAIK, training & testing models under these safeguards is more or less done blind. Build a model with the few examples you do have, and then run it against the corpus. If you see numbers change, you have no idea if that's good or bad, since you can't actually inspect the run.

(at least, this is the way it was a few years ago)

> and almost no one forwards along positive nor negative examples

One time, when I marked a bunch of incorrectly-classified emails as "not spam", the Gmail web UI asked me if I wanted to send these emails to the Gmail spam team. Was this what you meant, or something else?

Yeah, that sort of thing - by giving permission on a per email basis, they are then allowed to look at that particular email to see how/why the model is miss classifying it, AFAIK

(That's different than just marking an email as spam, though)

That's one of the interesting things about AI. There's no way to clairfy/correct when something is wrong, and most of the time you don't even know something is wrong.

It's not clear whether these AI models have much incentive to correct anything. If 99% of people with attributes x,y and z are bad candidates for a job, will you even get an interview? Is there any attempt to account for the fact that attribute x is something you were born with? Or that you are actually in the 1% and really are a good candidate? Or that you don't actually have attribute y, and it was just inferred from something else or some kind of mixup like an email address typo?

There are all kinds of interesting thought experiments. What happens when a classifier innocently discovers that the best classification is by race? Do we care? How about if we remove race but it happens to discover that four features which are very strongly correlated to race are the best way to classify?

If that second scenario were to happen, then I think we should take a serious look at why that correlation is occurring rather than just throwing out the data because it's "racist". That we removed the classification and then it was re-discovered by other correlations really should suggest something. On the assumption that it wasn't engineered to be biased and was naturally arrived at by the algorithm itself, then that actually seems like an important data point, and could even be a nice litmus test of how we're addressing racial differences if the models evolve to be more positive over time.

There are ways to account for that. A model can be fit to race, and then you only predict "on top" of race (meaning residuals). You use that model, which is independent of race.

This only works if you're aware that race is even a factor. If you're not aware of the problematic factors, then you can't correct for them.

How do you know you have the correct model, and isn't making the system more racist instead of less?

Machine learning is also very opaque.

But if the racial factors aren't all the same, then that creates an incentive for people to lie about their race.

If you verify the race field, then now you're in the business of enforcing racial definitions.

> If 99% of people with attributes x,y and z are bad candidates for a job, will you even get an interview? Is there any attempt to account for the fact that attribute x is something you were born with?

Why would that matter to the company? People are born with stupidity.

I have a fairly uncommon name and predominately use Gmail. Still, I receive a surprising amount of other people's mail in error. Apparently lots of people guess at email addys. I do often wonder at both the lack of privacy this engenders and what the Goog machine must make of it all.

I definitely worry about my life being affected by situations like this.

I receive a ridiculous amount of other people's email - for serious things like email account reset, to banking info. When the Ashley Madison hack happened, my email address was there multiple times! Imagine if my then-partner had bothered to look, what a mess that would be.

Lay down a trail of public comments for plausible deniability. Smart!

Your current or next partner might not like that you've signed up either :)

> for serious things like email account reset, to banking info.

I've called American Express a couple of times to report errant emails with account info ending up in my email. THEY didn't care that one of their customers had an issue, they thought it was a great time to have me fork over MY info "to check".

My partner received a few emails which a government representative had tried to forward from their work account to their private Gmail account whilst getting the address wrong. These contained personal information relating to correspondence with constituents.

From time to time I have had messages relating to a government planning committee as one of the members got the domain of someone's departmental email account wrong and the messages come instead to the domain I administer.

I know someone who has his (common) first name @gmail.com as he was an early employee on Gmail, and he regularly gets grandparents guessing their grandson's name as the correct e-mail address. He told me this happens several times a day.

I just got a funny email last week... welcoming me to the NRA. I was like... but I didn't join the NRA. I sent them an email about it, but I'm probably going to have to CALL the NRA to tell them to remove my address from their database.

Getting these things corrected can actually be quite difficult. I regularly get emails from an optician in another state. I have replied multiple times that they are sending information to the wrong person, but they continue to do so. Fortunately this has never included detailed health information, but merely revealing a patient relationship with a specific medical provider can be a breach.

In another instance I was getting emails about an account someone created with American Express using my email address. I sent multiple emails to their customer support to get them to stop sending financial information to the wrong person with no results. In this I also found it difficult to even figure out what agency to report them to for failing to take action when notified. Eventually I took the time to call them. It took around 20 minutes (not counting time on hold) of talking to multiple people to get them to remove the email address from the account. This included them asking me several times for my social security number - which I flatly refused to provide since I had zero business relationship with them. This refusal actually seemed to confuse them.

The thing that gets me is when people seem to be guessing at their own emails when filling out forms and such. How do you not know your own email address?

Many of the people I know who have very common gmail addresses have set up canned replies to let senders know they've reached the wrong person.

> On the other hand, that person is leaking his information directly to me just because of typos when he fills out online forms.

I often get some misdirected emails because of a very banal name in my country.

When the email contains a thread history, I sometime noticed that the address was simply corrupted by a recipient, such as numbers getting dropped from the genuine address in their reply.

I guess some systems can't handle correctly numbers or other characters in email addresses.

Wonder if one should press for having these services accept a public key alongside the email address that they then are obliged to encrypt all outgoing emails with. Thus even if the address is wrong, the recipient can't easily read the content.

How does that solve anything? Either these services will have to publish the key on your behalf (so you can lookup the public key for bob@gmail.com with some public API), or you will have to provide the public key every time you hand out your email address.

The former doesn't fix the issue at all, and the latter is unworkable because the guy reliably giving out the wrong email address will absolutely not remember his public key.

> the guy reliably giving out the wrong email address will absolutely not remember his public key

Have the browser suggest the key IDs from `gpg --list-secret-keys`.


Had the same thing, Google swore I was supposed to be booking into a hotel in Copenhagen, but most definitely wasn't me - interesting to see what happens are these predictive features become more prevalent.

This is often useful even if you're not the flier. When I'm picking someone up it lets me know when their flight was delayed, and when I'm traveling it lets my wife know when I'm available.

This situation is not like that. Presumably, ubercore is getting email for someone he does not know.

If you visit a place of business you are potentially under surveillance. I'm sure there is a distinction but I'm failing to come up with it right now.

I was somewhat radical about privacy in the late 90s (only person I knew that read every EULA) and am still a supporter of the EFF but I don't really understand the issue here.

The distinction is that when you visit a place of business, the data is under the control of the business owner.

When you visit you friend's house, the data is not under their control, it is under Google's control.

Therefore, Google has a WAY bigger responsibility than most people realize, once they decided to collect this data.

You are assuming that the business doesn't use a third party monitoring firm.

Ultimately, whenever you visit any place, business or home, the data is under the control of the owner and anyone they abdicate those rights to.

I tend to agree, though I tried to do some searches for a law citation here, but struggled to find anything concrete. I imagine this will be a big area of research and exploration for law in the coming years.

Some interesting scenarios to consider: If I visit a friend's house, and I start getting targeted ads for a service I didn't subscribe to without prior consent or my knowledge, can I sue her/him? What about a scenario where some service collects my data, said service is hacked, and someone commits identity theft on me, who is liable for damages? Do I need my buddy to sign a waiver when he visits to play some Xbox for a bit?

> When you visit you friend's house, the data is not under their control, it is under Google's control.

But it was your friend's decision to delegate that control. So, Google having that control is still a consequence of your friend's control.

Do you normally use your friends' wifi? I do that when it's family, but generally not with my friends (don't do LAN parties anymore).

With LTE, I don't need wifi.

Wasn't there a story recently about android phones still being tracked and tracking wifi hotspots even with wifi turned off? I believe location services still works pretty well even with GPS and wifi off.

I would imagine if you walk into a home with google's AI doodads all over the place, you're gonna be picked up.

Yes, there was recently a question of how Android knew someone's location with such high resolution, even when wifi and GPS were both turned off. The answer was a passive probing of wifi identifiers and using those against the Google SID to location database even when wifi was turned off.

There was even a note about this feature in the privacy policy, IIRC.

Makes you wonder how reliable airplane mode is.

Airplane mode doesn't turn off the radios, IIRC; it just stops transmissions.

Rumor is that even when your phone has been turned off, it can be turned back on remotely.

Perhaps people should be required to post a notice on their front door if their house has a Google surveillance device in it (or I suppose, verbally inform every visitor). It's common (and often required) for businesses to post notice that people on the premises are under video surveillance.

I have a couple of cameras, and at least one visitor has been uncomfortable with their presence, despite the fact they're not sending the data to a third party.

> If you visit a place of business you are potentially under surveillance. I'm sure there is a distinction but I'm failing to come up with it right now.

A business is typically located in a public space, where there is no expectation of privacy. In contrast, a home is by definition a private space where there is a strong expectation of privacy.

Exactly! In a public space, you can expect someone to be collecting data about you, even if it's only photographing you in the background. In a private space, you expect no one to be collecting data about you, except the people you are directly interacting with.

Aren't there some wiretapping laws around this sort of thing?

This seems a bit analogous to the treatment of attorney-client privilege (at least in the US.) If I have a discussion with my attorney in my house, or in my attorney's office, with no one else there, the conversation is generally privileged. If I have that conversation in the presence of a third party, the conversation is no longer privileged. So, if I'm in someone else's house, I may have implicitly consented to the fact that I don't control the environment and can no longer assume the same level of privacy.

(IANAL, YMMV, etc.)

A home may be a private space but it isn't <i>your</i> space. When entering someone else's space you open yourself up to having to abide by their rules, listen to their music, and be captured by their cameras.

Its one thing to be under surveillance by a grocery store's security cameras and another thing when those cameras are sending all your data to the same central organization. Right now we aren't that great at integrating all that data — knowing that I'll be in a certain bar on Sunday nights is a little disconcerting, but if I knew that was the limit of AI assistants' invasiveness I'd be embracing it for the convenience. But technology for AI and data processing will improve, and data gathering will become more pervasive and more centralized, and soon megacorps will have a much more complete profile of you that they can query if ever they or one of their employees develops malicious intent (not to mention the non-malicious actions that can cause harm).

Tl;dr there's privacy in decentralization.

There's privacy in decentralization but you have to recognize that for many of us, there's value in openness.

I'm very optimistic about Google making my life better in lots of little ways. I have virtually no concerns about my openness to Google causing me trouble.

"making my life better in lots of little ways"

I think that's a common goal and widespread belief and find its implication of having given up on "bigger ways" troubling.

Our industry was going to change the world. Rewrite the cultural rules of socializing. Rebuild the economy in a new form. Encourage major cultural and political shifts by empowering the little guy. Bring knowledge to the ignorant, companionship to the lonely, power to the powerless, jobs to the jobless.

What we have now is, well, maybe, with some luck, timidly, sometimes google can analyze the last time I went to a gas station, the number of miles I've driven at various speeds since then, the time of my next appointment, then it can adjust my google now card to encourage me to leave home earlier so I'll have time to fill up the tank and it'll find the cheapest advertised price along the way. That's nice ... but wheres my revolution?

Even worse in context of the article, OK well say I have to give up all privacy and go hard core 1984 telescreen big brother is always watching, to save endangered species and house the homeless and feed the starving and bring peace to the victimized. Well OK I'll think about it. Oh wait, we don't get any of that, all we're offered is slightly better appointment scheduling. Eh, no thanks.

Nothing is ever really new, and this era is likely similar to pre-quantum era Physics around 1890 where nothing is left to discover other than adding a few more decimal places here and there. The meme and speech patterns are all the same (although I'm not quite that old)

While it doesn't seem like that much of a revolution, things have changed significantly.

For centuries, people have navigated using stars and maps, yet now you have turn by turn navigation in your pocket with real time traffic and crowd sourced traffic incidents. You can reach most of the world population instantly by dialing a few numbers or even sending them a text message or mms, no matter where they are.

In the context of the article, a couple of years ago, Google Now told me I had to leave for a meetup in my calendar that I completely forgot about, gave me transit directions for a part of a city I've never been at and got me in exactly at the meetup start time.

While that all seems like modern conveniences nowadays, even thirty years ago if you'd have told people that you couldn't get lost anywhere in the world and could get ahold of just about anyone at anytime instantly, they'd think you'd be talking about science fiction, not today's reality.

Google has top notch practices in my opinion regarding privacy. If anything I would be more worried about smaller companies with shadier practices and lower security standards holding my personal information. There have been seemingly illegal practices from companies like Sears with how personal data is collected and used. It's easy to throw shade at a big companies, write a sarcastic title, then get clicks.

Example: "Intuit’s TurboTax stores highly detailed financial data for millions of users who import their W2s, their banking data, info about their mortgages and more. Right now, all of this data is locked into TurboTax, but the company is now thinking about how it can do more with it by giving its users the option to share this data with reputable third parties." ... https://techcrunch.com/2016/09/22/intuit-wants-to-turn-turbo...

> I would be more worried about smaller companies with shadier practices and lower security standards holding my personal information.

True, I don't mind Google today, but what about tomorrow? What about N years from now when they have failed to hit their financial targets 2 years running. Will that company have the same set of standards as the one today?

The reality is that once you've given up your privacy there is no getting it back.

For me the real problem is that large and trusted companies like Google are softening the public perception of what privacy should be and making it easier for smaller and more malicious companies to abuse the trust that Google and others generate.

"Google has top notch practices in my opinion regarding privacy."

Google might handle the data it collects better than other companies, but it would do much better still if it didn't collect any personal data at all.

Hands off my data, Google!

Google wouldn't be Google without all that personal data though.

Perhaps we need to start normalizing encrypted email. Just like https everywhere is no longer considered necessarily "tin-foil hat" SOP, moving this direction for email needs to be socially normalized.

Going further, given that an encrypted email to Gmail will simply be unencrypted and then available to GMail, include in the protocol authorized (via both white and black list means) agents of the recipient. So, if you are hosting your own email but the intended recipient is expected to be not hosting their own email, the sender can blacklist "agents" such as Gmail and Yahoo! Mail, or blacklist all except for those chosen to be white-listed such as Proton Mail.

Sure, except the recipient will read your email in Google Chrome, while sitting on a Google Sofa sipping Google Coffee? :)

ha, so true. one prob at a time :)

I was thinking about this just yesterday. It would be great if email providers made tools to painlessly create encryption keys and get them synced up to all your devices securely. From there each email you send can include an automatic signature of where to find your public key. On the other side when you receive emails from others with such a signature you can then choose to have the email reader fetch the key and let you start trusting it with a single confirmation click. From then on your emails to that person will be encrypted without you having to think about it.

For almost all users getting a message from someone you personally know containing a message that they would legitimately send would be sufficient to trust the referenced key. If you are a little paranoid you could ask them over the phone or in person to send a specific message that you would then trust, or just ask for the URL itself. I believe that in-person key exchanges utilizing large trust networks are overkill for the vast majority of people sending everyday communications.

Make it so simple as 1) one time key creation and copying to each device 2) one time trust per other person that is completely streamlined by the software. If the gmail team implemented this for example other email providers would soon do the same and it would spread like wildfire. Very quickly a huge amount of email would be encrypted. Of course this would require an open design that anyone can implement.

Maybe there's a fundamental flaw with this idea that I'm not seeing. If so, please say so because otherwise I'm going to just be disappointed in five years when email is still 99% unencrypted.

I'm not sure you are any less responsible for your own privacy despite the fact that companies like Google as well as others are making it more challenging. The example you mentioned seems easily fixed by using GPG.

Granted there may be a place for regulations to help us restrict what companies are able to do(perhaps making it easier for you to identify a region that is being recorded, right?), but, at some point society can't help the fact that you'd prefer if machines were unaware of your existence. That's just something you have to solve for yourself.

> The example you mentioned seems easily fixed by using GPG.

GPG doesn't hide that one is emailing, or when, or with whom; it only hides content.

If you don't care for the products or services, you could always opt not to buy/use them. That includes avoiding areas where Google appliances are present. I'm fine with trading privacy for AI-driven convenience, though I already have a phone I like and an Echo, so I have no compelling reason to buy these particular products right now.

And how are we going to know where google appliances are present or not?

By looking around and/or asking? Life isn't that hard.

"Excuse me, is that an android phone in your pocket?" Also, what about google self-driving cars with full arrays of cameras, radar, etc. recording everything as they drive past you?

a non-insignificant proportion of hacker news seems to actually work at google and run to their defense at anything somewhat critical of their overlord.

There just isn't any point to accusing people of shilling without evidence. The idea that one must be paid to hold a different opinion that you is offensive, and expressing it here isn't something the community wants.

i actually had 3 upvotes until the google people started to change that. so some do. i'm not accusing anyone of shilling. i've personally seen these threads and some proportion of people admit openly they work there. others, you click their profile and see they work at goog. don't need you to condescingly assume you speak for the community. Also forgot to mention all the people who defend google because they're desperate to work there

Sorry, but you don't think that it's somewhat unfounded paranoia?

Personally, I would have downvoted you because your original point seemed to condescendingly assume something about a large proportion of the community and their intentions - then you assume again that it's "The Google People" (and only them)?

Unless you can show that your downvotes came because of a specific reason, and from specific people...

my original comment was " a non-insignificant proportion of hacker news seems to actually work at google and run to their defense at anything somewhat critical of their overlord. ". it isn't paranoia. Like i said i've seen threads where 4/10 comments are defending Google and the commenters openly say they work at Google or one can click on their proifle and see that they openly list they work there. didn't imply shilling.

I fear you will be unable to recognize when that burger was your choice and when it was a reaction. You probably won't notice. And that is harmless.

I also fear you will be unable to notice in which areas of life and information the distinction between choice and reaction is harmless and which it isn't.

Of course, I'm not talking about "You" you, but just people. Me as well. I feel we are widening the field of unconscious decisions and I see that as inherently bad - in my fellow humans as well.

You could say that Plato wanted us to make easy things simple (link for distinction: https://www.infoq.com/presentations/Simple-Made-Easy).

I believe this to be a move in the opposite direction. We should have a care.

I'm sorry but that just sounds like blind fear mongering. What you're saying is vague and doesn't really mean much.

It's like saying we shouldn't use prescription glasses, or medication, or cars, because it's not "us".

Humans invent all these tools and systems to improve and optimize our life. Make our vision better. Make our health better. Make us move around faster. In the case of AI, make us do perform certain things more efficiently.

Imagine it wasn't an actually computer. Imagine it was a personal secretary you had that gave you the EXACT same information. Gave you your flight information, turned on the light when you asked, gave you the weather and your schedule. Would you think that was wrong? That this isn't "you"? No, it's just optimizing your life, but now available for a wider population rather than rich people.

"Imagine it was a personal secretary you had that gave you the EXACT same information. Gave you your flight information, turned on the light when you asked, gave you the weather and your schedule."

In your metaphor, you are implicitly paying the secretary, so the secretary is incentivized to maintain your interests.

How much have you paid Google for its free services?

Your metaphor is inapplicable. You don't have a secretary telling you these things; you have a salesman trying to sell you things, and the salesman is getting smarter every day while you aren't. Not the same thing at all.

Google earns most of their money through ads.

That's why I called them a salesman. They sell things. Their interests are not simply your own.

It seems to be a theme here today... a company can't serve both advertisers and customers. In the end, one of them has to win, and given the monetary flows, it's not even remotely a contest which it will be. https://news.ycombinator.com/item?id=12644507

They don't sell things. They forward you towards people who do sell things which you may be interested with. You're free to ignore it, and if you're not interested in what they're showing you, that means they failed at their job.

It's funny how bad of a stigma ads have gotten, but at the core, if you think of it, it's not necessarily a bad thing. Think of a friend recommending you a restaurant, a new game to play, a movie to go watch. In that case you'll be super interested, but now if this AI who probably knows your taste better than your friend suggests you something, you are instantly turned off and annoyed.

I think the root cause of this is that there is so much mediocre ads out there that ruin it for all. Your mind just blindly blocks all ads now.

Yes. Google is selling you, to advertisers, quite literally.

When you aren't paying anything for something of value, YOU are the product.

> Google is selling you, to advertisers, quite literally.

No, that would be slavery, which is illegal.

Google is selling advertising space on various channels that you provide in exchange for Google services to advertisers.

> When you aren't paying anything for something of value, YOU are the product.

No, when you aren't paying money for something of value, you are probably paying something else of value for it; often, something that the person with which you are trading is then selling for money, making you a supplier of an input to the good or service they are selling for money.

What he's saying is that this is not "humanity inventing something to make life better". It's a company inventing something to make money.

And it's not a simple product like glasses where you pay with money and then they improve your vision. It's a product which goes far beyond your understanding and for which you don't pay money.

Google isn't interested in making your life better. What they are interested in is getting you to believe that they want to make your life better and to then recommend going to that bar, because the bar owner has given Google money to advertise for the bar.

Yes, you might actually like that bar, but Google isn't going to recommend going there in intervals which are beneficial to you. They'd rather have you go there a few too many times. Because that's what makes them money. It's not improving your life, which makes them money. Their AI will always work against you, whenever it can without you noticing.

Imagine that you were trying to quit smoking and your electronic secretary kept updating you on the cheapest place to find your favorite cigarettes? With no way to tell it not to do that.

So your issue is your secretary doing it's job poorly?

First, there is a way to tell it to not do that. With Google Now, you simply tap the menu and say "No more notification like this". With the assistant, you will probably be able to ask directly.

Second, let's be honest, humans fail pretty often too, so that's just a weak argument.

Lastly, I think it's unfair to dismiss a new technology just because it could maybe fail, without having even tried it.

How about if the system is working exceptionally well, you're a depressed person, and the next ad you see is auctioned off between a therapist, a pharma company, and a noose supply store in the 100ms it takes to render your MyFaceGram profile?

The awful success cases are far more interesting than the awful failure cases.

I have no problem with ads for therapists or pharma companies competing for advertising space in front of me because they have algorithmically determined that I am a qualified lead. That actually sounds great from a mental health perspective.

Your noose example is pretty contrived, however.

Obviously the first two aren't the problematic ones. The issue is that an algorithm wouldn't know what distinguishes those from the third, obviously.

How about sleeping pills? Opiates? Local extortionist cult?

I think that algorithms, and AI specifically, are perfectly able learn what distinguishes those. Maybe even better than someone who might not be in their best state of mind.

The handwaviness is telling. Why would an algorithm or its creators even care about the difference? The highest bidder is the highest bidder.

Because the whole of Google's ad business stands on people wanting to click on the ads shown, and buy the products offered through them. That's why they spend resources on detecting misleading or fraudulent ads, which by your reasoning they wouldn't care about as long as they paid. PR is very important for this business to be sustainable: If the goal was for every user to click through one ad, and then never again, that might not even pay one engineer's salary.

What's misleading or fraudulent about those ads? Maybe you mean "morally reprehensible," in which case I ask where you draw the line between the morally reprehensible (auctioning off the method of suicide to a depressed person) and the morally questionable (say, auctioning off the final bankrupting car purchase to a financially irresponsible person)?

Detecting misleading and fraudulent ads is just an example of things they wouldn't spend resources on, if following your reasoning of "short-term money is the only thing they care about."

There's not only the "morally reprehensible" metric ("Don't be evil"); there's also the "absolute PR catastrophe" metric that printing such an ad for a rope would mean.

> So your issue is your secretary doing it's job poorly?

I think the real issue is the casual deception which you just fell for: It isn't "your" electronic secretary, and the thing it just did might actually be a "good job" from the perspective of those who control it.

I think you misunderstand me by a large margin.

I'm not saying we shouldn't use AIs. We should, however, think about how we use them.

To build on your example, what are the dangers of having a personal secretary on the payroll of anyone but you?

What I am expecting from this is a super devious filter bubble - because that's how you make money. Google's old slogan "Don't be evil" is long gone. "For a greater good" might be more on point.

>In the case of AI, make us do perform certain things more efficiently.

What does the Google Assistant help me do more efficiently? In all honesty, I can't figure it out. I don't need or want a secretary, and I can do written planning for myself.

I need less paperwork and fewer web forms and identities, but the Google Assistant only promises more of that crap.

I'm never buying one. It's a sacrifice of privacy for zero to marginal gains in convenience.

If you can't come up with uses for it, you weren't its target audience in the first place.


Ignoring your derisive tone, the statement "most people get through their daily lives just fine without it" applies to every new technology. Yet here we are, typing away on the internet.

To my mind, leading a simple life is enjoying a burger at a restaurant/bar I frequent already. Simplicity _is_ accepting that Google algorithmically noticed a trend and just helped me do things I already do.

Yes, traps are usually designed so that it is simple to get into them. It is not that cheese is bad, it is that you are trapped.

Are you comparing something designed to kill a rat with something designed to help me go to a burger place I like, or leave on time for work?

Yes, because when one has already decided that feature $FOO is a trap, any further discussion is likely to be limited to describing how "yes, just like a trap is designed to...so is the thing we're talking about" whether the analogy is apt or not. Something something supporting a narrative.

Yes. How does Google make money of this service again?

By having burger places pay money to get on the list of places it helpfully gives us when we want to eat a tasty burger. I still get my tasty burger.

Before replying, you could at least have made an effort to understand what I meant with the distinction between simple and easy.

If you do not care what I say, why even reply?

Sorry, who's mind? It sound like you are renting it out.

Presumptuous much? Comments like yours are what makes discussions like this so difficult, and so much less interesting.

Do you never use digital tools to outsource mental effort? Seems like a similar argument could be made for using a calculator.

Calculators provide you a completely fair assistance with your query. There is zero bias in a calculator. If you ask it what two plus two is, you're going to get four.

Google is designed to sell ads, and subtly influence your behavior towards the most profitable results. Please do not confuse a fact-based tool with an ad generator.

> subtly influence your behavior towards the most profitable results

This is the very common theory that a company will (shadily) try to offer you a worse product to make more profit. It fails to account for competing companies that would jump on that opportunity to offer their better product, and get the market share.

But what's funny here is that the suggested alternative is to not get any product at all. As in: "Poor OP, didn't realize that it wasn't really him who was enjoying that burger he was enjoying."

"Worse" is often subjective. And the problem is often just the removal of the possibility of a better product to take hold. For example, Google prioritizes Google services. It gets you on as many Google services as possible. Let's use, say, that it pushes you towards Play Music when you search for songs.

Maybe Play Music is the best thing. Maybe it is not. Neither of us can answer that. But if a definitively better product comes along it will have no way to make a foothold because Google is still pushing everyone to their own product, from their other product (Search), and even when people try your product, if they use Google's other products, they'll tend to stick to other Google products.

Honestly, the worst problem with companies like Google is vertical integration. The ability to provide a wide product line where you integrate best with other products your own company makes has an incredibly chilling effect on competition, and therefore, innovation.

And if your theory that companies prioritizing results for profit would lose to companies that always prefer the best products, why is DuckDuckGo still in what... fourth or fifth place?

> And if your theory that companies prioritizing results for profit would lose to companies that always prefer the best products, why is DuckDuckGo still in what... fourth or fifth place?

You'd need to argue that DuckDuckGo's search results are better; I don't think they are. That's what made Google first among many competing search engines, before there was even a clear business model in it. Today the incentive to outperform is bigger.

If a product Y definitely better than X comes along, and only Google Search fails to rank it higher, people will start thinking "I rather search on Bing too, as it finds better products in this category".

That's the thing though. I reject the notion that you ever actually make a choice. I would posit that 100% of the actions you take are simply the deterministic reactions when the current world state is filtered through your brain. Then, after the fact, your brain gets busy inventing a reason that you took a particular action and calls it a "choice" when really you were just going to do what you were going to do anyway.

"I ordered this burger because I was hungry and it tastes good" vs "I ordered this burger because Google was able to successfully predict that I would be receptive to having burgers, or the idea of burgers, placed in my environment"

Sure, but philosophical musings on the nature of free will aside, there's a practical worry about the amount of power a private company has over your actions. I'd rather be ordering burgers because they taste good than because a company wanted me too--I expect this will lead to greater happiness for me in the long run.

Yes, but only because your happiness metric maximizes when you exercise your freedom of choice.

Other people's happiness metrics work differently, and all popular web services are popular precisely because they satisfy the unconscious desires of the majority of people.

In effect you argument is that we don't have free will, right?

I wonder what is then causing inefficiency when we read restaurant's menu and can't decide what we will have.

I'm with those who think we make choices and decisions far less often than we think, but that we still do make them.

i am no longer intrigued by the privacy discussion but the actual possibility that we are just consciousnesses controlled by the google hivemind.

this is like absolutely full on plugged into the matrix world. and we're living right in it.

these guys are like the ones who've taken the red pill, and gone on to find out how far the rabbit hole is going.

(edit: i'm even more intrigued by the possibility that the future is not just the matrix singularity, but an oligopoly of several large singularities, all fighting to plug us in)

I think for quite a large number of people, allowing AIs to make decisions for them will probably be better for them.

When the AIs are working in service of corporations this seems incredibly unlikely.

We already see what happens when peoples decision making is coloured by mass media advertising. An obese population trapped by debts taken out to fuel consumption.

It is in other peoples best interests for you to work like a slave, be addicted to unhealthy habits & run up vast debts in order to buy their products.

We keep allowing those with power to distort the markets gaining themselves more money and more power at the expense of the little guy. I don't see any reason why AI in the service of the powerful will do anything but accelerate that.

Imagine some day your doctor advises to cut back on burgers and alcohol. Is Google going to incorporate that advice in its bar recommendations?

Is it Google's responsibility to? I would say no. If algorithms detect that an individual is going to a bar every Monday and Thursday night, and then starts providing information about said bar on Monday and Thursday nights I don't see the problem.

But I think it would be a problem if every Monday and Thursday night Google Now started providing information about AA meetings in the area, instead of bar information. It's up to the user to make the choice, Google Now just detects trends and then displays information based on those trends.

I go to the gym every Monday, Tuesday, Thursday, and Friday morning. And each of those mornings Google Now tells me how many minutes it will take me to get to the gym from my current location. Should Google Now start giving me directions to the nearest breakfast place instead? No, not unless that starts becoming my pattern.

If you're trying to change your lifestyle, it's more difficult when you have a bad friend constantly enabling the behavior you're trying to cease.

Google may not have a responsibility to be a good friend, but personally I'd prefer not to have a bad friend always following me around, thus I'm a little less excited about this feature.

You can just tell it to stop. It's not hard.

I think many would rather tell it when to start instead. What's hard about telling it to stop is when you can't tell it's started because it's something more nuanced than the obvious diet plan.

It may not be their responsibility (although if it had that information it would be the morally correct choice). However, regardless of the responsibility -- the CEO of the company saying "we're going to make your life better!" by an AI pushing products is almost certainly not going to make your life better.

> Should Google Now start giving me directions to the nearest breakfast place instead?

That may depend on how much Waffle House pays for advertising, and that is the problem.

Why not? As long as you're clicking their ads, they'll make money regardless of whether you're buying a burger or a salad.

That rather depends on the objectives of the AI.

If you replace "AI" with "marketing" would you still make that statement?

If you replace "ai" with "your spouse" would that change be as intellectually useless?

don't you think thats a pretty severe statement wrt to free will and agency? if i'm just a consumer wired up to a machine thats deciding whats best for me (even with the best of intentions), doesn't that make me less human?

should I just be a actor playing through a set itinerary of vacations and movies and burgers and relationships? maybe you think its that way already, except less perfect than it might be, but thats a pretty frightening notion to me.

The same argument was historically made to justify slavery.

And to justify the continued existence of the electoral college.

Given all the other points in life where, despite my awareness, I don't have much choice, how is an AI just directing me really any different?

My culture, education and skills limit what work I can do.

Our culture places limits on a vast number of experiences. On the road and the only thing is fast food? Welp, eating fast food. Live somewhere that only has one grocery store or cable provider?

I don't really see AI in the form Google is peddling as really all that much different. We're just 'more aware' that the world around us is really guiding us.

I may be somewhere new, and can only see the immediate surroundings without a lot of exploring. And let's be real, in the US, most cities are the same when it comes to restaurants/hotels and such. There are differences in culture but we don't usually see them if we're just visiting. Not in a way that matters.

Google will let me know that the things I prefer back home? there are equivalents nearby.

Fencing ourselves in is what we do. Who knows, perhaps a digital assistant would help us stick to our personal goals and decisions better. Rather than just having to accept what's there.

Almost all decisions are unconscious decisions. whether or not Google is involved. We usually rationalize our reasoning after the decision is made.

> I feel we are widening the field of unconscious decisions and I see that as inherently bad

I'm curious why you think this is bad. I don't necessarily think it is good but I also don't necessarily think it is actually happening

Which news-sources do you use?

Which news-sources are you going to learn about?

Which news-sources are you for some reason very unlikely to encounter?

Now apply a real-time AI filter-bubble, able to also include government policies in its decision-making, onto those questions.

I believe the most important thing in life is thinking. I believe a key element of thinking is looking at "easy stuff", the stuff we just live with every day and don't think about, and for some reason be forced to think about it and make it simple.

Take the Snowden-leak. We lived a nice life being the good guys and that kind of surveillance was publicly thought of as conspiracy theories. Suddenly we were forced to look at what was going on. How much of it are we okay with? On the grounds of what principles and tradeoffs? This is all very unpleasant, but we're all better off for facing those questions and work towards new principles. We take a chaotic gruel of cons and pros, and try to hammer them into a few simple principles our societies may function by. For instance, the separation of power into 3 has served us well.

I fear that we end up in a world where raising such unpleasant questions becomes almost impossible - and we'll never even notice. Not because of AI (I believe AI to be inevitable and fascinating) but because of the way AI is used.

Living a life assisted by an AI, made and paid for by someone else, seems like the epitome of naivete to me.

> I fear you will be unable to recognize when that burger was your choice and when it was a reaction.

Maybe the illusion is that it was a choice . . .

Not far from the mark. People have quite different behaviors when asked "what do you want?" vs a constant stream of "do you want X?" questions.

What I want mainly from Google is more and easier ways to customize my level of privacy. The article touches on the EFF's stance against incognito modes briefly, but it's an important one; I don't want lack of monitoring to be something I start a separate session for, with a logo of a creepy dude implying I should use this only for spying and pornography. I'd like to get as close as possible to an assistant that remembers relevant data on where I go and how long it takes, but ignores my browsing history to psychologically manipulate me into buying things--of course, that needs a different revenue model.

Does https://myaccount.google.com/activitycontrols?pli=1 do what you're asking for? It's fairly fine-grained.

My experience is that Google app and system updates have a tendency to force/trick/nag you into giving up privacy.

As example, the old weather widget from Google's "News & Weather" was replaced by Google Now. That provided a similar experience for some time but then stopped working with another update that required search history to be enabled and/or some other setting in privacy control.

Also the launcher integrated Google with a system update(Moto G line of phones). I have since replaced launcher, browser, search app (all with open source replacements), weather app (with a paid service). Convenience has suffered...

Indeed; after a time they required search history to be turned on in order for commuting traffic information to be supplied, which was not initially the case.

At that point I turned it on but deleted* the search history each day, until such point as they changed the delete controls to be more of a nuisance.

I now use DuckDuckGo and an iPhone instead.

* Or so their site claims.

I love that this uses 'pause' vs 'stop', 'off', 'disable' 'deactivate'.

Your location history tracking is paused..

There is a difference between 'the user requested to deactivate the service', and 'the user paused the service', he may or may not wish to continue using it, gray area right?

It can probably save them from a legal mess if they 'resume' it in future updates.

As was mentioned below, in Google's case I'm particularly looking for ease of use and a UI that lets me turn things on and off quickly and intuitively. That might sound odd, as if I care deeply enough about my privacy to complain about it, but not enough to quickly switch the settings on my Google account, but I do think it matters.

The more specific work I need to go through to set up my privacy, the less inclined I am to do it. If I didn't think I was able to be manipulated psychologically in this way, well, I wouldn't worry about advertising at all! If I were to ever do something politically dissident/personally embarrassing on the internet (not that I ever have of course) I'd go to the trouble of ensuring encryption and being hard to track, but I think it's important that I'm able to say to Google "Hey, I'm cool with you telling me when I should check off work to hit the bar, but it's super weird that you know what I should get my Mom for Mother's Day."

Of course, the simplest way of making a system that's both fine-grained and intuitive might involve... more AI, so I'm not sure how to crack that issue.

The link above is very intuitive. One toggle disables search history in all Google products, another one removes location history, etc. You can also click on "Manage activity" for more fine-grained details in some apps.

Did you check that link? It looks fairly intuitive to me. Given that I'm having a hard time parsing your response. However if there's some specific setting in there that's not available, that would make sense.

I think each individual action is fine, but I don't understand how it's easy to manage privacy as a whole. Each aspect of privacy is separate, and can be finely customized in one particular way per session. There's no way to set different arrangements of settings for different times, or even just a button on mobile to whack that indicates "Ignore my data for the next few minutes."

Those are good ideas.

You shouldn't have to spend that much time managing your privacy.

I think eventually they add so many capabilities and so many fine-grained controls and it becomes impossible to manage the UI or to find the right options. Even looking at the settings for Android privacy / settings, it's pretty hard to find anything.

This is by design, so that the majority of users are confused and leave the defaults as is, enabling Google to do whatever they like.

> logo of a creepy dude implying I should use this only for spying

I never thought of that before. But, what a subtle way for Google to dissuade people from using a tool that could impact their revenue.

When it first told me it knew hiw long my commute would take, I realised it was creepy as all hell that the people (in another country with few protections on data) providing my phone software knew enough about me to tell where I worked and when I was going there.

And it annoys me that on maps, when you turn off all the spying capabilities there's no fallback to local history. You either share it with us or you get none.

  > ... it was creepy as all hell that the people [...] knew enough about me
No people know. Software metaphorically ‘knows’, but don't anthropomorphize it (it hates that).

But it's accessible to people, that's the issue. Not my tech, but systems far away that belong to others.

Exactly. You're expected to naively trust 'the algorithm', because people are nowhere to be found near your supposedly anonymized data.

Speaking of which, anyone following today's stories about the Yahoo email scandal, the pressure on the folks who own Signal, and recent litigation from Microsoft against government gag orders?

But, let's go back to talking about none of us have free will and talking about how clever Now is.

Failing to provide local history is essentially one of the dark patterns to getting you to turn on their privacy collection. Most things Google requires it for could easily be done outside the cloud, but by making things depend on the cloud, and then telling everyone you can only do it with the cloud, you convince people that they need the cloud. When in reality, they never did.

GPS navigation devices with much less storage than a phone have been more than capable of what Google Maps offers for a long time. There's essentially no reason for it to do anything with the Internet except getting map updates.

And traffic.

I use Google Maps every single day to get to and from work, simply because it knows how to avoid traffic. 10% of the time it saves me half an hour on my commute.

It may be OK for you but there are at least three real concerns here:

1- There is no way to set your privacy level.

2- Things that Google/Siri/Alexa know about you are not limited with the name of the bar you go frequently. They know much more about you. And you don't know what they know. The sky is the limit here.

3- Things that they know are not limited with you personally. They know about you, your family, your friends and all their interactions. They know very much about the whole society.

1 - Sure there is, Google has fairly fine-grained tracking control. Not perfect, but as another commenter noted, this is a double-edged sword, as _too many_ controls can conversely hinder user control (see Facebook's privacy revamp)

2 - My point is that I personally am OK with Google's AI knowing more about me. I respect that others aren't. I'm not naive in my acceptance.

3 - I don't really have a response here.

Sure there is, Google has fairly fine-grained tracking control.

The privacy control where I disable location tracking and half a year later when I look in Google Dashboard I see months of travel history?

I respect that others aren't. I'm not naive in my acceptance.

So what do you do in a situation where your use of Google's data collection also affects people who do mind it? I would not be comfortable visiting a friend with an always-listening device like Alexa or Google's equivalent.

I nuked my paid Google Apps account a couple of months ago. I had enough of their total disrespect for privacy. E.g. conversations that I had in Google Mail (which is protected by the Google Apps agreement) were used for suggestions, etc. in Google+ (which is not covered by the Google Apps agreement and uses data for targeted advertising).

> So what do you do in a situation where your use of Google's data collection also affects people who do mind it? I would not be comfortable visiting a friend with an always-listening device like Alexa or Google's equivalent.

I'd turn it off if/when they ask. I don't think that's unreasonable in the least. I'm not responsible for enforcing everyone's privacy preference, but I also respect them and will accommodate guests in my house.

How do you know the conversations were used for suggestions?

It could have been a visit to a website.

I don't know about Google, but I know that Siri and Alexa only collect and send data when you ask it to.

You can monitor the traffic of the Alexa and see that it is only sending data when you ask it to do something, and furthermore, Amazon gives you a log of everything you've said to it and it recorded.

And you can also get a log of everything you've ever ordered from Amazon, but there are still loads of other signals which aren't visible to you, as evidenced by the fact that you browse a product on amazon.com today, and now for the next week, you see that product advertised back to you all over the web.

Others may be reviews left, reviews voted on, prime video consumed, audio/video/book samples consumed, kindle activity, how long you spend on a product page, how you scroll on it, the breadcrumb of how you got there, and surely dozens more.

Home claims this as well, and I for example have a log of audio clips and transcriptions of all of my google now searches.

"Hi Ubercore, Google and your health insurance company here. We are worried that you are frequenting a facility that serves too much alcohol and wings. We care."

I really wish this was up top. I can't believe that the top rated comment on HN is a thinly veiled "if you don't have anything to hide, you don't have anything to worry about".

My impression is that (in the main) younger people have lower privacy thresholds than older people. Not for everyone (of course). Just on average.

My impression also is that most early adopters of this kind of technology are younger people. (again: mostly)

So this brings up an interesting question about the future. As the young early adopters age, what will happen?

a) their privacy thresholds will also increase and they will have a "oh holy crap" moment in the future, where as a middle-aged or older person, who has lived a now much richer and problem-laden life, they will realize that google (and/or other co's) have what they consider now, as too much personal information about them,


b) they will keep their young-ish privacy thresholds as older people, and in general, across society, people will have lower thresholds than exist nowadays. In other words the world will change.

My money is on a)

Conversely, there are a fair amount of older folk who are very much okay with government surveillance and a general lack of privacy (particularly our inherent rights) because, e.g., they have "nothing to hide"

My impression (in the main) is that younger and older people have different views on privacy. Older folk might be creeped out by Google knowing their schedule, but okay with the NSA or FBI or whomever reading their emails because "because terrorists" whereas younger folk are more likely to balk at the latter, but very much okay the former.

Do you think my opinion is accurate? I'm curious because to some extent I completely agree with you.

Not OP but I think you're correct. Older people trust the government but the younger crowd trusts corporations.

I don't think A will necessarily happen for most. We are a product of our experiences. If, as we grow up, we get comfortable using always-online technologies and never suffer any consequence from those experiences, I don't see what would motivate us to suddenly doubt these technologies. I am confident B is the most likely situation; that's how societies move forward so quickly with tech.

Devil's advocate says that if you want B to happen, you should start working on more Ashley-Madison-style hacks, to enable more people to have their "oh holy crap" moments.

I met a good friend and current employer because Google Now told me about an event I might be interested in a couple of years ago.

Of course it also once told me how long it would take to get to an ex's house from my current girlfriend's place.

How about an 'OK, funny once' command?

As a possibly humourous thought along those lines, imagine explaining to your future child they were born because of a Google Now reminder. ;)

(eugenics through large scale suggestions anyone? ;>)

:-) do you know that getting pregnant isn't that easy and many couples track the cycles to see when their fertility is higher?

There are even apps to that... It wouldn't surprise me if someday this tracking was learnt by Google

Heh, uh, oh shit.

This isn't funny at all. "They" (or Skynet) could totally do this and who would be the wiser?

  > How about an 'OK, funny once' command?

I hope that's a Heinlein reference.


My experience has been quite the opposite in terms of convenience and relevance: I commute by car and train (for some parts of the same journey) and google Now, google Maps (etc) have been totally useless there: telling me about traffice jams when I'm in the train, not telling me about train delays, etc... it now somehow thinks my home is at the train station, it tells me the last bus home is leaving soon when I've been home for hours, also Google Now's insistence on bombarding the leftmost pane on my phone with the most click-bait articles ever, often about things I had a passing interest in months ago, is just laughable. I would be glad to give Google some of my very precious privacy, implementing some countermeasures like multiple and burner identities as needed, if I thought they had any chance of actually providing real value. So far they have failed miserably, I am not sure the economics of providing a really useful service there for free just with marketing information as source of income work now, or for a very long time. You'd need strong AI to actually help my day to day life, with solid non obvious guessing based on many very local and specific factors. I guess as long as people kind of believe that this future is coming, they may tolerate the invasion and forget about the promises.

To me it's pretty simple. If you don't like what Google is doing, don't use their services, and don't buy their products.

But as mentioned above, if you don't want Google to potentially track your behaviour and preferences, "don't use their services" encompasses "don't send email to anyone with a gmail address".

Which is the digital equivalent of "Don't walk out your front door".

My whole point in posting originally was to bring up the fact that the interesting discussion of different thresholds and having a spectrum of options is being lost in binary "privacy vs complete corporate control". It's not binary, and it does everyone a disservice to act like it can be for everyone.

We live in a world of Privacy Theater. Everyone is always complaining about "their privacy" as if they still have any, and it's kind of amusing.

Your phone's microphone can be turned on remotely and listen to what you say (I know several startups that do this). Security/traffic/drone/satellite cameras are everywhere. You are being watched literally all the time, but to think the watchers actually care about your personal life indicates a pretty inflated view of self-importance. We're starting to complain about it about 20 years too late.

Which IMO is a valid thing. If you are sending something to someone, they are free to do what they want with it. You can't force them to treat it differently.

If i'm on gmail, then you need to talk to gmail to talk to me. It's a tradeoff you'll need to make. If you want you can GPG encrypt the email, but there is nothing from stopping me (legally, morally, or otherwise) from just decrypting it and replying with the contents, or saving the decrypted message in my google drive.

I dunno, it's not quite that black and white. If I send someone (say) a poem I wrote, even if unsolicited, I retain the copyright and the moral rights in that poem. The recipient doesn't get carte blanche to reproduce it just because I sent it to them.

So, by the same token, does Google have the rights to profile people even if they haven't consented to that? One answer, as with copyright, is to see what the law says; and it's very possible that the law says no, especially in the EU. (I've not researched it; others will doubtless know much more than I do.)

Or Google could just seek to do the right thing and not profile people unless they've opted in. But my definition of the right thing may well be different to theirs.

At least in Germany, Google definitely does not have the rights to profile people that have not opted in. The interesting part is

1. finding someone who is not using any Google service at all, and

2. finding their data in a Google database.

It's also going to encompass "don't enter your friends' homes".

The sad part is that the user turned off Google Now because he didn't want google to know about the bar he visited. Google was tracking and recording his location before Google Now, he just didn't know it. It's still tracking it after he disabled Google Now.

The only difference being that the user now believes that she isn't being tracked now.

I hope that there will be a day when Google and Facebook will combine forces and work on a sequel to "The Lives of Others" [0].

[0] http://www.imdb.com/title/tt0405094/

> I hope that there will be a day when Google and Facebook will combine forces and work on a sequel to "The Lives of Others" [0].

Yeah, that's a great movie.

Yeah, but that is also the equivalent of 24/7 surveillance of all locations you visit. Google will end up figuring out whom you sleep with, etc. from that information.

Pretty much your only privacy is in your head at that point.

I'm not sure that is a "threshold" of privacy but rather a "I am okay with 24/7 surveillance of all of my activity."

hypothetically: you express radical political ideas to your friend with the expectation of your statements remaining in confidence-- but google was listening. now, your feed recommendations steer you further down the path google thinks you were already on. you are ready to attend a protest and perform civil disobedience, as google now knows based off of your interest in what it has been suggesting to you. it suggests (as facebook does now regarding making events for birthday parties etc) that you and some other people form that protest, and, because it said so, you do it. except it's a trap. the police's google feed tells them that some undesirables have planned a protest, and you're imprisoned.

is this story unrealistic, or has it already occurred?

I remember the huge smile I got on my face the first time Google Now picked up on the fact that I went to the same bar every Wednesday evening.

One Wednesday afternoon, at work, I got a notification saying "Travel time to the Lion & Crown". The first thing that ran through my head was "oh my god, I'm living in the future".

I am actually quite uncomfortable that my stock Android is making suggestions on how long it should take for me to get home or work (when I have never explicitly mentioned that it is my home or work).

The problem is that I want to use Google Maps so what choice do I really have?

Sure I use a dedicated gmail for my phone but that really does not help much.

HERE is an excellent maps alternative, I'd highly recommend it.

You should be glad that it doesn't remind you:

"Hey, it's a been a while. Why don't you go to ... today? Traffic conditions are favorable too."

Their business model is to attract and sell your attention to other companies.

Given that, and the existence of pervasive surveillance and data mining, the above is inevitable.

Or as one fellow once said on HN with regard to self driving cars and advertising as a business model:

"Oh you wanted to go to the bank today? Fuck you, we're going over by the mall" doors lock shut

Or maybe:

"I was feeling bored, so I drove down to the mall. Look what all did I get!"

And then prepares to rush the owner to the hospital.

Personally, I wish it did have reminders like that. Offer me a coupon too, while you're at it.

I would not want Google knowing I sometimes drink too much, or that I do so and get behind the wheel of a car. Easy inferences it could make, given the time I spent at the bar, and the purchases on my credit card. That could even have consequences for the cost of my auto insurance. Edit: and health insurance.

But wouldn't that be better for us as a society to know? The outcome sounds positive here.

I actually thought the most interesting point this article raised - for me at least - is the implict branding associated with the "OK Google" command. All privacy concerns aside, if I'm going to have a "personal Google" I want to be able to thoroughly personalize it.

but I think the fact that it's different for everyone is being lost in articles like this.

I think that it is different for everyone is completely and utterly obvious. It's clear the author doesn't think it's OK, but that it's his/her opinion.

This location history is really bad though. I added a test gmail account to my device a few weeks ago, but didn't remember location tracking was a per-account setting - now Google has nice big logs to hand whoever wants them and I can only delete them on a day-by-day basis (from the Android app at least).

Extremely annoying. This sort of thing should not be acceptable, an honest mistake results in every place I've been being logged in such a way that anyone with access to my Google account, access to Google servers or with a subpoena can have my full location history in a matter of seconds.

This needs to be a big red option every time you add an account "we're gonna log everywhere you go and hand it over to whoever we feel like, you cool with that?". It'd be different if the log and analysis were done only on my device, but doing this on Google's servers is completely unacceptable by anyone with even the weakest standards of privacy.

Find Google settings in one of these places (depending on your device):

In your Settings app Settings, scroll down and tap Google. Open a separate app called Google Settings Google Settings.

Tap Location and then Location History.

At the bottom of the screen, tap Delete Location History.

Also https://maps.google.com/locationhistory allows for delete all history.

What makes you think that deleting location history actually deletes the history? Reminds one of the group of students called Europe v. Facebook [0]:

  Schrems described the file obtained through a legal request as
  a 500MB PDF including data the user thought they had
  deleted. The one sent through a regular Facebook request was a
  150MB HTML file and included video (the PDF did not) but did
  not have the deleted data.
[0] https://searchenginewatch.com/sew/news/2114059/facebook-file

Definitely, and this is why it needs to be a global option to disable location history - just sending it to them in the first place is the problem!

Can't you simply disable the GPS radio?

I still want maps, I just don't want my every move logged or sent to Google.

Yeah, that's what I remembered, but it seems they replaced "Delete Location History" with a "Manage Activities" button which opens Google Maps and shows you location history, allowing you to delete it but only one day at a time.

EDIT: Okay, found it. Follow your instructions then open that "Manage Activities" -> Menu -> Settings -> Scroll down -> Delete All Location History.

You _must_ do this for every account if you have multiple.

  You _must_ do this for every account if you have multiple.
As I would expect? I don't know of any service that has a "And also do this for all my other distinct acounts too".

A global "turn off and delete all location history on this device" would be best, really.

At least for turning it off I expected it to be phone wide, not account specific. Deletion yeah, I suppose you're right.

It depends how you view what you're setting.

Do you think you're setting a behavior of the phone, or a behavior of your account?

I guess it could be both but I view it first as a setting of the account since you can also be location tracked perhaps by a browser, or your Wear device? That would be my first go-to on that setting but I guess it can be seen both ways.

Wear devices have to be paired with a phone and browsers don't really have the constant access they want for this.

just having a mobile device you're being tracked in Australia now the Government now also has access to this data from the network providers the only way to be free is to not carry these devices

What I think is interesting is that many of us nerds have probably innocuously fantasized about having a Star Trek-like AI assistant with us, but now that they're taking the first steps towards that, we're starting to realize that in order for it to do everything for us, it has to know everything about us, too.

Nobody was thinking about "the cloud" back in those days. Back then, your data, you programs all lived and ran on your own computer in your home. Most people didn't go online, and if you did, it was mostly to read and download data to use locally on your own computer. Connections were intermittent and slow. The idea that your own data would be stored online was almost unimaginable; even using network-depending applications like usenet or email involved downloading everything first before using it. Online applications were hardly even dreamed of.

Our expectations of how "Star Trek AI" would actually be implemented were completely different than how highly connected cloud-based services like Google Assistant work today.

Anyway, the point being, if the assistant lived entirely in your own computer, it would entirely different. Most people are not concerned about what their "computer" knows about them, they're concerned about what companies and their employees do.

> Our expectations of how "Star Trek AI" would actually be implemented were completely different than how highly connected cloud-based services like Google Assistant work today.

The trope-namer (Star Trek AI) was a ship-wide AI - when considering the ship sizes, it definitely is closer to the "cloud" model and not limited to a private instances on officers' bunker/bridge terminals/tricorders. Perhaps a hardcore Trekkie could answer this question: is there any canon that defines the AIs scope? Is restricted to just one ship, or could it possible be a Federation-wide presence with a presence/instances on ships?

The AI for the Enterprise D was run from three computer cores on the ship (two in the saucer, one in the engineering section), made up of Isolinear Chips subjected to a small warp field (iow, it makes computes using light faster than the speed of light). Subspace communication bandwidth is too limited (and potentially affected by latency, since it had to travel through repeaters throughout the galaxy) to provide realtime cloud computing as we experience it.

There are some cannon exceptions to this (such as in Nemesis where the subspace communication interruption affected the star charts), but even then the functionality of the ship was not impacted.

The Star Trek ships are very analogous to our own ocean-bound ships, where satellite communication is possible almost anywhere, but they don't rely on it.

So, yes, the AI is completely confined to the ship.

What about when someone was (permanently) transferred between ships?

Was there ever an indication that their AI-level data was transferred along with their personnel file? For example did the replicator know what food to offer them on day one?

If so, then it's seems reasonable to assume that the Enterprise's AI data was backed up at Federation HQ during routine maintenance, and that the "IT department" at Federation knew exactly what you liked to do on the Holodeck.

> did the replicator know what food to offer them

Through specific indication from the user. Recall the constant utterance of "tea, earl grey, hot"?

Ultimately, I imagine the user's information (documents, etc) was passed directly between ships, or through (as you say) Federation HQ.

> Enterprise's AI data was backed up

Ultimately, I think this is where AI will differ from ML. An AI won't have data that isn't a part of the AI - i.e. you couldn't separate out information specific to Picard from the rest of the AI code. An AI might be able to "scribble" down some notes about interacting with Picard and pass them off to another ship's AI, but the second AI would never treat Picard quite the same way as the first, even with those notes.

This stems from my belief that how ML interprets data is different from how an AI would. If you were to copy all of the data used to build a ML model and apply it again, you'd end up with the same ML model. An AI, on the other hand, if built twice twice from the same data would end creating two separate AIs.

Thank you for that very thorough answer!

I never got the feeling that there was a lot of mistrust of other federation people.

for example, the Star Trek universe didnt seem like a universe where you had to shop around for a trustworthy mechanic, who wouldnt overcharge or over diagnose. (e.g. headlight fluid)

Maybe the implicit trust of other people was integral to the AI being successful in that universe.

It's definitely a per-ship thing, because there are several episodes where they have to down/upload updated data to/from the Federation (at least in TNG canon).

I also think the comparison isn't perfect, because Federation vessels (in my mind) are similar to today's Navy vessels. All onboard systems are connected to other onboard, but opsec demands the ship systems not be influenced by external actors.

And a big point is that on a Navy-analogous vessel, it's reasonable to assume most activity on the ship is being monitored, for the safety of the ship and crew. The fact that the ship knows where everyone aboard is or that they can pull up the full Starfleet record for anyone who's ever been in Starfleet is not surprising, this is the military* and records and accountability are a big deal. But there's nothing to indicate that Federation civilians are monitored to that extent, and I'd argue enough episodes are strong on fundamental individual rights, that it's hard to imagine Federation life for civilians being a surveillance state.

* Don't start that argument. Seriously.

> But there's nothing to indicate that Federation civilians are monitored to that extent

The clearest example of extensive off-starship monitoring (within the Federation) that I can think of in TNG is a civilian (though, to be fair, a civilian in a role analogous to a "defense contractor"), Dr. Leah Brahms.

> I'd argue enough episodes are strong on fundamental individual rights, that it's hard to imagine Federation life for civilians being a surveillance state.

Actually, I'd say that its quite plausible that the Federation is a "benevolent surveillance state", that is, one with pervasive monitoring but a very low incidence of "serious" abuse (that is, the kind that substantially limits practical liberty -- casual intrusions on privacy may be more common.)

While the Federation seems keen on "fundamental individual rights", it doesn't seem to exactly mirror, say, some modern views on what those rights are -- and not just in terms of privacy.

I saw you give the Brahms example in another comment before I posted, so I am not surprised to see you bring it up. But I think you answered your own point. She worked heavily on the Galaxy-class starship's warp drive, which would be a relatively classified project that they would be very unhappy if the Romulans or any other hostile parties were intercepting data on.

And arguably, if she was working at Utopia Planitia Fleet Yards on the Galaxy class project, she presumably worked on a Starfleet orbital facility (technically, a number of facilities) over a span of years, where certainly enough data would be collected to make a poor replica of her personality, as in the show. I don't see anything suggesting specifically outside of basic biographical data, that she was being monitored in her civilian life.

La Forge, having interacted with that hologram extensively, and having surely read Starfleet's records... apparently didn't know she was married.

Even then, IIRC, it was limited to the ship(s). If you didn't want the ship to follow you, you could simply avoid entering the ship.

We're starting to lose that ability now.

I feel your message is the most important in this thread because it's the crux of the whole concern about privacy and the cloud.

Where technology has failed us most is in the utterly stagnant evolution and maturation of secure private networks. The following is a utopian notion, but had private networks seen as much R&D as the public clouds, they would be significantly less cumbersome than today's clunky VPNs. Imagine all of your devices collaborate directly with one another and with you on your own secure private network—no central cloud servers needed. Your personal assistant is software running on a computer you own rather than a third-party's centralized server.

I still feel this ideal will eventually be realized, but for the time being, no large technology company is willing to take the necessary risks to buck the trend of centralization.

The biggest fiction propped by up centralization and cloud proponents is that it would be impossible to provide the kind of utility seen in Cortana, Siri, Google Assistant, Alexa, et al without a big public cloud. A modern desktop computer has ample computational capability to convert voice to text, parse various phrases, manage a calendar, and look up restaurants on Yelp. Absolutely nothing the public clouds provide strikes me as something my own computer would struggle to do (to be clear, I would expect a local agent would be able to reach out to third-party sites such as Yelp or Amazon at your command in order to execute your desires, but they would do so directly, not via an intermediary).

A few years back, when Microsoft was at the beginning of its Nadella renaissance, I had hoped it would be the first technology titan to disintermediate the cloud and make approachable and easily-managed personal private networks a thing. Microsoft's legacy of focusing on desktop computers would have made it well-situated to reaffirm your home computer as an important fixture in your multi-device life. They could have co-opted Sun's old bad tagline: "Your network is your computer." But they elected to just follow the now-conventional public cloud model, reducing everyone's quite-powerful home computer to yet another terminal of centralized cloud services. Disappointing, but I think it is ultimately their loss. I suspect a lot of money is on the table for someone to realize a coherent easy-to-use multi-device private network model that respects consumer privacy by executing its principal computation within the network.

>Where technology has failed us most is in the utterly stagnant evolution and maturation of secure private networks.

Not just secure private networks, but secure and programmable personal computing in general. The amount that I can actually do with my workstation PCs, let alone laptops or mobile phones, is now thoroughly restricted compared to problems that require a full-scale datacenter.

I originally enjoyed computing because, so to speak, it was an opportunity to own and craft my own tools, rather than being forced into the role of consuming someone else's pre-prepared product. Now we're being boxed into the consumer role in computing, too.

People at work keep asking me why I reinvent a few wheels here and there on my personal projects. "Why are you wasting your time with WebRCT? Why are you not using phaser.io?"

idk man. Computers are powerful. I like seeing what I can do with them.

And with UEFI-level code on mainboards and baseband software on phones, the era of "owning" a computer is basically over. All you can do is hope and trust that the manufacturer isn't co-opting your experience or data somehow. As someone who grew up hacking on a C64 in grade school, and never stopped, I find this utterly depressing.

Let me preface this by first saying that I absolutely can't wait to have my own personal home automation, AI assistant, etc. on prem without the cloud:

I think that as far as the nascence of these features goes, the cloud model will beat the on-prem features any day of the week for several reasons. Lack of configuration to set up, ease of use from anywhere without network configuration, etc. are table stakes. But the biggest at this point is the sheer amount of training and A/B testing data you can ingest to determine what is useful for your end users.

The velocity of cloud-based products is nothing short of amazing and I doubt that on-prem will compete with the feature set and ease of use of always connected solutions until there are feature-complete, mature cloud versions to then bring in.

As we just learned with Yahoo, though, once the ML models have been trained, they can be disseminated and used without the need for "cloud-scale" data or compute resources.

And, for better or worse, Dragon's text-to-speech is pretty damned good after a rather minimal amount of training.

I don't think there's anything stopping voice and intent recognition from coming back to our personal machines other than the ability to keep making money from having it come up to the cloud.

The cloud is all about scale and only having to rent resources when you need them. If you have your home server you have to buy and maintain and pay for those resources at all times. When you make a quick cloud request you only "pay" for the resources you consume.

When I was working on Google Search what really astounded me is how we could leverage hundreds of machines in a single request and still have virtually no cost per search. The reason was that each search used a tiny amount of the total resources of those machines and for a very short time. A total search might have (made up numbers) one minute of computation time, but spread across 200 machines it only takes 300ms from start to finish.

That's the benefit the cloud will provide. You don't want to have a 1000-machine data center available at all times to store billions of possible documents and process your requests with low latency. If we went to a private-network model I fear that the turn-around time would be a lot closer to a human assistant. You'd ask it to do things and then it would get back to you sometime later (seconds? minutes? hours?) when it had finished it's research and come up with an answer.

>I still feel this ideal will eventually be realized

Quick shoutout to Urbit here.

> The idea that your own data would be stored online was almost unimaginable

Except that's what I did for many years using a computer only as a terminal for an AIX mainframe. My mail was there, I browsed what was the web, used gopher, wrote programs, all stored there.

On top of that, the cloud we have now is commercialized, opaque and constantly under pressure to comply with a government that many distrust, for what I would say very good reasons.

I would like to say that the cloud we have is a privacy concern because we don't know the full scope of data collected, nor what happens to it, nor do we own any of "our" data once it's in the cloud. But not every cloud would have to be that.

There's a perfect world where one wouldn't have to be paranoid about this stuff, but it's not what we have right now.

This is a great point, people didn't imagine a world where we became the product.

This right here. When I first heard about android, I imagined an open system I could tinker with like I can on my PC.

Instead I get the current evolution. I want a 3rd party.

The same thing comes to my every day of usage, I'm still on win 7, and short of leaving Linux(yes I should have already), I can't upgrade without becoming a product.

I want what we all imagined and dreamt of, and I pay for it.

What I won't do, is become a product.

Instead I'm stuck with multiple fake accounts on Gmail, using a pseudonym on everything(including programming contract sites such as upwork), just to keep some semi iota of privacy, and to enjoy the benefits of what we all want.

Imho we need a new major party to emerge, that will charge an initial fee(like windows 7), and let us do what we want with those services(with caveats of course).

But I fear we won't get that anymore.

Learning* not leaving Linux. Too late to edit. Damn you fat thumbs.

That explains my confusion.

It's not that complicated. Use a common distro like Ubuntu and everything's documented online.

I keep meaning to learnt it, and try it.

But my main coding(admittedly amateur and earning very little) uses .net. on top of that most of the games I play to relax are Windows does only(as far as I know for most).

O keep meaning to make time, but it just hasn't happened yet. I don't get paid as a programmer (English teacher), so I need to spend my free time earning money on what I know.

As always, one day when I have money to spare(or time, which is basically the same thing haha).

You can move to Mac, AppleID is not really required.

Without trying to look like an arse, why? I have been on pc since dos, I can hack my way around any pc.

I will fully admit I have not spent enough time in Mac to figure out the file system, but from what (little) I have seen, your not in control.

I like my pc because I can see what sub-processes are running, who is taking up how much memory, install things to where I want and if worse comes to worse, manually change how windows runs. (I apologise if you can do all that in Mac, as I said, I don't have the experience certificate - I bounced off it hard).

So go Hackintosh. At least you don't have to distrust your OS vendor.

To answer your questions, you're in complete control with macOS, you can turn off SIP, turn off Gatekeeper and install whatever kernel extensions you want. Apple doesn't snoop on you like with Win10 telemetry.

The problem with Mac is that one needs to have Apple hardware, you can't even change your RAM sticks to the ones that Apple didn't approve. There is nothing (apart from maybe some fancy Adobe software) that you can't run on Linux just as well or even better.

Totally agree, I remember when that was the promise! I wonder what Ununtu smart phones are like ?

Maybe people could imagine a world where the adage "if you aren't paying for the product, you are the product" would be widely relevant, sure. But not a world where that phrase would be irrelevant, simply because today if you're paying for the product, you're still the product.

That existed at least as long as cable TV, probably longer.

Nailed it.

I think there's an additional nuance, that of Google knowing everything about us. If I hacked together my own home automation AI system it would need to know everything about me too and that worries me far less.

...until it's broken into.

I suspect that chances of would-be burglars or identity thieves breaking into Google data are pretty slim, in comparison to a home-installed system.

OTOH both Google and a private person can be strong-armed by a court order, or even a three-letter agency, to open up their AI knowledge vaults.

The difference is one of targeting.

It's like saying "People would never break into a bank, when they could break into someone's house and steal their stuff"

It may be easier to break into someone's home-brew system, but generally it would be unlikely to happen unless you were being otherwise targetted. Whereas google has a lot of users data, which could make it a more attractive target.

It may be easier to break into someone's home-brew system, but generally it would be unlikely to happen unless you were being otherwise targetted.

Just how "home-brew" are we talking here? If there's any web-facing code that you didn't write yourself, whether commercial or open-source or whatever, that's a target for attackers that just scan everything looking for known vulnerable services.

If it is entirely custom, I'm fairly sure there are a few classes of common security errors that can be reasonably well tested for without direct human involvement. Which brings back the threat of attackers just scanning for all available targets.

Very much so!

If you're a "person of interest", you'll be attacked either way.

If you're just a regular person, like me, you won't likely be targeted. But my chances of getting a collateral damage are much higher in a centralized system: when it's broken into, my data have a chance of being siphoned off along with data of someone less ordinary, who was the target of the attack.

It's wider than this. A "person of interest" will be attacked either way. A "normal boring person" won't be targeted directly, but may be swept up in an attack on a "person of interest".

But there's a third case: "normal boring people" become interesting just by being together in a big group, even if no particular "person of interest" is among them.

> I suspect that chances of would-be burglars or identity thieves breaking into Google data are pretty slim, in comparison to a home-installed system.

The odds of someone physically taking my computer and decrypting my data are zero (128-bit encryption for the win!).

The odds of someone electronically breaking into my computer are higher.

The odds of my data being misused or misaccessed at Google are one: it will happen.

Im more worried about turn key totalitarian nation states having this info and then being able to walk the cat back.

Exactly. For example, see the news yesterday with Yahoo. Imagine instead of just emails, it was all of the data about all of us.

only if you hacked together your own search, maps, calendar, etc. as well.

Maybe someday that will be a realistic endeavor, but it would take a lot of effort to set up and maintain your own personal versions of all of google's services, and integrate them

I think it's possible to create a hybrid whereby your personal AI - has access to those external systems to check public (maps, search) and cloud data (calendar, mail).

But it stores the private data about your location, your searches, your purchases. this could even be an encrypted private fire-walled bit of cloud rather than a robot in your house.

The point being your AI is serving you. And you can delete/edit this private data (or even the whole AI) if you wish.

That is the whole point of Intelligent Agents.


Perhaps using Urbit as the network architecture since it already has pki. As Yarvin calls it, true 'personal cloud computation'.

Really it's the only sane way to live in the 21st century.

This is spot on. I'm okay with having my home automation system tap a weather API to collect data. But I don't need an Internet service to know the thermostat settings in my house.

I've designed my home automation system on the concept that the only route to the Internet is my computer (and therefore, my home automation software). My computer is a secure, well-managed intermediary that can store my data, and decide when and how to receive and send data to the Internet.

The idea of dozens of Internet-connected devices in the home is :terrifying: in comparison, especially considering that badly-secured IoT devices are now powering some of the biggest botnets out there.

My light switch, however, cannot talk to the Internet. It has local-only communication protocols that are simple. It knows how to be told to turn the lights on or off or dim or a handful of other settings, but it's literally incapable of doing anything else... and why would I want anything different? Why should my light switch have Bluetooth and Wi-Fi and software updates and a miniature flavor of Linux... It's a switch!

Your description sounds like a very intelligent cache/proxy.

When my AI can talk directly to your AI, we can transfer mail etc without cloud services. If it knows our social network, perhaps it can remotely store encrypted backups of our data, but only with people who we already trust.

Hosting your own mail, calendar, cloud storage, and even a mapping application is quite doable right now. There are open-source projects for that, and you can have your own physically secured machine in a co-location facility for a reasonably small sum.

It's just not easy to do, both in setup and maintenance effort. Most people would not care about their privacy nearly enough to justify that.

My recommendation for people to look into here is Sandstorm.io. It runs either on-site or in the cloud, and it already can replace a lot of what one uses cloud services for. In time, hopefully open source projects will develop "intelligent" apps that can then tap into that data and do more with it, still within the confines of your secure, private storage.

You don't have to develop, setup and maintain all that by yourself. What we need is a user friendly (dead easy install/setup, no administration) way to use alternatives to these services. Some of these services exist, some not, but it's a very achievable goal.

The only thing we got wrong. We expected the machines that knew everything about us to be ours. Not someone elses.

Well the missing part is the dedication to ideals and to the greater good of all life that was supposed to be core to the federation. I realize Star Trek is fantasy, but the reason people are more at ease with the omnipresence of technology in Star Trek is because you see people living by these humanitarian and noble ideals. People fight and die to defend thems. The right to self and privacy and protection is held very high in the st universe, even if challenged.

When's the last time Google risked itself or business or any tech ceo risked their livelihood for the sake of the greater good? The problem isn't necessarily the knowing everything part, it's who does what with it that's the problem. I can't really think of any company or person with influence in tech that'd be willing to dive onto that bombshell to protect us all.

> When's the last time Google risked itself or business or any tech ceo risked their livelihood for the sake of the greater good?

February of 2001.


The former CEO of Qwest, a massive telco, spent years in prison for insider trading. He says this is because he resisted the NSA's demands to tap Qwest's network and hand over customer data.

The big difference is the Star Trek computer wasn't using its data about Kirk to provide him "enhanced advertising experiences", there wasn't a big corporation controlling the computer and no government was accessing the computer's information.

A truly user-aligned AI assistant would be great. Ideally in the future these things will not be tied to indirect business models, but rather will be something you buy and all data/services will be under your control.

Capitan Kirk was a government employee. It's implied the government/StarFleet could access the ships logs.

In the Star Trek world they had no advertising because they were a communist society. Everyone dressed the same or slightly differently based on rank. It's interesting how the new movies play over that.

In Star Trek you couldn't choose your AI. In our world you can. At the start of their development most of them are targeted at selling you stuff - but the industry is young and who knows where it will go.

It would be more accurate to say that most of Star Trek depicts life in Starfleet, a pseudo-military institution (Roddenberry insisted that it isn't, while war isn't Starfleet's main purpose it shares enough traits with modern armed services to suffice for this discussion). There is always a lesser expectation of privacy for military personnel, even in the US armed services.

While there are elements of communism depicted in the Federation as a whole, those are facets of a global post-scarcity society that has somehow evolved beyond the less "progressive" bits of human behavior. I'd argue that the biggest fantasy of Star Trek isn't warp drive, but the notion that humans are somehow less violent than Klingons.

Indeed, their efforts to run the "Klingon culture is bad" thread got comically awkward in TNG, where they also really hammered on the multiculturalist "all cultures are equally valid" theme. All cultures... except for Klingons, whose barbaric and violent customs are just obviously inferior.

Yes. Also running the AI on local computer made sense because there is no incentive to run it on cloud. Since nothing is gained. Star Trek is set on post scarcity economic society. We will get there once basic income becomes the norm in all countries.

Has Star Trek even a currency? It always feels like a socialist/communist thing to me.

You cannot compare the world's biggest seller of advertisement space with the ST universe. The motivation's aren't aligned: Google/Alphabet want to make sales based on my information.

I agree that I found these oh so clever AI fantasies interesting in my youth, still do to a degree. But I always pictured the data being held inaccessible to humans in general ("Where's my wife right now?") and not in the hands of a golden few with no oversight.

Star Trek's world seems to be a 'utopia' of scientific-military governance. Most of the key players have a military rank, wear color coded uniforms, and appear to be under 24/7 surveillance (which is OK since this is a very nice and progressive scientific-military goverment and you know War On <scary-alien-specie> and all that. :)

Was the society under constant surveillance? or just the part of society the show highlighted?

I don't remember seeing any Star Trek episodes that showed people under surveillance in their private quarters or in bathrooms.

The public areas which were under surveillance on Star Trek tended to be only on military ships and star bases. I don't remember seeing much surveillance in public areas on planets. There certainly wasn't the sense that everyone was under surveillance on every street and in every shop, unlike most people in major metropolitan areas on Earth today. Nor was there anything on Star Trek like the ever-present spy satellites that can see in great detail anywhere on Earth today.

For the public areas which could be observed through cameras on Star Trek, the surveillance seemed mild compared to today because of Star Trek's lack of massive computers and artificial intelligence analysing what is seen for anomalies, using facial recognition, constantly recording everything and having those recordings instantly available for playback, sophisticated search, and computer analysis.

The reading and viewing habits of Star Trek denizens weren't recorded and analysed, unlike those of many people on Earth. Their positions weren't tracked wherever they went, unlike those of many people on Earth.

The so-called "24/7 surveillance" of Star Trek was very limited and even quaint compared to what we live under on Earth today.

Most of Star Trek is aboard starships and stations - I wouldn't imagine that leaves much for privacy.

Plus, most folks think ST is where hippies took over. Center of Federation is in San Francisco...

Given the "War On $species" thing, "War On Romulan Ale" might work as well?

Remember kids: Don't intoxicate yourself with substances we do not condone. :)

In the original show there wasn't a currency but in order to have aliens who exhibit avarice and the worst part of capitalism they had to include a currency (latinum, I think it was called).

> In the original show there wasn't a currency

Incorrect, in the original series, there was a currency ("credits") that was explicitly referenced several times; it was also referenced in at least one, and possibly more, early TNG episodes.

Sometime in the TNG era, Roddenberry laid down an edict that money, including the "credits" that had been repeatedly referenced previously, did not exist in the federation, and so they weren't mentioned again.

Currency was later re-introduced to enable the Ferengi race to be portrayed as greedy merchants.

> Currency was later re-introduced to enable the Ferengi race to be portrayed as greedy merchants.

I don't think that's really accurate; the Ferengi were portrated as greedy merchants focused on profit starting fairly early in TNG without direct reference to currency (gold -- not the later "gold-pressed latinum" -- was mentioned, IIRC, as an item of interest, but not in any context which implied it was used as currency); I think gold-pressed latinum as introduced as a currency in DS9 because DS9's role as commerce hub was central to the theme of the series, and having currency just made telling stories about that a lot more convenient.

It wasn't a currency in the same way that gold isn't a currency. Latinum is supposed to be a substance which cannot be replicated unlike most things so it's valued by, as you say, the avaricious.

> It wasn't a currency in the same way that gold isn't a currency.

Throughout much of history, gold in standardized sizes was a common form of currency. Gold-pressed latinum in standardized "slips", "strips", "bars", and "bricks" is exactly the same thing.

Nope, it's a post-scarcity economy.

Post-scarcity in some ways, perhaps. In many other ways, they're not.

Star Trek still had merchants who sold various wares. That would not be profitable if nothing was scarce.

They still had planets that lacked necessary medicine, requiring The Enterprise or some other ship to go on mercy missions to deliver the meds.

The Star Trek universe had pleasure planets which had highly desirable things that other planets did not.

There was clearly a shortage of starships and crew, as The Enterprise explored alone and not in a fleet, and couldn't just create a hundred others to help it when it was attacked by some alien enemy.

The Enterprise couldn't even use their on-ship replicators to make themselves some dilithium crystals (fuel) when they ran low.

Many articles say that. Do you have a source .

It doesn't though, does it? The only reason this is a problem is that Google's business is still advertising and they act like our problems can be solved by tools made to sell more ads. The moment I could buy an AI service for like $10 a month (it had to be good), I'd trust them with using my data responsibly.

The Star Trek fantasy is, "Computer, what were the principal historical events on the planet Earth in the year 1987?", and it could totally answer that without sending your entire fucking message history to google for deep AI inspection.

> The Star Trek fantasy is, "Computer, what were the principal historical events on the planet Earth in the year 1987?", and it could totally answer that without sending your entire fucking message history to google for deep AI inspection.

That's part of the Star Trek fantasy. But so is, "Computer, locate Commander Riker" and "Computer, use personal logs and personality profiles from compiled databases to create a personality simulation of Dr. Leah Brahms."

I think people also forget that the Star Trek AI was in a semi-militarized scenario where efficiency and information greatly outweighed individual privacy needs.

I think most fantasies are okay with the anthropomorphic AI assistant knowing everything about us, but don't involve the AI transmitting all of it's data back to "the cloud" where advertisers can mine this data or the NSA could listen in with a secret gag ordered wiretap. Probably wishful thinking, but maybe one day a privacy first company will dip their toes into this arena.

We always assumed they wouldn't spill their beans to third parties. I guess that was assuming too much.

Google doesn't "spill their beans" to third parties-- what Google actually sells is the opportunity for third parties to be included in the advertisements Google is targeting to their users.

Google has a strong incentive to not allow their aggregated user data to leave Google-- the behavioral data Google collects is the reason why Google is valuable; if they start shipping that data off to third parties, suddenly the third parties don't need Google anymore.

(Same with Facebook-- they're not "selling" your data; they're selling the opportunity to target you based on your data, but the data itself is too valuable to Facebook to sell.)

> Google doesn't "spill their beans" to third parties

Except the government who gets unfettered access. Not into conspiracy theories but Im definitely not a fan of this (and it goes for all social media and technology companies)

> Except the government who gets unfettered access.

Does you have a source for that?

There was this Snowden guy who leaked some documents about that...

Then it will be easy to provide a link.

>Does you have a source for that?

Yep. They all do it.

[1] http://news.trust.org/item/20161004170601-99f8c

You have put a link about Yahoo! and that link says nothing about Google or others doing the same

No, its safe to say all major tech company's have have NSL's, and are required to allow the government to search and request data, it would be foolish to think otherwise. [1]

[1] http://www.nytimes.com/2016/10/05/technology/subpoenas-and-g...

Are you aware that your new link doesn't talk about unfettered access?

Yes, but this is all we know about now, every day there is a new revelation about what the government has access too that we previously thought they didn't. I think it's naive to assume they don't have access to all of it.

Your claim is that the government have unfettered access to ALL the data.

Can you provide a link to your claim or it is just your opinion because there is no proof of that?

Evidence has been slowly being released for years, what more do you need. Here is my third example for you [1], after this Im done.

[1] http://www.washingtonpost.com/wp-srv/special/politics/prism-...

You still have not provided any evidence for you first claim that the government has total access to those companies. And your last link also doesn't say what you think it says.

Google (or Facebook) is the third party. The first two are you and your computer.

It really is the big problem at the moment with the cutting edge of AI.

ML relies on large data-sets and if anyone tried to release a personal device it simply wouldn't even work, let alone compete with the mass surveillance google/ms/amazon are bringing to bear.

Unless the state-of-the-art in AI suddenly morphs, we seem to be stuck between giving up our privacy or having vaguely intelligent AI.

I personally fall heavily on the privacy side of stuff, but I can see the intellectual and commercial appeal of pretending it doesn't matter in order to get there.

This is simply a matter of AI not being advanced enough at the moment. Take how a human learns: it acquires experiences and information over a long childhood, and from this "data set" is able to do millisecond predictions of future events based off completely new data.

What needs to happen is a company needs to come along and create AI that is trained off of generalized information... some kind of socially accepted public data set... then the trained core is sold as a seed to individuals who then feed it their personal data.

It'll be the equivalent of buying an AI "teenager" and slowly training them to be an "adult".

I thing the fallacy here is that a single person can't create a large enough dataset, while I content they can. Combined with tools like pocketsphinx I think its very doable to have a privacy concious AI system.

That's the thing.

As sensors become richer and the data becomes more valuable to the ML, consumers are becoming more aware of their privacy.

That means to get to a 'good seam' in the future instead of trawling through trash, you're going to have to convince millions of people their interests won't be affected.

That means in time there is an opportunity for a Google-killer with a different business model not based on using the raw data OR using by the use of intelligent agents. Google goes down because its stakeholders are contingent on getting to the raw data.

I think we just envisioned a highly anthropomorphized AI: essentially, a very smart and entirely obedient person to serve as the perfect aide. The Star Trek dream emerged before computing technology was very far advanced, and well before the idea of constantly mobile wireless communication. We thought our AIs would be small and physical, easy for a single person to entirely own and unable to remember more details than a human; instead we got unfamiliar algorithms run on machines far away.

> The Star Trek dream emerged before computing technology was very far advanced, and well before the idea of constantly mobile wireless communication. We thought our AIs would be small and physical, easy for a single person to entirely own and unable to remember more details than a human; instead we got unfamiliar algorithms run on machines far away.

Which is kind of funny, because even if it might be accessed by personal mobile devices, the Star Trek "library computer" AI was never "small and physical, easy for a single person to entirely own and unable to remember more details than a human", it was an aspect of a large server (or networked cluster, the actual architecture is somewhat vague) that was part of a capital ship or base, had access in the server/cluster to a library of very nearly all generally available knowledge and extensive personal information about both its users and about people with little direct connection (and could reach out across a galactic network to access additional remote information sources to handle requests).

"Unfamiliar algorithms run on machines far away" is much more like the source of the "Star Trek dream" than "small and physical, easy for a single person to entirely own and unable to remember more details than a human".

I think Star Trek was very prescient here as well: Security is terrible in the Star Trek universe. Nearly every antagonist is able to break into the computer within about a minute.

True! I suppose I was thinking of earlier space-age fantasy more than TNG. It was rather odd how easily people could look up details on crewmembers aboard the Enterprise, but I suppose the situation's different on an organized ship.

> It was rather odd how easily people could look up details on crewmembers aboard the Enterprise, but I suppose the situation's different on an organized ship.

Not just crewmembers: TNG showed some of the broader implications (both useful and creepy) of the convenience-oriented panopticon, e.g., when Geordi used the Enterprise library computer to construct a simulation from data (including personality profiles) of Dr. Leah Brahms, who later meets him (and encounters the simulation.) (S306 "Booby Trap", S416 "Galaxy's Child")

That was a substantial plot hole where she had no idea someone was simulating her.

I suspect a huge part of the panopticon culture would be / is being informed that you're being peeped at. 99% of the time someone asking "Computer locate commander Riker" involved commander Riker knowing all about who's looking for him and why and having a substantial conversation with the requester.

I don't recall any plot along the lines of Deanna getting jealous and spamming the computer all night asking where Riker is and he better not be in that cute ensign's bedroom... Because it seems logical the computer would inform him each time and he would eventually tire of the interruption and nature would take its course, WRT his relationship with Deanna.

A better analogy would be technically I could walk up to the company president's office and stalk him, but culturally that is so not going to fly and I would have a lot of explaining to do. Merely using the computer instead of walking there in flesh isn't a major cultural shift.

However, every single "holodeck creeper" plot line involved the simulated attractive real world woman not knowing she's being simulated until the plot reached maximum spaghetti spilling cringe, which is one of the few Trek panopticon situations where people being spied on did NOT know they were being spied on, which seems very un-trek, although it made for some entertaining stories.

An alternative interpretation is I believe over the course of the series every attractive woman on the ship was simulated on the holodeck at least once by at least one lonely guy, and its possible that culturally they just got used to it, although I find that unlikely. People do get conditioned to become used to the weirdest things, so its not out of the realm of possibility. Possibly a culture of what happens in vegas stays in vegas develops and its just the sexism of the TV show that they never showed the women turning the tables on their fellow male crewmen on the holodeck.

Isn't it the case that Apple's attempts at similar AI assistance store much of the data on the user's iPhone?

The fundamental issue comes down to one of trust and the real question is, do we trust Google to do the right ethical and moral things with our data that they are collecting en masse?

I think some people would be okay with that if:

1. That information were secret (you and service/device/implant) but not a whole company and its third party interests.

2. It wasn't making money for a third party after the initial purchase price for the device, service, etc.

So Siri.

Siri recordings are uploaded to the cloud for transcription, and reviewed by third parties to improve transcription and the AI. It also can arbitrarily decide to query search engines to answer questions.

So, better, but nowhere near "on the device only".

> reviewed by third parties to improve transcription and the AI


Siri commands stored by Apple: https://www.wired.com/2013/04/siri-two-years/

First party report of reviewing recordings: https://www.reddit.com/r/technology/comments/2wzmmr/everythi...

News article with more investigation and citings: http://sanfrancisco.cbslocal.com/2015/03/12/strangers-apple-...

Siri Privacy Policy excerpts:

When you use Siri the things you say will be recorded and sent to Apple to process your requests. Your device will also send Apple other information, such as your name and nickname; the names, nicknames, and relationships (e.g., “my dad”) found in your contacts, song names in your collection, the names of your photo albums, and the names of apps installed on your device (collectively, your “User Data”)


By using Siri, you agree and consent to Apple’s and its subsidiaries’ and agents’ transmission, collection, maintenance, processing, and use of this information, including your voice input and User Data

Until it started happening I always assumed it'd be powered by a central computer I had in a clean area of the attic or something. Not some DC somewhere.

"Knowing" in some sense everything about us is not the same as owning that information or trading in it. There are many other possible approaches to applying ML to our personal needs and data, so it is worth being careful about not conflating issues with a particular implementation and issues with the area as a whole.

> in order for it to do everything for us, it has to know everything about us, too

But does it? Does it have to know your birthday? (leave alone the fact that bdays are somehow part of a superkey for your identity).

Why should it know my residence, my spouse, or my CC# (with Apple's TouchID maybe it won't need to)?

Google's concept of AI is too creepy for me. It can be useful without being creepy. They're not even trying to make it less creepy.

Overlay on this the subtext that NSA and other tla's are monitoring all this (leave alone other countries). While I may trust Google, I don't trust them to not be forced to collude with the government.

Not to sound glib, but the idea of persistent data acquisition and aggregation has been pretty well known to be in the path for anyone seriously researching AGI or other human level AI systems.

How could it be otherwise?

I have to admit this is true -- I used to dream of a Star Trek like computer where you can just speak to it but I never imagined that such as system would be strife with privacy and security issues.

You could have realized that from watching Star Trek, as the computer in the Enterprise can always tell the captain where every crew member is, whether there are strangers on board, etc.

Yep. I see us somewhat conflicted about this. We would love the (imagined) benefits, but we have no idea yet how to avoid the dangers.

Bingo. The movie Her was awesome. But rewatch it. In every scene the AI does something cool, think what permissions it would need and what data it would have to have access to about you to accomplish the task. It gets scary pretty quickly.

Daydream is way better.

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact