Hacker News new | comments | show | ask | jobs | submit login
Amazon device recorded private conversation, sent it out to random contact (kiro7.com)
1314 points by spking 53 days ago | hide | past | web | favorite | 710 comments



I am oversimplifying here but:

* install device that is designed to listen to speech in the house

* the device is connected to internet

* the device is capable of contacting other internet peers/services/hosts

* the device knows a bit about its owner's internet presence such as contacts

* the device is equipped with simple conversational user interface based on fuzzy human speech-command detection

These basically bound the failure modes of the complete system. I am not surprised that a glitch like that happened. I can certainly attest to how shocking it might be to discover first hand that it really did happen, but given the context I can't say it's hard to not foresee something like this.


I remember the first voice command stuff on google. Buddy of mine was trying it out and I couldn't help myself but shout:

"NEW TEXT MESSAGE TO JENNY! FUCK YOU! SEND!"

Fortunately the phone was too slow to take it all (even if it had Jenny would have found it amusing, she was a cube or two away). But yeah first thing I thought of was that if it's listening how easy is it for it to mistake intent or someone else or etc.

Humans talk funny, computers don't get it, they're gonna mistake commands all the time.

As it is I hate how I can't talk ABOUT "ok google" around my phone... because it always goes off.


Back in college, long ago... the Mac 660 AV was state of the art for personal computing. The computer lab I worked in had just updated all the old SEs to these over the summer - I was part of the group that set up and plugged in a few score computers. With the microphones.

Our manager was a bit concerned about if the microphones would get stolen as he looked out over the rows of computers with the microphones on top of each monitor.

I yelled "Computer, Shut down. Yes."

A good chunk of the computers had indeed shut down.

We decided to go back and remove all of the microphones from the computers. Theft was not as a large of a concern as a bored college student with a prankster attitude during finals and the riot that would have ensued.


Our computer lab had a bunch of SGI Indy's, and it was first machines the CS department had with cams on them. It took less than a semester from they were installed until every student had learned that if the camera cover was not in place it was practically an invitation for someone to log in remotely and prank them while using the camera to observe from a safe distance. I like to think that it infused a healthy dose of concern for privacy and security in the student population..


FIRMLY within the spirit of good prank hacks like what has just been commented: a small number of USB webcams' LEDs are user-controllable, such as the random old Logitech C300s I have floating around here. You want webcam-tools from SF.net (other versions will spout obscure errors), then you can do

  $ uvcdynctrl -s 'LED1 Mode' <0..3>
0 is off, 1 is on, 2 makes it flash (and you can even set the flash rate), and 3 is auto.

0, 1 and 2 apply regardless of whether an application has an active video stream coming out of the camera.

Shh. (Hi NSA)


> As it is I hate how I can't talk ABOUT "ok google" around my phone... because it always goes off.

On several tech podcasts I listen to (http://atp.fm, https://daringfireball.net/thetalkshow/) they use fillers like "Yo, dingus!" to talk about alexa/hey siri/ok google


"My lady in the tube did this weird thing today" is something I've heard


It is interesting. But, how long before Alexa knows that is how people refer to her alternatively?


It seems to me that once she knows _that_, it shouldn't be too hard for her to understand when she's not being referred to.


But will she like it, and what will she do about it?


Best not mess with someone who knows where you live and has your credit card. If she's in a good mood, you might be hit with just a pizza prank. Hope she doesn't realize she can swat you without moving a finger.


Mine will sometimes / often go off when I say "ok, cool". It's simultaneously amusing and annoying.


"Cocaine poodle"


I was really hoping you were joking about that, but I can indeed get Google Assistant to pop up reliably by saying "cocaine poodle".


Well that's just great. How am I supposed to talk about my cocaine poodle now


I mean, that's an ambiguous term to begin with, and you'd confuse even humans with it.

Like, are you referring to a sculpture made of cocaine in the image of a poodle, or are you talking about a poodle train in the ways of a St. Bernard rescue dog, capable of relaying quantities of said substance, or perhaps something else entirely?


I was referring to the poodle I snort my cocaine off the back of. You guys are crazy.


I like how you went with the at Bernard analogy rather than day a traditional police drug dog.


[Google plays David Bowie, Klaus Nomi, and Joey Arias performing TVC15 on SNL]

https://youtu.be/ogUJ7JwkLNs?t=3m7s

https://en.wikipedia.org/wiki/TVC_15

>The track was inspired by an episode in which Iggy Pop, during a drug-fueled period at Bowie’s LA home, hallucinated and believed the television set was swallowing his girlfriend. Bowie developed a story of a holographic television, TVC 15. In the song, the narrator's girlfriend crawls into the television and afterwards, the narrator desires to crawl in himself to find her.


Haha nice, I've used "Okay poodle" a lot before.


I like this a lot! It seems to work reliably for me, and is a lot easier to say than "ok google". Thanks :)


"OK Poop Hole"


Another convention seems to be 'Alexo' when you want to talk about Alexa but not to Alexa.


We just call her "you know who" (a.k.a. the thing who's name we are too afraid to mention)


Why not just call her Lord Voldemort then?


I got a touchscreen for my desktop PC once (it was the Windows 8.1 era, after all). I immediately discovered that it provided me no significant practical benefit, but it was now infinitely easier for someone walking by my desk to poke my screen and mess up what I was doing.


I had the inverse experience. My wife's laptop had a touch screen and I thought "What a dumb idea, what use is that?"

Then she'd ask me for help and rather than have to wait for her to find what button I was talking about I'd just tap the screen a few times and we were off and running again!


I bought the original Surface Pro the week it came out. I actually really like the interface and thought that it was great. I didn't really understand why so many people hated Windows 8 until I tried to use in on a desktop. Even with a touchscreen it's pretty annoying.

I'd been using Linux as my primary OS for years at that point, so I think I'm pretty critical of Microsoft in general, however they did a good job building a touchscreen interface.

The state of touchscreen support in Linux was pretty bad the last time I checked though.


I'd definitely agree with regards to the Surface, and a lot of Windows 8/10's UI makes a lot more sense once you see it on a phone as well. I'd say Microsoft did a poor job retaining the existing experience for desktop users while focusing so heavily on new form factors and experiences.


> The state of touchscreen support in Linux was pretty bad the last time I checked though.

Anecdotal — but have had no trouble for the last two years with my dell xps. Other issues with said laptop (keyboard), but touchscreen support in linux isn't one of them.


Ah, I saw that the Dell XPS came with Linux and had a touch screen, and I was kind of wondering about that. I guess I'll have to try Linux with a touchscreen again.

My Surface Pro 1 is the only computer I have with a touchscreen right now, and it has enough other driver issues (the wifi and the Wacom drivers suck) that I probably won't be trying again on that.


Also, that'd be long-term usable only if the screen is ~flat on the desk. Otherwise it becomes exercise.

And in effect, that's what I've used for many years. I have a relatively large touchpad. And when I need resolution, a graphics tablet.

The Surface Studio is cool, but I'd want something that resisted coffee and food splatters ;) As the original Surface did.[0] It could also scan text :) Which is something that I, when I was first learning about PCs, naively expected.

0) https://www.windowscentral.com/microsoft-surface-pixelsense-...


Ok Google has been a massive disappointment every time I've tried to use it. I tried "next song" while in Google Maps and it marked Exxon along the route. Then it played a song in a streaming app by "Max," wasting my data and making me close the streaming app I didn't even know I had and restart the music I wanted. "Next" turned out to work a bit better but it took multiple tries.


I remember switching to Android way back whenever the iPhone 3gs was "second-best" iPhone (was the 4 the next?), from the 3gs.

The 3gs had perfectly functional "play my xyz playlist", "next song," "play songs by artist" functionality, and to this day I don't understand why it took so long for Google to implement it, and why the implementation is so shitty.


The issue is google doesn't care about they're users. As long as it's good enough to keep the data-gravy-train going.


Well, it’s probably not that bad. I think it’s more like google makes everything work ok, apple tries to really nail core functionality and let the rest slide.

Music interaction is a great example of the trade off.


Not sure why you're being downvoted. Google has different business model than Apple and expecting it to care about user experience in the same way simply doesn't make sense. And yes, they mostly care about data, because that's what they're good at - collecting data, analyzing it and turning it into profit.


Really? I find it very accurate and helpful, but that could be because i come from siri hell. Which couldnt ever seem to get it right, especially using gps.

though i can easily see with background noise, next song sounding like exxon. I think due to frustrations with siri, i talk to google in very clearly enunciated words and havent had any interesting failures yet.


We did something similar to a coworker when Apple Watch came out. He tried to dictate a text response to someone that said “thanks for doing that.” Someone else yelled “DICK!” afterward. Apple Watch is notoriously slow on processing text to speech, so he didn’t think the watch picked it up and hit send. Then the text was added.

He had to call the guy he tried to text to in order to apologize.


I was so hoping the guy's name would turn out to be Richard.


Is that acfeynman reference?


It reminds me an old trolling technique - using "XBox Shut Off" as your nickname in video games ;-)

https://www.youtube.com/watch?v=hGZeU4s28kk


"What's up Youtube?! It's your boy hey google rm -rf"


"What's up youtube? It's your boy Alexa order me an 80 gallon barrel of lube every week!"


My co-workers have Alexa. I love shouting out "Alexa play Justin Bieber!" or "Alexa! Add ice-cream to the grocery list!"

It worked several times and I found it really fucking hilarious. They did, too and now they guard their Alexas and use headphones.


My friend installed IBM OS/2 WARP beta when it came out, it too was voice controlled. We would shout into his room, "select all, delete" or "format c" or other childish fun commands.


Humans are terrible. When the computers take over.... I won't be happy about it, but I'll kinda understand.


Just shout "self-destruct" all the time and we'll be fine.


Alexa has 4-5 joke responses to self destruct. E.g. "I'll start the countdown but only on the understanding you'll dramatically cancel it at the last second", and "3-2-1 Boom. Hmm that didn't quite work did it?" or something like that, and a few others.


Like the countdown at the end of Monsters vs. Aliens:

https://www.youtube.com/watch?v=FJ5DA22tBOY#t=4m27s

“8… 7… 6… 5… 4… 3… 2… 1…

Hmm, nothing happened, maybe my count w… [BOOM]”


Selling a product that had a voice command UI we found that people will go out of their way to mess with it. Sometimes for fun, sometimes to unload their frustration. There is something strange about a voice UI and I think it is the expectation that the intelligence should be about the same level as human intelligence. If it is not there, and it is making the user look silly by making them repeat something obvious multiple times, they will quickly start hating the product as a whole. It's almost better not have a voice UI unless it is pretty darn good.


Yup, Mac OS 7.5 had speech commands, my colleague named his machine "Oi!" and I would routinely shout "Oi! Shut down!" as I left the office. Never got old.


Of course, and as was widely known (and mentioned when this subject came up) at the time, FORMAT C: would not have actually worked (on systems where C: was the boot volume) because OS/2 locked volumes that were in active use as filesystems, preventing them from being reformatted.


Hahah so my favorite thing to do when people use Siri, Google voice, Alexa etc is shout

"WHERE TO BUY COCAINE, SAN FRANCISCO"

or

"HOW TO JOIN ISIS"

or

"PLUTONIUM FOR SALE NEAR ME"

The commands are usually recognized but turn up nonsensical results.


Why?


Because it supposedly puts people on some kind of watch list? Tho pranking with stuff like that ain't exactly the coolest thing to do: "Haha I framed you for terrorism, gotcha!"


It's not illegal to search Google for how to join ISIS, last I checked.

Also, Google saves all voice commands. If the NSA wants to check they'd just hear all of us doofuses giggling and trying to one up each other.


Yeah it’s incredibly irresponsible and not really funny at all.


I do not think this is funny, I think it's rather immature.

But if things like this _actually_ trigger any sort of response, then the effed up thing isn't the (childish) prank, it's the system actually acting on nonsense.

If I call the police and claim that you're armed and hold hostages, then that's irresponsible. If I shout bullshit instead and these novelty devices turn that into a web search, there's no way in hell that should have any consequences whatsoever.


Why is it irresponsible?


If that person does make it on to a watchlist it's wasting resources and causing them 'harm' (eg always stopped at border checks).


We always get stopped at borders anyway, what do you mean?


In my friend group, we're all on watch lists anyway. I guess I should have clarified, I'm not doing this to strangers, we all do this to each other and find it hilarious.


Yeah, but how old are you and your friends? I think eventually most people grow out of this sort of immature 'pranking' and just find it a bit boring...


The need for keyword escaping is entering meatspace.


For years Ive been expecting to hear about some shock-jock in a big city saying "Hey Siri, find child porn" over the air during morning rush hour.


Burger King intentionally set off Google in a commerical. People were not amused.

https://www.theverge.com/2017/4/12/15259400/burger-king-goog...



That's good but I think I prefer http://dilbert.com/strip/1994-04-24


This is even better because of the date, we had voice operated computers 24 years ago that ran on machines that wouldn't even be called a potato these days and they didn't need to send the voice off for cloud processing.

And voice control didn't fail way back then because it wasn't good, it failed because it was a terrible input mechanism and that is why it will fail again.


It's a terrible input mechanism for general usage. It's a fantastic input mechanism when you don't have another one handy, open and ready to use. E.g. I use Alexa to trigger my Harmony hub to turn on the TV and switch to the right input with one command, instead of figuring where the hell someone left the remote or digging my phone out and open and app. It's faster, and easier. If a phone call comes in, I don't need to find the remote or the app to mute or pause what's on TV, I just ask for it and it happens. If my hands are full I can still turn the light that I forgot about off on my way to bed.

The "failure" the first time around was that people wanted it to be the main interface, instead of accepting that where it works is as an ubiquitous auxiliary interface that's available everywhere in the house, rather than trying to compete with a keyboard when you're right in front of said keyboard.

Voice competes by working at a distance, even when your hands are not free.

It will never replace a keyboard, as there are so many instances where speaking commands is impractical, but it doesn't need to replace a keyboard to be useful.


[flagged]


> I’m a millennial who grew up with Dilbert, and it just feels so wrong now.

There's nothing wrong in enjoying the writings of people whose political ideas you disagree with. I'd rather say that if you cannot do that, there's something wickedly wrong in you.


Different types of comedy are impacted very differently if you discover the author believes what they're saying.

"A Modest Proposal" [1] would be seen in a very different light if we learned Jonathan Swift was serious.

[1] https://en.wikipedia.org/wiki/A_Modest_Proposal


Hell, that’d exclude a whole load of decent sci-fi also.


Adams has been a loon for a long time, his Dilbert Future book talks about his personal philosophy which is straight up "Name it and claim it" theology except without the Jesus. Adams basically attributes all his good fortune to his own talent and hard work.

And then of course that means everybody who is struggling, well those people just didn't want it enough, they aren't putting in the hard work like Adams. Frankly it's their fault thinks Adams.

Dilbert Future is twenty years old.


To be fair that line of think is a problem here on HN, among millenials, and among older generations as well. Americans in particular, but our species as a whole need to learn to separate the ideas of success and merit.


I can't find anything about him being a moon landing conspiracist. Link?


You don't like Trump or people who support him. What does this have to do with this article?

Not everything in the world need to be connected and associated with US politics and trump vs anti-trump politics. If we brought in the political views of every creator, inventor and artist than we would not be able to discuss anything on HN other than politics since every creator, inventor, and artist has a political view and people who agree and disagree with that political view.


Can you show me anything other then the parts where Scott Adams simply debunked Trump's tactics for conversational interference as related to business negotiation?


Just browse his Twitter, he regularly retweets Trump's ad hominems.


Retweets don't necessarily mean endorsement.

Adams frequently pitches his book "Win Bigly"...marketing himself as one of the few people to predict Trump's rise, due to his rhetoric. Showing examples would just be marketing for Adams.


https://www.smbc-comics.com/comic/dear-god-3

And really, why not make it a 55 gallon drum of personal lubricant instead?


I always think if I ever present somewhere again, I'm tempted to start off by asking people to put their phones into silent mode... wait a moment, then "OK Google, Siri show me dick picks"


Hey Siri is keyed to a specific persons voice. Is the google equivalent not?


It is. OK Google won't unlock a locked phone unless the voice matches, and you specifically enable the feature that allows it to unlock the screen. Once the screen is unlocked however, only certain functions that access your account will need to match your voice (if you spent the time to get Google to recognize your voice.) Performing web or photo searches wouldn't require you to match the voice if the phone was otherwise unlocked.

Asking someone to silence their phone might just be an attempt to get them to unlock their phone before issuing the offending command.


It’s silly that radios and podcasts and random people have the ability to trigger Alexa just by saying the name. “Hey Siri” gets around the issue by only listening for your voice, not just anyone. It’s shocking that anyone would build a voice assistant with zero authentication or authorization built in, so much so that an advertisement on the radio or TV can trigger an action on your device like on an Echo.


s/silly/fucking ridiculous/

but sure, I agree. That's why I questioned it.


> "I felt invaded," she said. "A total privacy invasion. Immediately I said, 'I'm never plugging that device in again, because I can't trust it.'"

Let this lady's reaction be a lesson to all of those who say "people don't care about privacy." She wired her home with Alexa devices, and yet when she found out how her privacy could practically be invaded through such devices, she seems like couldn't get rid of them fast enough.

People care deeply about their privacy. They just don't understand the true implications of even a device that's supposed to "listen to you" and the possibilities of what someone else could do to their internet-connected devices.

This is why governments must intervene with laws such as GDPR or even much stronger ones in the future to protect people's privacy, because the companies themselves have no incentive to "self-regulate" other than a negligible amount.

The only problem is some governments will not intervene unless enough people go through similar situations such as this lady. In the US even that may not help much because the vast majority of politicians care most about continuing to receive (however modest) donations (also called bribe money elsewhere) from the industry.


People care about privacy to the extent that they don't want people in their social circle knowing stuff. They don't care so much about a faceless organization taking it.


> People care deeply about their privacy.

No, they don't. Do you really think if, before she made this purchase, someone explained to her all the things that could theoratically go wrong with this device, she would have chosen not to buy it?

She only cares now and acts like she doesn't share quite a bit of the blame because it actually affected her. I think it is highly likely she'll just switch to an Apple or Google equivalent instead of noticing the larger problem with these devices.


> No, they don't. Do you really think if, before she made this purchase, someone explained to her all the things that could theoratically go wrong with this device, she would have chosen not buy it?

I think this would only indicate that she might care, but thought theoretical worst case scenarios are not sufficient evidence for an actual, non-negligible risk to her privacy. If they had told her that exactly this would definitely happen, I think she would not have bought the device.

I think one problem of explaining privacy issues to non-techies is justifying why theoretical problems often pose a real threat without sounding (to them) like a conspiracy theory nutcase.


This indicates that people may care deeply about their privacy when they've experienced a breach thereof. It is consistent with people not caring about their privacy because they can't imagine such a breach or project how they would feel if it did happen to them. It does not support their not caring about it.

The loss of privacy spreading throughout civilization is recent and rapid. It's not surprising that somebody might think about it differently after they've experienced it. We haven't had time to come to terms with what out technology can do.


You're ignoring risk. This is one failure out of millions(?) of customers. It's like someone being seriously injured in a bicycle crash and deciding not to cycle to work anymore. Or having their house burgled and moving out of the neighborhood. People get hurt when rare bad things happen to them and it scares them off.


It turns out that people react irrationally when rare bad things happen to them; that is, they over-react, generally...


I’m not sure sometimes if people are oblivious to consequences or they are aware of them and brush them aside until said consequences occur.


Completely agree with you. I've seen this over and over and over with people I know. They always want something badly enough that they'll find any excuse to dismiss any downsides to it.

"Amazon/Google/Apple/Facebook/X have very smart people who will not allow this to happen."

"I have nothing to hide."

"People who want to listen to our boring conversations are losers. Let them waste their time doing so."

And the list goes on. I'm not making any of this up.

And then when it (occasionally) happens to them, there is a whole lot of dramatic outrage and someone else must be blamed. (Although in this case, I agree someone else is to blame - but I've seen this play out when people explicitly agree in the contract to consequences, and are warned quite clearly by a human about the implications of the contract).


That's not unreasonable. Lots of good things come with risk and when they go wrong we blame someone. Your house burns down and you blame the arsonist, not yourself for having a house made from fuel! Car crashes kill people and we blame the drunk driver not the victim innocently using the road which is known to be one of the most dangerous places to be.


"I have nothing to hide."

Response: "Cool. What's your e-mail password?"


Because emails only have info about the recipient; and email isn't used for access to anything of worth, like bank accounts. /s


No, response: "Cool, why don't you send all the e-mails you've received this year to the police, then?"


> No, they don't. Do you really think if, before she made this purchase, someone explained to her all the things that could theoratically go wrong with this device, she would have chosen not to buy it?

Yes. Do you think if everyone knew what was going to happen with the Facebook scandal before it happened they would have signed up for Facebook? That's why it's called a "scandal."


>Do you think if everyone knew what was going to happen with the Facebook scandal before it happened they would have signed up for Facebook? That's why it's called a "scandal."

Yes.

I also noticed how they haven't left once the scandal came to light.


>I also noticed how they haven't left once the scandal came to light.

You and I must read different news sources.

From what I've read, Facebook is down over a billion visits in the last quarter.


Half of Facebook's users visited one less time in a 3 month period? I think they'll survive.

I've visited less in the last month, but because they've changed the algo on my feed and it's not as interesting ... though that could be a network effect, but I'm getting quite different stuff.


> This is why governments must intervene with laws such as GDPR or even much stronger ones in the future to protect people's privacy, because the companies themselves have no incentive to "self-regulate" other than a negligible amount.

Market failure does not imply government solution. You're going to need a much stronger argument than "companies aren't doing a good enough job."

Amazon invaded her privacy, she realized it, and now she doesn't use the product. If you want others to do the same, start raising awareness yourself instead of enacting nanny state laws with other people's money.


I fail to see how market failures are not a reasonable point to look at whether governmental solutions are applicable.

Case and point - 2008 Market Crash, everyone agrees the dishonest practices that led to the crash should have been made absolutely illegal (in the cases where they already weren't)

In this case, we have MANY significant and potentially extraordinarily damaging failures in the markets to effectively protect people's privacy occurring fairly regularly. GDPR is not the ideal solution my any metric, but that's a failure of government to create effective legislation - not a lack of need for effective rules.


Indeed, "if you apply General Curtis Le May to a situation and you get havoc, well, that s what you called General LeMay in for" (to use an Eben Moglen quote out of the context in which he said it); if you host a device in your home running proprietary software you ought not be surprised that it is spying on you and the proprietors determine what to do with that data, not you.

We must not forget another critical component of all this: these always-on listening devices run on proprietary (nonfree, user-subjugating) software. Therefore the user has no permission to: inspect what it does, modify it to do only what the user wants (or, since most computer users aren't programmers, get someone technical they trust to perform such modifications for them), run the modified software in their device, and distribute copies of the improved software to help their community.

If these devices ran on free software (software that respected the user's freedom to run, inspect, modify, and share) the more technical among us could help them. But as it is even the most technically-minded willing person cannot legally do this work to help them.

As Eben Moglen reminded us after the Snowden revelations came out: It's critical that we don't fall into the trap of saying something akin to 'those kids take too many darn pictures' like concluding that we just can't have these devices or their services at all. We can have all of their alleged conveniences but only if we have free software implementations.


The idea that if these devices ran free software they would be more humane than the proprietary software they currently run is ridiculous.

What are you giving your mom? A Mac running OSX or a PC running Linux?


> What are you giving your mom? A Mac running OSX or a PC running Linux?

I gave my 60 year old mother my old PC running Debian with dwm as its WM.

She likes it.

> The idea that if these devices ran free software they would be more humane than the proprietary software they currently run is ridiculous.

Is it though? I am pretty sure that Amazon already gathers free data from its users, such a thing would not exist in a free as in freedom device.


that... is an ironic example.

To be fair, the community came down on Ubuntu like a ton of bricks for the whole... Amazon search thing.


To be fair, don't the Alexa service and the Echo devices run on Linux?


I dont know if it does or not but even if it did, that wouldnt make it free software. You cant compile your own alexa, or modify it to remove code you dont want, or submit improvements to it. Android is open source (excluding drivers), but google play isn't, and neither are the apps on it.


So this is what poe’s law looks like.


If you replace "fuzzy human speech-command detection" with "buttons which can accidentally be pressed" - how is this different from butt-dialing?


Most phone now have some kind of lock screen, which makes it pretty difficult to get to the butt dialing stage. Will speech command recognition get to that stage?

The main reason I won't have any of those products in my house is because of that. I'd much rather have a confirmation of some kind before the system takes action.

"...random talking..."

Assistant "I am recording"

Me: "Stop recording"


You'd be amazed at my dad's ability to butt dial people with an iPhone. It's possible. Like multiple times per week possible.


I always press the lock button before I put the phone back in my pocket, but I've seen people put the phone back in their pocket or purse unlocked. If you have the phone app already running, butt dialing is very likely to occur.


Lock doesn't have to be swipe-to-open. I'd be amazed if he could buttprint-unlock ;)


http://anuscan.com/

It might be a bit awkward at first to do that in front of strangers, but once it catches on, that will wear off. ;-)


My friend does this fairly often. He'll finish a call, and turn off his screen, but he has a tendency to to hit the fingerprint scanner as he's putting it into his pocket due to the design of the case and the way he holds it.


Got an LG G5 and quickly learned that they added a "feature" where double tapping each volume button opened a preconfigured application (Camera or QuickMemo+). Even when locked. Phone had to be completely powered off for it not to trigger. Went in settings and disabled that on day two when I saw how many pictures I had of the inside of my pocket.


Some Androids (like mine) won't lock for a few seconds after turning off the screen. It's a nice feature when you press to turn off the screen, and quickly realize you need to do something else. Just turn on the screen and it's still unlocked.

Entering the pattern after it's locked is pretty much impossible though.


I can configure how long that delay should be. IIRC it's int he display/lockscreen menu. I don't like it tho, as I much prefer the device to be locked when I tell it to, so I won't have to care anymore. The fingerprint scanner on the back makes the unlocking trivial though.


But then that means if you put it in your pocket it might suddenly do something because it's not locked yet.


It might, but it hasn't in the 18 months I've been using it. I don't know what the timer is supposed to be, but it seems like something less than 2 seconds. I'm pretty sure I couldn't get my phone in my pocket and press the power button that quickly if I wanted to.

Also, the case adds to the pressure needed to depress the button anyway.


I dropped my phone in the gym and it somehow typed garbage all over the Notes app where I keep my grocery list and record of what I did in the gym last...

This sort of thing makes me nostalgic for the old days of explicit "Save" commands in every application.


I'll settle for a reasonable "undo" feature.


Undo seems to be a concept in Android apps sometimes, but I'm not aware of a universal way to access it, like Ctrl-Z.

And once you have Undo, it's nice if you always have Redo, in case you Undo once too many.


Yeah, a digital assistant could (should) verbalize "uh huh, yes, i'm listening, interesting, hmm" like a real, nosy human. Then you get a chance to say, "go away, alexa".


The Echo devices will light up when listening, and can also given audible warning.

But it'd certainly be good with more feedback.


I will have one of these in my house when it doesn't send recordings of me to the Internet in the first place. It can do the voice recognition locally. I'm actually really interested in having one of these in my home, I'm also actually really not interested in it being cloud based at all.


some kind of acoustic biometrics would be helpful here (ie respond only to account holder, or disable some actions for others) along with better heuristic recognition of directives

thats not foolproof but much better than what we have now, and i think we are pretty close


Agreed, but I think vocal biometrics is a significantly more difficult problem to solve when even the best speech recognition still has issues like this.

Phonetics is hard. Especially with ambient noise, echo and such. I had a conversation with one of the speech engineers when I worked at a speech recognition company and the level of detailed problems to solve was impressive. Totally made sense after talking about it, but things I would not have thought about before.

I'd imagine the next thing to come in this area that would really make an improvement is an "on person" microphone. Maybe it's a pen in your pocket, or some kind of vibration detection (that could pick up the wearers voice), that would then allow some improvemnts in the domain of "who is talking" and how well the voice is processed.


The vibration detection is commercially available, in the 3-digit $ range, but it's unfortunately hideous as it needs some mild force against the throat and thus requires a very high neckline or a scarf. The potential alternative, Subvocal recognition, suffers from the need to have nerve/muscle-sensing electrodes on/around the throat, which have poor long-term bio compatibility (think >10h/day, on average) and the alternative, implants, suffer from growing out if a wire sticks out, at least as far as I know. One might be able to implant special electrodes the skin would not reject, but I don't know of any suitable material. The obvious benefit is that you use your normal speaking logic, just stop at the point where you actually move your throat (afaik), and don't modulate with your mouth or lungs. It's silent for anyone around, so one could, theoretically, call someone while sitting inside a meeting, and hear both speakers, while being able to selectively talk to the other end of the phone line.

Both of these technologies work in nightclubs and fighter jets. I assume they are very high SNR, as far as ambient noise is concerned. The subvocal one might just be a phonetic/intonation input, which then requires voice synthesis if actual voice is the goal.


And once you've managed to achieve perfect biometrics your next task is to prevent replay attacks.

An alternative, "wireless" approach to near-perfect security would be to invent-plement some kind of vocal, human-executable GPG/TLS.


You don't really need perfect biometrics for it to be useful, or even biometrics at all. A "my human just spoke in this room, and here's a signature of what the fleshbag said" broadcast from a microphone to let devices within a very short range to get confirmation of which person spoke if they picked up a command, coupled with pairing your speaker to your devices to give the notification from your mic more authority would already help substantially with most of the issues people have. It wouldn't stop a determined adversary, but for most people it's not a determined adversary that is the problem.

It doesn't even need to get good audio - just enough to give a bit of an indication that what the device picked up was me talking and not random noise and ideally some way to somewhat correlate it to the audio the device picked up to give it an indication it was me it heard. It'd also give you the option of setting the devices to require confirmation for certain types of orders if they were not confirmed by an authorised device, or if they were not confirmed by a device (so you could let people present give instructions but not some random joker on voice chat in your online game for example).

If we could get support for that into e.g. a watch, it'd be very much useful.


I agree, it would probably not even be close to foolproof.

I believe any implementation of security through acoustic biometrics would be vulnerable to replay attacks.

Systems to reproduce acoustics with high fidelity are commonplace - You might be using the output component of such a system right now if you're listening to music.

You could make the Assistant remember the exact fingerprints of all previous activation phrases and only trust you if it was original. This could be circumvented if you spoke the activation phrase at any point where your assistant could not hear you, for example to another Assistant of the same brand.


Wouldn't modifying the sample slightly, like lowering pitch by a few cents or stretching parts, also make it seem original?

Audio is definitely too easy to spoof for it to be a security method IMO.


i meant biometrics merely as a UX improvement, ie, to help prevent the device from responding to the wrong thing "accidentally"

it may have a place in security as well but i can only see it as part of a much more holistic model


"respond only to account holder, or disable some actions for others"

This is exactly how the Google Home works right now.


On the Echo at least, there's an option to have the device make a thump-type sound when it's activated. There's also a light ring that activates out of the box and shows where it believes the audio is coming from.


I tested mine right now, and I can't see any correlation between the light ring and my location relative to it.


could be picking up a bounced audio if it close to a wall or other surface.


Maybe, but if it can't handle that, it's kinda pointless to try to detect direction - all the places in my house where I'd be likely to want to place it are <0.5m from a wall or less; I'm certainly not going to place them in the middle of a room.


as long as you remember to lock the phone before butt pocketing it. I still butt dial today but its mainly right after i talked to someone.


I can't pick up my coworker's phone and manually dial a number, because years of learning from our mistakes have led us to implement lock screens and such.

I can, however, say "Hey Siri, call 867-5309." Or "Hey Siri, Facebook status: my anus is bleeding. SEND!"

Because we are apparently incapable of applying past lessons to new technologies.


It really shouldn't be hard to do a quick safe filter and ask "You sure you want to send that?", again, like a real human.

My point is they're really half-assed products right now that are behaving badly/immaturely.


Apparently it did have a confirm question.


> "Hey Siri, Facebook status: my anus is bleeding. SEND!"

Serves me right for having screen reading on near my phone.....

(jk, jk, but seriously though)


If they operate anything like the way I operate, they know there should be some kind of lock on there, they just haven't gotten around to building that part yet.

And if their management is anything like my management, they go ahead and release it anyway.


Hm. I can't "Hey Siri" someone else' phone, can I? It's certainly never worked for me.


To my knowledge, “Hey Siri” has never been tied to the owner’s voice. I was able to post to a colleague’s Facebook account a couple years ago, using nothing but voice commands. It was sitting on his desk, locked. I haven’t tried it since, and I keep it disabled on my own phone.


"Hey Siri" has in fact been tied to the owner's voice since the iPhone 6s (2015): https://www.macrumors.com/2015/09/11/apples-hey-siri-feature...


Progress! I stand corrected. Thank you.


Meanwhile, I have my Pixel lock on screen off (only because the fingerprint reader makes unlocking/turning on easy) with OK Google unlocking enabled.

The only problem here is that Google still doesn't recognise my voice half the time, perhaps because I'm never sure about how I should speak to an inanimate object.

Speaking of unlocking methods, face unlock appears to no longer be available as an option for me. I can't find it in settings anymore (smart unlock etc).


If the iPhone has a lock screen, Siri will not do any of those actions without it being unlocked first FYI.


That was definitely not the case when I tried it a couple years ago. If they’ve since locked it down, then I’m glad to hear that.


a) Your phone has a microphone that picks up information spoke more-or-less directly into it. An Echo is specifically equipped with a microphone array designed to extract audio from anywhere in the room.

b) There is a great deal less ambiguity around "pressing buttons" than there is around interpreting speech. While it is unlikely that your phone will incorrectly detect button presses, it's very common for voice-activated devices to a) incorrectly detect a wake word (either negative or positively); and b) misunderstand some particular word used in the command. Your phone is not going to think you pressed the "Clothes" button when you actually pressed "Close".

c) The entire functionality of the device is accessible from behind that big ambiguous interface. On a phone, there are many distinct steps and screens to step through when you want to do something (this complex interface, by the way, is a non-trivial part of why butt-dials are quite rare these days). On a "smart speaker", most things are just one misheard statement/command away from occurring.


It's interesting that the pattern of human communication where things are just one misheard statement/command away from occurring has been well modeled in these devices.

Does having the device programmed to make mistakes make it more comfortable to use, because we know humans are infallible too.


I don't think there is much of a difference. People took a while to understand butt-dialing, and not do it as much. It'll take a while for people to get used to voice assistants that trigger seemingly randomly.


when my phone is locked (which it always is in a pocket) it can't butt dial.

whats the analogous behaviour for alexa?

also remember these are shared devices in a home, not a personal device in your hand. could i shout into somebody's window, "hey alexa send my browsing history to bob@hotmail.com"!


The analogous behaviour would be you being aware of Alexa's flaws and not triggering it accidentally.

While you may have never pocket dialled, plenty of people have. I've received more than a few accidental calls in the past.

I equate this to some phone users placing their devices hanging over the edge of tables - as if they don't care about them hitting the floor. Should phone makers toughen their phones and should Amazon improve Alexa anyway? Sure.


They can still call the police while locked, which is something that happened to a friend with an old feature phone while were were in the process of illegally hiding/dumping an old car.


Mike, is that you?


The "lock" analogue on an Alexa device would be the mute button on the top of the unit.

https://www.safety.com/wp-content/uploads/2017/06/mute-butto...


AFAIK, pocket dials have mainly been reduced by improvements in phones, rather than changes in user behavior; if the camera just sees inside of a pocket, it's unlikely to be an intentional button press.


I mean way back in the day when people had candy-bar phones and had to remember to enable the keypad lock :D


I made a number of emergency calls from my pocket on my Nokia 1100 even with key lock enabled. It was infuriating. It happened once when I was having a heated argument with somebody and it took some work to convince the dispatcher not to send a law enforcement officer. (There was nothing physical happening and no threats of violence, but we were speaking very sharply to each other.) I do not miss that about my Nokia 1100 at all.


Simply: no one else is dialing via my butt.


Just as nobody but the primary user(s) triggered the failure in this case!


hold my beer


I haven't butt-dialled anyone since I got swipe to unlock on my iPhone. A 6 digit unlock code/fingerprint would make it seem even less likely to happen again.


I do not want to set up this feature on my Alexa for precisely this reason and I am finding Amazon to be really aggressive about forcing me to enable this feature and harvest my contact information. The Alexa phone app is almost unusable for me, as it constantly goes into the telecom setup screen whenever I try to do something else in the app. I am forced to exit the app and come back into it, whenever this happens, to escape that screen.


Wait! So you're saying that Alexa actually includes the capability to record voice, and send recordings to contacts? I suppose that some would love that. But, as this demonstrates, it's prone to fail spectacularly.

You'd think that there'd be protection. Maybe a restricted set of contacts who could get such messages. And a confirmation step, such as "Do you want to send this recording to 'my mother'?" Or "... to '911'?"

Edit: Damn, didn't see Clubber's comment.


What do you actually use Alexa for?


I use Alexa for listening to music. I rarely go into the app, but occasionally attempt to use it to set up a new music service, control the volume or track (if I am unwilling to shout down the music to tell it by voice). When I open the app, I get immediately thrown into the setup screen for the voip feature they're pushing.


Every time my wife says to me, "Are you serious?" Her iPhone chirps up with something like, "Yes, I am Siri."

Somehow it hasn't broken her (my wife) of the habit yet.


To be fair it hasn't taught you not to say such astonishing things either!


They key feature you missed,

* devices can operate without user's knowledge

These types of devices are easy to accidentally trigger; it must be vastly obvious when that happens.


Sounds like the equivalent of a confirm popup would be a solution. Are you sure you want to send this to so and so?


i think there’s a market for non cloud home automation. i will never buy alexa


Others agree. Silk Labs[1] (founded by former Mozilla CTO Andreas Gal) seems to be active in this space

[1] https://silklabs.com


That's almost the description of a smartphone.


you just described a smartphone


* the device is composed of proprietary blobs

Amazon might as well record everything that you say and send it to the NSA.


You are right, but your argument is beside the point. The point is, that those devices are not designed to work 100% accurate. So even if Amazon has no malicious intent, such privacy-invading events will happen!

They are just unlikely, not impossible.


I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce individual components that I can trust to piece together.


> I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce individual components that I can trust to piece together.

You don't need cameras for that, just motion sensors.

> That server makes decisions on if lights should be on...in that room.

You don't need a server for that, just a motion-sensing switch. They can be totally offline. My office has them to shut off lights automatically when a conference room becomes unoccupied.

https://www.homedepot.com/b/Electrical-Wiring-Devices-Light-...

It can get annoying if you're not moving much, but the key point is: no camera or server required.

> That server makes decisions on...if A/C should be running in that room.

Ecobee thermostats can already do that using motion/temperature sensors. They run fine without an internet connection.

https://www.ecobee.com/room-sensors/


But then how can I sell the analytics data for billions of dollars? /s


Might be cool to add thermal sensors to that, so the lights don't shut off because you're not moving enough.


I'd also like them to dim if I'm holding a glass of wine and/or move in a particular manner.

Edit: typo


Haha, and play some jazz tune maybe?


That's the main problem with motion activated office lights tbf, especially when you're doing software development for example - not enough motion to keep them on.


My problem with the motion activated office lights (at least the ones at the wework my company is located) is that you can't turn them off. The button appears to turn the lights off and disable the motion sensor for some (short) period of time, then movement turns the lights back on again. Very annoying.


My office has a piece of paper taped over the motion sensor on the roof for that reason.


Eh, it encourages me to get up and have a stretch periodically.


That doesn't work in a toilet stall.


May I suggest some fiber?


Every time! The coffee shop by my old house had motion sensing lights in the washroom, and every damned time I would be plunged into darkness half-way through my visit... Urgh.


Our company has this in the toilets, when it goes dark you just wave your hands


Ours does too, but there's no sensors inside the stall... You can wave all you want, there's no way to turn the lights back on without opening the stall door.

Thank goodness for phone flashlights!


could be fixed with sensors in all the seats, or on the floors where people with standing desks are


I wish more offices would install motion sensors facing keyboards from above, tune-focused for finger motion detection.


> I wish more offices would install motion sensors facing keyboards from above, tune-focused for finger motion detection.

A new dystopian developer-productivity metric.


Erm, could you not also just install a keylogger?


The NSA could just give you access to theirs.


That would still mean that the lights would go off if my boss was the only person in the room.


That's alright, he doesn't really need the light anyway.


If you're staying that still, you likely don't need room lights on. Why not just use an area/task light if you're performing a stationary task?


For people with glasses the reflections can be pretty bad. In addition I require bright cold-white neon light or I fall asleep.

By the way, German workplace safety regulations require a certain minimum of light at a workplace.


OSHA also requires a minimum light level in the US but I believe it only pertains to hazardous working environments (like on a production floor in a manufacturing facility for example). Either that or the enforcement in white collar environments is so weak that literally no one cares to follow the law which is surprisingly common with reagrd to many workplace regulations.


commodity motion sensors are thermal sensors[0]. Perhaps a FLIR[1]?

[0] https://en.wikipedia.org/wiki/Passive_infrared_sensor [1] https://en.wikipedia.org/wiki/Forward_looking_infrared


FLIR for turning off a light in a room seems like a ridiculous amount of overkill. I don't know numbers off the top of my head but I feel like at that point you're using more energy for FLIR than the light itself. Not to mention the cost of a single FLIR sensor is probably pretty high.


I think clever placement of traditional IR sensors is the answer here.


These cheap 8x8 sensors may be useful[0].

[0] https://www.adafruit.com/product/3538


Could be a problem if you plan on sleeping, but you could some rules to get around that


Add in voice, image recording to get gov income as well.


One reason you might want a server is to have more expressive power than "if someone literally enters the room, turn on the lights". You might want to turn on the lights and run the AC a little before you predict people will arrive, for example.


> One reason you might want a server is to have more expressive power than "if someone literally enters the room, turn on the lights". You might want to turn on the lights and run the AC a little before you predict people will arrive, for example.

Maybe, but that seems like a lot of work for little gain.

You probably don't actually have that much control over the AC in your rooms unless you have zoned heating and a lot of zones, regardless of how smart your sensors are.

Intent is a very important factor when answering the question "should the lights be on," I don't think you'll be able to predict that. For instance: if movement is detected in the bedroom at 2AM, should the lights come on? The answer is: a very strong maybe.


> Maybe, but that seems like a lot of work for little gain.

That's how I feel about Alexa.


While I haven't had the urge to get any 'smart speaker', I have a hunch that the current hype* around these devices comes mainly from the novelty of being impressed with voice recognition.

As someone who doesn't have a voice for radio, I much prefer being able to interact through dexterity. Besides, I'm very picky when it comes to which specific track of music is to be played (out of various similarly named pieces of music).

* I don't think take-up is quite as big as the tech world currently makes it out to be. People in my circles don't really have that kind of disposable income and are prioritising other purchases first.

So yeah, Alexa is a lot of extra literal work for little gain.


If you don't have multiple zones you can still effectively have zones by opening/close the registers/vents. This will prevent hot or cold air from entering the room. However, it will also increase duct pressure and that could be somewhat problematic. Either way, "smart vents" are on the market. They're ridiculously overpriced though and one could hack together an alternative for under $50 per vent (source: personal experience). Probably less if they are clever. Of course then there is the problem of running power to the vents or having sufficient battery. But that's an exercise left for the reader.


> Intent is a very important factor when answering the question "should the lights be one," I don't think you'll be able to predict that. For instance: if movement is detected in the bedroom at 2AM, should the lights come on? The answer is: a very strong maybe.

You would be collecting data from many sources in order to predict intent. This is why you need a centralized server.


> You would be collecting data from many sources in order to predict intent. This is why you need a centralized server.

Please explain, exactly, what other sources you would collect data from and how the central server would process it to determine if I want the lights on in the middle of the night.

IMHO, determining intent in this scenario is impossible without...

1. a mind-reading sensor, or...

2. an explicit user signal, such as a button-press or command.

The only realistic option is a user signal, and most of those options obviate a lot of these prediction ideas.

I think there's a lot less practical value to having a "central server" controlling everything than you seem to assume.


> Please explain how, exactly, what other sources you would collect data from and how the central server would process it to determine if I want the lights on.

Machine learning navel-gazing is the new "throw a start-up at it", so I'm guessing the answer to this is going to be "if we have enough data..."


> Please explain how, exactly, what other sources you would collect data from and how the central server would process it to determine if I want the lights on.

The more data, the more smarter. In fact, the only difference between a thermostat and the human mind is the number of datas. This is because the Law of Averages predicts that half of all datas will be relevant.


> The more data, the more smarter.

There are several important caveats to this statement.

> In fact, the only difference between a thermostat and the human mind is the number of datas.

This is an oversimplification that borders on dogma.

> This is because the Law of Averages predicts that half of all datas will be relevant.

An interesting interpretation and application.


I really really hope this was a joke, because it was incredibly funny if so and terrifying if not.


Your solution is simple. However, it doesn't allow for as much modification as the original poster's idea. With his idea it could be modified in so many ways to add functionality because it wouldn't be limited by the technology. The solution you supplied will eventually be limited by the technology if the original poster wants to add other functions.


http://www.danielsen.com/jokes/objecttoaster.txt

> The second advisor, a software developer, immediately recognized the danger of such short-sighted thinking. [...] "A toaster that only makes toast will soon be obsolete. If we don't look to the future, we will have to completely redesign the toaster in just a few years."


YAGNI. You can get to a 90% solution today at 10% of the cost and effort. If, later, you want to extend the system you’ll not only have learned a lot about the operations and failure modes of the current system, but hardware purchased later will be cheaper and may support even more functionality.


Sure, but what is the value gain/opportunity cost between a simple, two hour installation that achieves a majority of the desired outcome with nearly rock-solid stability vs. sinking a massive amount of time into a bespoke and likely fragile system?


> However, it doesn't allow for as much modification as the original poster's idea.

My main point was the original poster's idea was probably focusing on the wrong kind of sensor for what he wanted to do. He could still network a bunch of motion sensors.

Also, I'd dispute the idea that my proposals were less "modifiable." If only because they're far easier to implement and a couple of orders of magnitude cheaper, so it's practical to replace if more capabilities are needed.


There may be a market for something between a standard one-bit low-rez motion sensor and a full color TV camera. Maybe a 16x16 pixel IR sensor with a fisheye lens and a puny CPU that reports an approximate number of people in the area, for HVAC and lighting control, and security.


Reporting an approximate number of people is not a trivial task, even with a real camera. Depending on what exactly you mean with "approximate", of course.


0, 1, a few, many. Just enough to tell you how much to crank up the HVAC.


> 0, 1, a few, many. Just enough to tell you how much to crank up the HVAC.

I don't think the number of people in a room will give you meaningful information to tell you "how much to crank up the HVAC." Also, the HVAC systems in most homes aren't capable of even cranking up the HVAC in a particular room.


True. The market for this is schools, offices, and hotels, where the people load changes drastically and the HVAC has to react to that.

This is what the "Internet of Things" should be doing, but seldom does.


If you buy occupancy switches instead of vacancy switches, the light will stay on until you actually leave!


https://www.home-assistant.io doesn't have computer vision, but can use motion sensors and the like to tell where people are, and use that to activate/deactivate devices. It's all open source and runs entirely locally.


PIR sensors can tell you whether there are people (or pets) in a room. Never tried one, but they should work pretty well.


On making, not buying...

A lot of home automation stuff seems like something that would be fun to make, but not something I really want to have. I mean, the AC might be a good energy saver (a smarter thermostat) but that's not something I want, more like something that I would buy if it was cheap and practical enough.

To some extent, this is a phenomenon of early technology. It feels full of potential, so you want to play with it but it doesn't really do anything you really need yet. The early web was ki d of like that. We made sites because we wanted to make sites, moreso than because we wanted to have them.

Home automation though... Im kind of skeptical that this goes anywhere useful. The useful examples people think of (eg remote close all the windows and lock the doors) are more about mechanisation than automation.


At my house, we have blinds installed on every window, and every evening I go around the house, pressing 12 switches (2 for every blind, because UX is obviously optional) so it's perfectly dark in the bedroom. In the morning, I go around the house and press those 12 switches again, to let the sunshine in. Now that's a task I would love to automate!

Alas, remote controlling those blinds would be a major hassle since AFAIK I would have to install 6 wifi-enabled devices and tear holes everywhere (not even sure there is a powerline near) - and likely do all the programming myself. Thanks but no thanks.


If it bothers you enough, take a look at ZigBee switches. Not sure I understand how you operate the blinds (since the current switches are apparently not on powerlines?), but you could still make a centralized solution to control them over ZB. There's https://www.home-assistant.io/ if you don't want to program (much), and probably others. And you will smile every evening and every morning for the next few years, thinking of the time when you had to do it manually... not to mention, it's cool. :) You might need to invest something in these devices and controller though.


Thank you, I will take a look!


Yeah it's not really relieving a pain point. I've never thought "O why lord, why must I toil away flipping these light switches as I enter and exit rooms?"


Another pain point with making is that it's really hard to get it integrated with stuff you have bought. I built my own garage door automation with a Particle Photon board. It works great and can do things like text me if I leave the house with the garage door open using the IFTT support from my WiFi router. The problem is that it's really hard to get it integrated with any other control system that the rest of my house uses like my ZWave light switches and Hue lights.

I've been working on a custom UI that sits on top of the Wink Hub API to unify everything, but I'be been stuck with their almost completely undocumented Pubnub event API.


I would check out https://www.home-assistant.io/ for integrating multiple systems together. I use it on a RasPi to integrate a few disparate systems (Amazon, Nest, RadioRa2) and it works very well. There are modules for most existing systems and it's easy to write your own in python.


I probably should try it. I've been writing my own partly as an excuse to learn React. I've already got a Pi with a touchscreen and a 3d printed enclosure setup to run whatever solution I actually end up using.


I know I’m a little late here, but I would look into MQTT as a transport layer for messages across your different devices. It’s super easy to interface with via python or a host of third party services.


> "That server makes decisions on if lights should be on [...]"

Why would you even want such a system?

My flat uses the Button-Framework which provides a really convenient UI/UX. The edge between 2 nodes (rooms) has a button (a haptic device to switch between boolean states) at around hand-height that you can press to switch the light on or off. It works quite well (it uses a switch-system technically) and the user decides by her/himself if the light should be on or off and just presses the button after careful reasoning.

Granted, it doesn't use Docker, but it really works well and needs low-maintainance. My model is running since 23 years and I never had any issues - it's even open source.


Is there a Github repo?


There's a whole class of tech like this for me. An Alexa/Echo/etc, a fitness tracker with GPS and sleep monitoring, a maps program that learns my routine and integrates with a weather app, and so on.

And ideally? All of it integrated. It actually sounds nice to say "I'm going home", and have Maps say "today that will take 35, should your oven start preheating when you're 20 minutes out?" IoT devices are overrated, but I can absolutely imagine a critical mass of integrated tools being very useful.

But I'm not even slightly willing to do that. It's too much information and too much risk surface. I'd pay a hefty premium to get local-data-only versions of these products, but no one is offering that, and it doesn't look like they're going to start.


>a maps program that learns my routine and integrates with a weather app

That sounds awesome until the company providing that service starts abusing their knowledge of their location. That abuse doesn't even have to necessarily be malicious in nature either. For example, Google Maps on Android started asking me to rate, review, and/or take photos of my present location if they deemed it a point of interest (certain restaurants, parks, etc). I never opted in to this feature and the only way to disable it that I've figured out is to literally disable all location services on the phone.

I really dislike the idea of Google storing a timestamped record of almost every place I've ever visited. Tt has to beconstantly phoning home in order to deliver the request to document my visit within a minute or two of my arrival and that constant reminder that Big Brother Google is tracking me at every moment is just disturbing on so many levels. Even if they aren't using that data right now, remember the "data is never destroyed" principle of the internet.

On a side note, I would have ditched Android if I didn't need it for work simply because use of the GPS radio is hidden behind the acceptance of enabling Google's Location system and all the invasions of privacy that entails.


I feel the same way. Between that and my overuse of my phone for checking hn, reddit, etc. I am considering trying to make a habit of leaving my phone at home when I leave the house, at least some of the time. It's a shame that I feel that way about such an incredible useful device, but there it is.


I like home automation but there are certain things I would never connect to the internet. Oven is one of them.


Maps currently does do what you're saying - Google knows the time I'm going home and displays current traffic information and estimated time to arrive, at least for me.

Not yet preheating the oven though :)


Same. But it gets such obvious things wrong. Instead of "traffic to [daycare center] is light" it says "traffic to [some weird company name probably registered in the same building] is light".

Waze also knows where I'm going in advance and most of the time it gets it right, like every Thu evening "are you heading to [evening school] ?". My weekly schedule is exactly the same 99% of the time but sometimes it still makes obvious mistakes. If you need an AI for that it must be a very simple one, yet it still fails.

Google maps also auto saves my parking location most of the time but other times it doesn't, for reasons unknown. Things like that make it hard for me to have faith in future versions of this stuff.


The bizarre thing is that Waze is owned by Google. People speculate they haven't merged information due to regulatory reasons.


Check out https://www.home-assistant.io/

Open source, gets you most of the way there.


I actually used to have a solution with home-assistant, find (uses wifi signal fingerprinting for location) and hue. It worked okay with very little work (sometimes 1-5s delay turning lights on/off) but I never cared enough to put more work in to get it better.

I think "find" and similar tools are now much more advanced so it might work better out of the box now.


Even if you build it, knowing guest may never feel completely comfortable visiting your house.


They should feel more comfortable than an equivalent system built by a company that is looking to profit off of your data - and additionally, you can give the guest stronger guarantees that when you say that the system is "off", it actually is.


>> Even if you build it, knowing guest may never feel completely comfortable visiting your house.

> They should feel more comfortable than an equivalent system built by a company that is looking to profit off of your data - and additionally, you can give the guest stronger guarantees that when you say that the system is "off", it actually is.

They won't, because they'll be concerned the person that implemented it is a creeper. When regular people think "cameras installed in the room," they think this: https://www.bustle.com/p/airbnb-host-was-caught-with-a-hidde...

Distrust of a person is more more immediate in visceral. Distrust of a company is more distant and esoteric. The former usually trumps the latter.


The chance that a random implementer has a security vulnerability is much higher than that Jeff Bezos is listening to me watch TV. A private system is more vulnerable to target attack and an Amazon system is more vulnerable to mass surveillance.


You are on the right track, but I don't think you are quite right. Amazon like systems are more vulnerable to mass surveillance.

Your vulnerability isn't a targeted attack: you are not valuable enough to be worth the effort to figure out your system. As an attacker on your system I'd have to figure out how to break in, and then how to use the hardware you have. You are more valuable as part of a botnet - attacks that already exist. (if you are a politician then maybe, but that person is also vulnerable to a targeted attack on their amazon system - probably more so because the target is easier to figure out).


And what exactly does this "vulnerability" mean in real terms? Let's be honest. No one cares what conversations are going on in your house _unless_ you're someone specifically targeted. There's little use in mopping up data with no goal.


If I'm using a dragnet to grab every conversation and filter it for things I personally care about, your conversation could make you someone I want to specifically target.

As a random example: let's say that I want to kill someone who lives in your neighborhood. I could analyze your conversations, comings and goings to figure out when and how to kill that person, and blame it on you. I could on a continual basis run numbers on everyone emitting data in your neighborhood, until someone had arrived at a point where their friends would testify against them on the basis of conversations with the potential patsy, that patsy had no alibi, and tailor the murder on the basis of the means that the patsy had available to them at the time.

I could also just be trying to figure out if you were a homosexual, or muslim.


If mopping up everybody's conversations is cheap enough Russia/Iran/China/(insert your favorite large evil) does. If you happen to run of political office 15 years from now having all your conversations available to analyze will be useful. If they don't like you, you might find some out of context snippet of "private conversation" all over social media killing your campaign. (or alternatively the blackmail threat if you don't X)

That is they will target everybody because they know in a few years that will include somebody who they currently think is a nobody.

Of course as AI gets better and cheaper they may eventually listen to everything to see what who can be targeted automatically for what.


Economic espionage is a mainstay of intelligence agencies, so you don't even need to play the potential politician scenario out.

Just working for a company that an agency is surveilling is enough to make you worth targeting.


It depends on what future you want to live in. I prefer one, where this is not okay.


Why not just program it in Rust if you want to guarantee that the system doesn't have any security vulnerabilities?


A secure language does not protect you from insecure design. I could very easily build a system in Rust with gaping security holes, purposefully or accidentally.


Nice troll


That's true - and I suppose that some people are much more concerned about the former threat than the latter.


The Personal Cloud is what we should be building for ourselves. Servers running in your closet at your house.


I like the idea (though I think with enough abstraction, you could have it also replicating itself to "regular" cloud).

The main fight needs to happen at application level, not infrastructure. Cloud services are already mostly transparent and interchangeable. But applications aren't. The problem is, it's the application vendor that owns the code, determines where's going to run, and asks you to send over the data. How it should be working, is that you own the data and determine location of computing, and own or rent code to be run on that data.


Any idiot who breaks into your home would be well positioned to steal your server along with all of your memories and use them for ransom/blackmail/etc.


As opposed to any idiot who breaks into your IoT provider's server?

An encrypted hard disk with the key on a USB stick would be enough, just keep the key somewhere separate and you'll only have to plug it in when there's a power outage.


What about embedding into the walls themselves?


So like a computer then.


You don’t need to use cameras for this. A simple speaker and microphone is all you need to make a functioning motion sensor. Just exploiting the doppler effect. And even better, you can do it all outside of the human audible spectrum. And it can work with capturing motion around corners, too, since sound bounces off walls.

I’d recommend something like this in every room, or even IR sensors, over cameras. You don’t want to capture video of your children jerking off.


> A simple speaker and microphone is all you need to make a functioning motion sensor.

You could just use an infrared motion sensor. They are dirt cheap and widely used for controlling lights.

The AC unit at work has one, too.

So, all of this already exists. Without spying capabilities, that is.


And then the distro will be abandoned and be an open sore for hackers. IoT devices are a mass grave of abandoned Linux distros.


I've made plans for essentially this, but using infrared sensors to track occupants rather than cameras.

There are many methods for occupancy tracking without cameras. IR sensors, antennas in the walls, NFC (bleh), proximity sensors.


This is eerily similar to the concept of a 'cookie' seen in the Black Mirror episode, White Christmas. (Spoiler Alert)

A cookie is a device "that is inserted under the clients head by the brain and kept there for a week, giving it time to accurately replicate the individuals consciousness. It is then removed and installed in a larger, egg shaped device which can be connected to a computer or tablet (to automate their smart house controls as if the house was knew their personal preferences moving from room to room.)"


I want to say "that episode really freaked me out" but... That's true for almost every episode of Black Mirror.


I get freaked out every time someone on HN proposes a tech not too far removed from Black Mirror plots. Great show!


IR camera would probably make this simpler than a visible-light camera. You want to know if there are people in the room, you don't really care who they are specifically.


I had exactly same idea just about the lights but am too lazy to even try to implement it. Plus sensors require power and that requires cables and that's way too much bother.

Low tech alternative solution from my friends? Get LEDs. Never turn off the lights. This way the room you are going to will always be lit.


Indeed, I'm working on something similar: https://github.com/CtrlC-Root/mdcs


OpenCV + some kind of industrial controller could do this.

hackaday.com/blog/ will have some details on how people set up computer home controllers as well as working with OpenCV


You can literally make it on Arduino or Raspberry Pi for like 100$ and self host it in your LAN in Docker with no Internet access.


I thought this was going to be sarcasm until I got to the end. It sounds like a project where the security would make it too much of a burden.


You might consider investigating say ESP8266 or ESP32, MicroPython, Raspberry Pi, OpenHAB, MQTT, OneWire and wave goodbye to your free time!


From what I gather, if you need fine-grained location information in a building, you can do this without cameras and just a handful of strategically placed wireless access points.

Less likely to be seen as creepy and such systems already exist.

If you just need 'is a person in this room' or 'how many people are in this room' levels of data, solutions to this problem have existed for decades. No need to over complicate a solved problem.


This assumes you don't leave your phone in the charger as you wander around your house.


If you are cleaver you can use the wifi signals as a radar and find people/objects even when they don't have a wifi device on them.

I'm pretty sure all current technology only works with the have a wifi decide on your person. It is significantly easier to do this and for the most part good enough.


Sounds far-fetched to me. If you are cleaver, you are also likely to get stuck in a coffee cup billboard or Eddie Haskell will trick you into insulting your Spanish-speaking friend.


https://www.engadget.com/2017/10/09/origin-wireless-motion-d...

Not only can you detect motion and objects through physical mass using WiFi spectrum with a properly equipped device, you can detect if someone is breathing.

“What's more, this "Time Reversal Machine" technology is essentially just some clever algorithmic work with little burden on the processor, so it can potentially be added to any existing WiFi mesh routers via a firmware update. In other words, security system vendors should take note.”


It doesn't. There are methods that do not require a person to be carrying a WiFi-enabled device for location tracking.


Don't those require more equipment than just access points?


Haven't burglar alarm systems been doing something similar for years?


Well that's what www.aerial.ai is doing using wi-fi signals


How to determine Presence is a very interesting topic.


Such a system would be a huge GDPR nightmare. You'd need consent from every visitor to your house to collect data on them, and also you have to delete it if they ever request you to. Best not to even try.


Interactions between private individuals are out-of-scope.

The following processing is outside the scope of the GDPR:

- any activity outside the scope of EU law (e.g., activities of a Member State in relation to national criminal law);

- any activity performed by Member States when carrying out activities in relation to the common foreign and security policy of the EU;

- any activity performed by a natural person in the course of a purely personal or household activity;

- any processing by the EU itself;

and

- any activities performed by national authorities for the purposes of prevention, investigation, detection or prosecution of criminal offences, or performance of judicial functions.

https://gdpr-info.eu/art-2-gdpr/


Does the GDPR even apply to private individuals like that?


No, definitely not.


OK, the GDPR conversation has officially gone off the deep end. The EU does not care about your server with videos of your house.

More

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: