I wanted to create a voice assistant that is completely offline and doesn't require any internet connection. This is because I wanted to ensure that the user's privacy is protected and that the user's data is not being sent to any third party servers.
I would love for Apple/Google to introduce some tech that would make it provable/verifiable that the camera/mic on the device can only be captured when the indicator is on and that it isn't possible for apps or even higher layers of OS to spoof this
Thinkpads come with that, an unspoofable indicator that will tell you with 100% certainty that your image is not being recorded or even recordable unless the physical operator of the machine allows it. Can't beat a physical cover if you really want to be sure!
Good point, I did conveniently gloss over that one didn't I!
A switch to break the physical wire connection is about the only thing I can come up with out of hand right now. I believe the PinePhones come with physical switches for their antenna's so there's that.
What feels like a self-inflicted issue though is that the switches are fail-open, not fail-closed.
To allow for swappable bezels, the switches (on the plastic bezel) in fact just introduce obstructions in optocouplers (on the camera board screwed to the metal lid), which—I don’t know what they do due to Framework’s refusal to release the schematics, but I guess just cut the power line of the camera resp. the signal of the microphone using a couple of MOSFETs.
The problem is, the camera and the microphone are live if the obstruction is absent and disconnected if it’s present, not the other way around. So all it takes a hypothetical evil maid to make the switches ineffective is to pick up the edge of the bezel with a nail (it’s not glued down, this being a Framework) and snip two teensy bits of plastic off to make the switches nonfunctional while feeling normal. This is not completely invisible, mind you—the camera light will still function, you’ll still see the camera in the device list if you look—and probably not very important. But I can’t help feeling that doing it the other way around would be better.
I vaguely remember another vendor (HP?) selling laptops with a physical camera switch, but given the distance between the switch (on the side near the ports) and the camera (on the top of the display), I’m less than hopeful about it being a hardware one.
You could probably add that if you were sufficiently motivated. Either by adding an LED on the power or data path, or by adding a physical switch. I think it should be fairly easy on laptops; I'm not sure where you'd jam the hardware in a phone or if you can access the cables for the camera/mic without messing with traces.
I'm a little curious if iPhones could be modified to route the {mic,camera} {power,data} through the silent mode switch, either instead of or in addition to the current functionality. I don't really have a need for a physical silent mode toggle, I'm totally fine with doing that in the settings or control panel.
I'm thinking perhaps a standardized open design circuit that can you can view by opening up back cover and zooming in with a microscope.
feel like privacy tech like this that seemed wildly overkill for everyday users becomes necessary as the value of collecting data and profiling humans goes through the roof
The value of the data you willingly transmit (both to data brokers, as well as in terms of the harm that it could do to you) via the use of apps (that are upfront about their transmission of your private data) and websites is far, far greater than the audio stream from inside your house.
If you don’t want your private information transmitted, worry about the things that are verifiably and obviously transmitting your private information instead of pointlessly fretting over things that are verifiably behaving as they claim.
Do you have the Instagram or Facebook apps on your phone? Are you logged in to Google?
These are much bigger threats to your privacy than a mic.
The sum total of all of your Telegram, Discord, and iMessage DMs (all of which are effectively provided to the service provider without end to end encryption) is way more interesting than 86400 images of you sitting in front of your computer with your face scrunched up, or WAVs of you yelling at your kids. One you knowingly provide to the service provider. The other never leaves your house.
Definitely not. Neither Face ID nor the recently added “screen distance” feature in Screen Time use the camera indicator, and both use cameras (and lidar scanners) continuously
It's annoying difficult to try and find an article about it, but IIRC there was a hack for the MacBook Pro's green camera LED where toggling the camera on and off super quickly would wouldn't give the LED time to light up
I think so. I plan to update the README in the coming days with more info, but realistically this is something you could run on your laptop too, barring some changes to how it accesses the microphone and camera. I assume this is the case too for other boards.
The only thing which might pose an issue is the total RAM size needed for whatever LLM is responsible for responding to you, but there's a wide variety of those available on ollama, Hugging Face, etc. that can work.
Thanks. My ideal use case would probably involve a mini PC, of which I have a few spares and most of them are faster and can be upgraded to more RAM than the Pi5, which especially in this context would be welcome.
Extra kudos for the name - and extra extra for using the good old "Picard facepalm" meme.
But seriously - the name got my attention, then I read the introduction and thought "hey, Alexa without uploading everything you say to Amazon? This might actually be something for me!".
> The default wake word is "hey assistant" - I would suggest "Computer" :) And of course it should have a voice that sounds like https://en.wikipedia.org/wiki/Majel_Barrett
Seriously yes, that's what I came here to say - the default wake word should be "Computer", with edit access to a captain's log granted if spoken with sufficient authority in proper Shakespearean English.
Props, and thank you for this.