Home Assistant[1] is the best approach I've found for this. It's pretty amazing but has a pretty steep learning curve (although they're getting better and better at this). The real power of HA is the rich integration ecosystem[2], the Community Store[3], and that it runs locally (more reliable and much faster for things like motion sensors, etc).
It expresses everything from every integration as an entity in various device classes so as long as some random tech is supported by an integration you can group them seamlessly. For example - I have Z-wave based motion detectors but I can control Hue lights and Wemo switches for motion detection - but only if I'm home as reported by the zone feature in their iOS companion application.
Their supervisor/docker based install for Raspberry Pi is pretty slick[4]. Write an image to disk as you usually would and you get a bare-bones OS with Home Assistant running in docker. Then there are additional docker-based add-ons you can install for stuff like MQTT, Z-Wave, Let's Encrypt, SSH access, nginx, and more. All managed in the web ui. Pretty cool.
I'm still figuring out Home Assistant. I love that it has a real ecosystem and that it seems to have a decent HTTP/WebSocket API (that I just started playing with).
Some things I don't love:
* A lot of the integrations seem half-baked or to not quite fit the generic Lovelace cards. I think to some extent this is an inherent problem in supporting a lot of devices that have crappy, non-standardized APIs. Example: if I bump up the low temp on my Venstar thermostat on the Home Assistant dashboard, Home Assistant will immediately set it back. If I dig through logs, I see the thermostat complained that in auto mode (heating or cooling as needed), the low set point and the high set point have to be (at least) six degrees apart. When I adjust the thermostat through its own touch screen, the high set point automatically raises to meet that constraint. Home Assistant should do the same. I've been meaning to look into fixing it myself, but there seem to be weird things like this on every integration.
* You need multiple automation rules to do almost anything. I think it's common to want the state of one thing to match another (for example, garage door open => sticky notification on my phone, person in driveway after sunset within last 5 minutes or switch on => driveway light on). You need a rule in their YAML DSL to for the off->on transition and another for the on->off. It'd be nice to set a rule that defines a level rather than an edge and have it internally do the transformation. It'd also be nice to define the actions of a notification's buttons inline, rather than as a separate automation.
* The companion Android app's notifications seem to be flaky. At least, they're sometimes slow. I think they can be delivered out of order and possibly are lossy, which aggravates the problem with not having something that just sets the state reliably. I assume it's pretty hopeless to have it reliably be in the right state if your phone was off when a notification was supposed to be delivered or Home Assistant was down on state transition or the like.
* Some things seem to be only checked at startup, so my house has race conditions after power outages. For example, if Home Assistant starts up more quickly than my Yamaha AV receiver (+ network switch + DHCP server), I can't control my home theater setup until I restart Home Assistant.
> ... for the off->on transition and another for the on->off. It'd be nice to set a rule that defines a level rather than an edge and have it internally do the transformation
The "solution" I found was to poll, and use "choose", and sometimes some helper switches for state, to do what is needed. But, I think at any reasonable level of complexity, you're better off using one of the three python components: the one built into Home Assistant, pyscript, or AppDaemon.
Thanks for that. I just tried out pyscript. I found it a lot more pleasant to write automation in a real programming language. It also more or less addresses my complaint about needing multiple rules. In the tutorial, [1] the motion_light_rear example addresses this via sleeping + task.unique. It's not the approach I imagined, but it's aimed at the same problem.
> The project group was launched and introduced by Amazon, Apple, Google, Comcast and the Zigbee Alliance, now Connectivity Standards Alliance (CSA).
I'm having a real hard time believing that anything user-beneficial will come of this, unless you're willing to sell out your smarthome and privacy to big tech.
Another way to look at this - maybe this is a way to give a pressure relief valve for all the privacy conscious so they stop harassing them so much for the 90% of consumer case that doesn’t care?
I think people are also very unhappy with latency. The only way to get low latency is to keep things local, and for devices to work together, directly. But, people still want remote access.
> I'm having a real hard time believing that anything user-beneficial will come of this, unless you're willing to sell out your smarthome and privacy to big tech.
At this level, the way data is handled is vendor agnostic.
I see Matter as more akin to USB, or indeed, the internet, as the idea is that devices are supposed to be able to communicate via IP, not just by ZigBee.
> my house has race conditions after power outages
I'd be inclined to find something with an SNMP counter that counts up either "seconds since boot" or "megabits of traffic passed" or whatever else and have a script that runs shortly after the RPi boots to reboot the Pi if that counter if below some threshold.
Running this from the pi will query the uptime on my router in milliseconds:
snmpwalk -v 2c -On -c public 192.168.2.1 .1.3.6.1.2.1.25.1.1.0
You could have a startup script on the Pi do something like "wait until you get a valid reading of this uptime and if that reading is less than 180000 [3 minutes], then sleep for N minutes [long enough to not double/triple reboot on a power outage] and reboot the RPi (or restart HA)"
If HomeAssistant needs to be recycled after every router reboot (not just on a power dump), have that script run in a cron job looking for an uptime less than some threshold and rebooting the Pi.
It might feel inelegant or even "gross", but I look at it as just reliably automating the fix and therefore better than what you're doing manually already.
I also hate that you have to install it as an OS or VM to be fully featured.
If you use the docker version it is crippled with no "supervisor" mode.
Also every now and then a light or switch will go greyed out in HA, their official apps still work, and I have to restart HA to be able to use it in HA again. Pretty annoying when you get home after a long day, soaked in rain, fumbling in the dark to turn on lights and now you have to fumble more to restart the damn server before you can use your lights.
It seems fine that this isn't available in docker, since you can do all of this via docker and your host's configuration system. It would be nice to have a GUI for it, but if you need a GUI then you probably aren't running HA via docker, you used their image.
Same here. Running on a pi 3 with nodered, both in docker containers. Works perfectly and reliably with hundreds of entities, numerous automations, etc. Just make sure you have an official pi ac adapter and boot off an SSD drive not SD card otherwise you will have problems.
I'm running HACS in the container release right now, it's worked for a long time. So long as you have your HA configuration folder as a volume etc it will preserve across restarts/upgrades.
I'd go as far as to argue the (officially supported!) container release is the best way to get a production quality install of HA - containers are a great way to package and release complex web apps like HA. Mines automatically updates itself every time new container image released, has done so with no intervention from me for over a year. With the container lifecycle/config, you don't really need Supervisor mode either.
To say it is crippled is nonsense, it is literally one of the two officially recommended install paths:
HACS works in a container, or is there specific functionality of it that doesn't work? I've been using it inside a Docker HA container on a Pi without issue.
They've also been pretty hostile to OS packagers trying to let you run it without dedicating an entire device to it, most publicly NixOS but they indicated they planned to insist the same against Fedora and Gentoo too: https://github.com/NixOS/nixpkgs/pull/126326
That behaviour kind of makes it hard to trust that it won't lead to the next Marak situation.
This kind of pushed me away from HA. I've ended up using integrations built using hc [0] to bridge devices to HomeKit, either existing or ones I've built, and then using Promtheus to collect metrics. It's not something that 'just works', but I feel much happier with how it works than HA.
For what it's worth frenck does not represent Home Assistant and the first sentence of his profile is "Slightly assholic at first sight"...
That said you have a point but in my usage of Home Assistant almost since birth when it was a single pip install command I can tell you the sheer functionality, integrations, etc make getting it fully running a nightmare. Not to mention from their perspective actually supporting it almost impossible. Maybe this speaks to a wider problem but I'm very happy to install it with docker on the distro and hardware of my choice and have everything more-or-less just work.
I referenced the operating system approach here because it's the quickest and easiest path to get up and running on a Raspberry Pi and therefore relevant to the article.
I gave up on it solely because of the GUI. It's confusing as heck and very far from Apple's Home app, which is the preferred UX of the CFO (Chief Family Officer), even for fairly complex automations.
So I just used homebridge for everything, with a smattering of Node-RED for (very few) device shims (it is _very_ easy to do MQTT handling with it).
Also, I now have zero YAML to maintain, which is also a boon.
Kudos, though, for Home Assistant auto-discovery, which I use with my Tasmota-reflashed devices (there is a homebridge plugin that transparently adds them to HomeKit as soon as they're set up).
For folks here I strongly recommend installing the appdaemon integration and just writing python scripts. The downside is there is a good bit of learning curve and time investment just to get the hello world going (turn on a light at sunset?).
Once you are there, you can write shared libraries, abstract away whatever you want, have legit tests, and just do anything you would normally want in your coding environment (vscode, auto-formatters, source control, etc). Obviously not the right recommendation for a consumer system, but if you are like me and feel more comfortable with a programming language, it's the way to go.
I can do that with homebridge-mqtt (I just stopped doing it in Python because Node-RED did what I needed and gave me a dead easy chart for temperature sensors).
Interested about the tests though. I’m not a professional python developer but would love automated tests for my appdaemon code. How did you set it up, and how do you run the tests?
I used https://pypi.org/project/appdaemon-testing/. You can write a simple test file which does things like sets a device state, then asserts that services were called or whatnot.
It doesn't test that the service calls are correctly written, just that they were made. So, very useful for unit testing syntax / logic, but not a full end-to-end integration test solution. Still, with python being an interpreted language, this is super useful. I don't want to have to wait for it actually to be sunset to determine I have a syntax error in that branch of the code, for example.
I don't actually have the tests running on the HA device. I have them on my desktop where I develop the appdaemon scripts, and then I copy only the scripts over to HA. I'm sure there are multiple ways to do it.
I recently set-up HA on a Raspberry Pi, integrating my lights, security system and other devices. You're right about it having a steep learning curve but once it's up and running it's pretty much set and forget. With HA I'm able to use my Hue switches to toggle a TP-Link Kasa smart power outlet. It also has the added benefit of only requiring ONE application on my phone instead of nine.
I'd also suggest to everyone looking into HA to take the time to learn Node-RED. After running HA for a year and a half, but only ever creating basic integrations, NR has really unlocked the power of HA for me. Part of it is the tutorials for NR explained the internal data model of HA in a way that I finally get it. Plus just ergonomically, I find NR MUCH easier to deal with than HA's native automation UI.
Same. I've been a huge Home Assistant fan and minor contributor for many years. My house is among the smartest houses I've ever heard of. HVAC control, security system, air quality monitoring, multi-site IT systems monitoring, lighting controls (including LED strips that update to the right color scheme for the next holiday), whole-house networked audio (including streaming off-site if connected to the VPN), on and on. All self-hosted, on-prem, zero cloud, coordinated by Home Assistant. Offsite notifications via email, offsite control and monitoring via my Openwrt router's VPN server.
Tuya Zigbee air quality sensors on AliExpress start around $20. I can't speak to their accuracy, but they are supported by Zigbee2MQTT and provide a CO2 reading.
Home Assistant has been nothing short of amazing. I started out just wanting to control my Christmas lights back in early December; tried going the HomeKit with HomeBridge route, only to realize I needed an iHub or some other (i) device on my network to have it work remotely. It was a bad experience even while on my network, luckily stumbled upon Home Assistant looking for better options a week or so into the journey. Now I'm controlling everything, even watching my security cameras, through HA.
The one caveat in my experience has been trying to run Pi-Hole on the same server as HA Supervised or using Adguard Home. Trying to run HA and PiH, even though they were both in docker containers, caused the Pi 3B+ to freeze every few hours. HA makes it clear running anything else is a bad idea, and that proved correct. I deleted PiH and tried the Adguard Home addon for my DNS for awhile, it was really slick and had a lot of features, but I found myself missing the PiH for several reasons. For one, it really sucked to be tinkering with HA while other people on the network were on the web; every time I needed to restart HA (which was often setting up the cameras), I'd get constant groaning and moaning about the internet being out. I also found it harder to control white/black lists; adding individual sites here and there was not as simple as it is in PiH. It also lacked the simple "disable for X minutes/seconds" feature I used so much.
I ended up installing just PiH on the Pi 3B+ and grabbing a cheap Chromebox off ebay to run HA. All the things I was doing on HA really pushed the Pi at times and I read one too many stories about SD cards dying from heavy use. Now everything is zippy and I feel safer with HA installed on the internal M.2 SSD.
I've had luck running HA and node red on a pi 3 but with 250gb SSD drive, responsive and reliable. I think that's the bottleneck. You can't use SD card. Also must use AC adapters specifically for pi, usually slightly above 5 volts and 2 amps I think. Also pinhole must be on separate device I agree with that.
Yep, I could have made it a better situation with some extra addition$ (I always put my Pi on a quality AC adapter from the start, those constant command line voltage nag warnings had their intended effect), but the cost of the SSD + another Pi is actually more than a used SFF device with a substantially more powerful Intel processor that comes with a M.2 SSD. It's amazing what you can get for < $50 these days. Great time to be alive for us tinkerers!
I spend my last weekend setting up Home Assistant using VMWare on my spare machine. To say that it has been very rewarding is understatement. I am completely blown away by the possibilities that it enables. For instance, I have all my unsupported devices magically supported in HomeKit via the HomeKit Bridge integration in HA.
I've really wanted to get into this to some extent but it's such a complex (and expensive!) topic that I really would like some distilled knowledge from someone who's been through it and figured out what works and what doesn't, the pitfalls, etc. This is probably a good place to ask - does anyone know of a reliable guide (or ideally blog) to get started? Or maybe would be a good topic for a future HN submission.
My personal focus would be on cloud-free/self-hosted systems. Looping through the cloud as a convenience is fine but I don't really want to rely on cloudshit 100% for the operation of my house.
Also I've heard really good things about the Lutreon stuff? Not open, and not cheap, but it works, and I've heard the z-wave stuff isn't always 100%.
I’ve used Home Assistant on a Pi 3B+ for about a year now. My main objective is like yours, I want as few devices as possible connected to the internet and generally only buy devices that have a local API (local polling in HA terms). I do pay for Nabu Casa, HA’s cloud service for convenient access to everything but nothing I use requires it. This also help support a project that I really enjoy using and rely on.
I have most devices connected to an old wireless router that is completely disconnected from the internet. The raspberry pi is connected to my main network through ethernet and is connected to my “smart network” over wifi. I like to think of the HA instance as a two way mirror since it can reach the internet and see the devices but the devices themselves can’t phone home or access the internet in anyway.
For bulbs, light switches, and smart plugs I generally stick with TP-Link Kasa. The local API works great with HA though I believe they are removing this in future firmware updates [0]. This appears to be for the UK only but I’m not 100% sure on that. I purchased a plug a few months ago that still works.
My thermostat is a Venstar color touch with the local API. I have it connected to the internet but I could disconnect it and everything would still work how I want.
You can flash Wyze v2 cameras with the official RTSP firmware to view them locally. You lose some of the features of the Wyze cams but I think this is a fine trade off. While this works with the Pi, streaming video taxes it a lot. It isn’t a perfect experience by any means.
I also use Zigbee devices instead of Z-Wave (no real reason other than the first device I wanted was Zigbee). I have a Sonoff Zigbee bridge flashed with Tasmota that allows me to use any compliant (or quasi compliant in the case of Xiaomi Aqara) plug, motion sensor, temp sensor, etc across brands. The Zigbee bridge is also has access to the internet because it needs to connect to an NTP server. This means that these devices work fine when the internet goes out but if for some reason the hub gets power cycled while having an internet outage they stop working.
You can get Zigbee antennas that connect directly to the Pi but I avoided this because the USB ports can cause interference. You can fix this by getting a USB cable extender.
Feel free to ask any other questions about my setup.
I personally just flash everything possible with open-source software. All my light switches and Wi-Fi lightbulbs run Tasmota, which I password protect and semi- regularly update. The one odball is my Gree AC unit, which is on a separate WiFi network and subnet (I just run hostapd on my RPi since it's close-by). But it's more to protect it from the outside, not the other way, because it's controlled with JSON sent over UDP, and if you send a malformed packet the microcontroller inside crashes. When this happens and the compressor is engaged it will freeze up, possibly destroying itself and start flooding the floor, because the condensation pump won't turn on.
> But it's more to protect it from the outside, not the other way, because it's controlled with JSON sent over UDP, and if you send a malformed packet the microcontroller inside crashes. When this happens and the compressor is engaged it will freeze up, possibly destroying itself and start flooding the floor, because the condensation pump won't turn on.
>> When this happens and the compressor is engaged it will freeze up, possibly destroying itself and start flooding the floor, because the condensation pump won't turn on.
Or maybe not automate destructive devices. Might be simpler and more responsible anyway, since equivalent harm can be had if your kids ask Siri to warm up the house...
Automation bugs are dangerous on any home device. I once accidentally programmed an infinite recursive loop on all the lights in the house so had on and off signals being sent nearly simultaneously. Scary and not a good thing but easy to do particularly with wifi and mqtt messaging. Had to shut main power off. Luckily only one light switch broke.
Smart lightbulbs are crappy, since If you turn them off using a light switch, you can't turn them back on via Wi-Fi. I've installed smart relays (Sonoff mini) behind light switches, which means I can control them localy and remotely at the same time.
That being said, anything that can be flashed using tuya-convert will be good. But beware, because not all Tuya lightbulbs hava an ESP chip inside and they have patched the exploit tuya-convert uses in newer firmwares. So check your model on google.
Before I got a Raspberry Pi with home assistant, I was trying to write a simple client for this protocol in C so I could host it on my OpenWRT router. Of course my client bugged and sent some random junk which caused the unit to completely lock up, it displayed 88 on it;s display and didn't respond to the IR remote. After this I had to reset it via the circuit breaker.
If I recall correctly, the data was encrypted using a static AES key, and I think the unit checked if the data decrypted correctly, but don't quote me on that.
In addition to all of the "use Z-Wave/Zigbee as much as possible" comments (which I personally do), for stuff where there's no choice like Roombas, I have a separate SSID on a VLAN that can only talk to the internet, my DNS server running Adguard, and HA (which is on the main network). I also have stateful rules that allow connections to be initiated from the main network to the IOT network, but not vice versa. I also try to find things with ESP* controllers, as mentioned.
I've really been trying to move away from cloud based stuff not only because of data privacy, but concerns about services going away and (most importantly) latency.
My old wifi smart plugs had 2+ seconds of latency from when I'd hit the button in HA to actually turning on. Zigbee/Z-wave stuff is (from my meatsack perspective) instant. I have a zigbee door sensor on the door to get into my garage, and a zigbee smart plug connected to the overhead florescent lights. By the time the door opens enough for me to actually see inside the garage, the lights are on. Sometimes I think they never turned off. It's fantastic.
For an actual router, I run OPNsense in a VM (on the same box as HA, funnily enough). My server has dual 10 gig ports, so I pass through one of them to OPNsense and run it as a "router on a stick", where the basic internal network is the untagged VLAN, and the public side is VLAN 99. My switch then strips the VLAN 99 tag from the packet before sending it to the cable modem, and vice-versa. My switch is https://mikrotik.com/product/crs328_24p_4s_rm, which was $379 in 2018. However, you can get similar functionality from MUCH cheaper microtik or unifi/ubiquity switches if you don't need 10 gig support.
If you're starting from scratch I might get something like https://store.ui.com/collections/unifi-network-unifi-os-cons..., and an el cheapo dumb switch, as long as it'd pass through vlan tagged traffic. You want anything that's crossing a vlan boundary to end up on the router anyway, so you can apply firewall rules to it.
Do you feel that using a virtualized router with other things on the box puts you at greater risk? I.e. you're putting an awful lot of eggs in that one basket. I've considered setting that up before but shied away from it.
Availability risk? Sure, but that's 99.9% my own doing (constant experiments, etc)
Security risk? Not in the slightest. I'm running an up to date proxmox which is just KVM+QEMU with some scripting and a website on top, basically. I know how to setup IOMMU groups and such. If it's good enough for the big cloud providers, it's good enough for me.
Yeah primarily I'm concerned with availability. I have a "production" hypervisor now with all of the household services running on it that I've promised my spouse I wouldn't mess around with which cuts off one avenue of experiments.
Yes, the Unifi OS console they talk about is similar to the Unifi controller that you can run standalone on your desktop. It'll handle the adoption, handoff coordination, everything you need.
The Ubiquiti Access Point AC Mesh Pro[0] (~$200) appears to support multiple SSIDs and VLANs. Of course you'd need at least two of them to count as a "mesh", which is $400… was your $500 budget per access point or for the whole system (and if so, for how many APs)?
2 will cover my needs. Do they actually work like a mesh? As in use backhaul radio to communicate between them and allow seamless switch from one to another as you travel around your home?
Yes, this model supports Wireless Uplink with Plug & Play Mesh:
> Wireless Uplink functionality enables wireless connectivity between APs for extended range. One wired UniFi AP uplink supports up to four wireless downlinks on a single operating band, allowing wireless adoption of devices in their default state and real-time changes to network topology. For devices that support Plug & Play Mesh, this functionality is extended to allow multi-hop wireless uplink – so wirelessly uplinked APs can support uplink to other wirelessly uplinked APs.
I prohibit IoT devices from connecting to the Internet at all, and only use devices that can be controlled by the local network [0]. Thus firmware updates don't matter (as long as the manufacturer hasn't included any logic bombs), as the trust model is that devices are just an extension of the network segment.
The least-involved way to set this up would be to set up an old wifi router to create a second network, don't connect this router to your existing network or uplink, and then set up your home automation server with two ethernet ports.
[0] eg TP-Link Kasa, although I heard this may have changed for recent ones? Either way, with this setup you'll be immediately aware of whether local control functionality works, so you're well within the return window. FWIW I stay away from Amazon's GENSYM brands, even though they'd be easy to flash with Tasmota etc, because I don't trust them to get line voltage design considerations right and I don't feel like QAing every single device.
if you flash openwrt/ddwrt to your router, you can create an isolated subnet with its own DHCP server and enter your own firewall rules (iptables) blocking all traffic from your local subnet to your IOT subnet. You can then bind that to a wifi ssid (the guest one). Its kind of a PITA to setup, but it works great once you get it. Netgear Nighthawk routers have worked great for me in the past.
Don't forget to set up a reverse proxy with ssl and automatic cert renewsl, even in your home network. Wifi can be hacked with trivial ease. Caddy or nginx/certbot will do you well there. If you also run a pi-hole you can have the pi on your local network pass ssl checks by overriding some DNS entries.
I just don't use Google or Alexa (devices from those ecosystems punch holes through my home gateway for a bunch of server-side glue that is extremely ill-advised, and a bigger risk than LAN security IMHO), moved most things (sensors, typically) to ZigBee, and reflashed everything I could with Tasmota.
It's extremely inconvenient to have IoT devices segregated from, say, TVs and media devices, especially if you rely on AirPlay or Chromecast to get audio around the place, so I just secure my Macs (and PCs) properly as if we were still traveling and visiting clients.
I keep tabs on Apple security bulletins (HomeKit has relatively few issues, and works mostly inside the LAN except if you're outside the house - it then switches to a variation of the old iCloud "back to my X" tunneling).
One, I avoid WiFi products entirely if there is a good alternative. Zigbee and ZWave are two good ecosystems, you get a USB radio, plug it Home Assistant machine and you're up and running.
A second option is finding devices that can be flashed with Tasmota or ESPHome. This could also mean putting together your own devices with an ESP8266, ESPHome is basically plug and play for simple things like temperature sensors. You assemble the device, configure which pins to use via YAML, and then flash it to the ESP.
You don't even need to download a local build toolchain, the ESPHome add-on to Home Assistant can flash devices plugged into your computer just from the web interface, and then do OTA firmware updates.
Not OP, but I make extensive use of VLANs to isolate off security cameras, IoT devices, etc. etc. into their own mutually isolated network environments.
I also don't use Wifi for any of these devices anymore. It's usually a bad experience. Either use ethernet or a dedicated IoT-oriented wireless protocol. Z-wave seems OK but not very flexible, Zigbee is a pain in the rear (but the only option for many device classes), and I'm hopeful that Thread will actually be good.
But, I think it's actually because wifi is easier for the average person to get working. With Zigbee, you need a Zigbee hub, but sometimes you need a brand specific Zigbee hub, sometimes you don't (even though it's advertised that you do), and sometimes the Zigbee compliance is so bad that adding a device from another vendor break your whole Zigbee network (looking at you Aquara). Zwave throws more incompatible hubs to the mix. And, even within these, it's rare to have devices work with each other in a way that makes sense.
Hopefully matter saves us, so I don't have to install 5 integrations in Home Assistant to remind me that my car isn't plugged in at night or my back window is open, while the heater is on, and then another to make any of it accessible to HomeKit.
I must be lucky then because I have lot of Zigbee and Zwave devices all running Transmitting to Nortek GoControl USB stick attached to a rPI running Home Assistant.
I have several vendors of both Zwave and Zigbee Sensors, HVAC thermostat, Bulbs, Switches, etc.. All of them play nice with each other, and FAR FAR simpler to setup than WiFi which often requires the use of some weird mobile app, and play hopsotch with the networks..
Zigbee I just pair them to the GoControl and it is done
hell I even bought some no name used Door Sensors off ebay that were Zigbee, I mainly use them for Temp monitoring in various places.. They had no problem connecting to my network either
Yes, but you are not the average consumer, who is making Wi-Fi popular.
> all running Transmitting to Nortek GoControl USB stick attached to a rPI running Home Assistant
> and FAR FAR simpler to setup than WiFi
These two sentences are absolutely silly, for that average consumer. An average consumer can install a proprietary app in a few minutes and is totally uninterested in installing operating systems on little computers.
> Before I moved to HA, I was in the smart-things Ecosystem. That was also very easy out of the box, far easier than WiFi Device
No offense, but I think you've already proved that your perspective is a bit skewed towards the technical. I don't understand how buying and installing a hub, then installing an app to connect with that hub, and then using that app to pair is somehow easier than installing and app, and then using that app to pair.
> It has nothing to do with ease of access...
Looking at Amazon, I'm seeing many times the reviews for WiFi smart switches. This would make it appear that people are purchasing them. Why? Are they already locked in? My theory is that people see the product description, see WiFi, and say "I have that!". People see Zigbee or "hub" and say "I don't have that, it's $50 more!". If we could do a word count for "incompatible" between the two, I'm sure it would be many times lower for WiFi devices. I know I see that word often when I look through Z-Wave and Zigbee reviews.
If you don't have any need for bleeding edge devices, Openhab still works on a rpi4 as an alternative to HA. I like that I can just install it as an apt package onto an existing rpi with other things and java just does its stuff, rather than dedicating the entire rpi4 to it or running in a container.
> I like that I can just install it as an apt package onto an existing rpi with other things and java just does its stuff, rather than dedicating the entire rpi4 to it or running in a container.
If you try Home Assistant again, you'll probably like Home Assistant Core. No separate OS or container needed, just a Python virtual environment. That's how I have it installed. I have to do the updates and backups manually (rather than it self-updating automatically / through the UI) but I find that preferable to dealing with an extra machine, VM, or container.
I had a terrible experience with Home Assistant. Slow, buggy, unreasonably difficult to implement non-trivial automations.
My heuristic since then has been to minimize the amount of node.js in my home automation codebase and this has worked wonders for its reliability and usability; currently the only component I still have that's node-based is zigbee2mqtt, which (possibly not) incidentally gives me more issues than any other component.
The best approach I've found has been to have (in isolated jails/containers) zigbee2mqtt dealing with zigbee network stuff (you must use CC2652 based coordinators and routers for any kind of reliability), mosquitto acting as an MQTT broker, and a custom program (write it in whatever you like) talking to mosquitto, with Pushover for push notifications.
One of HA's selling points for me originally was that everything was written in Python. It's frustrating that I am now faced with pulling a javascript service into the mix. Most of my devices are z-wave and I'm still running on the legacy integration and dreading this migration.
I've had mixed experiences with python (mosquitto is fine for example) but more uniformly bad experiences with node stuff.
I would vastly prefer all of this crap to be written in Rust or Haskell or OCaml or anything oriented towards correctness. It's relatively very important that the code which runs your home works correctly.
I do the same for the automation part (mosquitto+zigbee2mqtt+custom scripts), but I do have Home Assistant for GUI and logging. For example I have a card in Home Assistant that shows whether any system at home doesn't ping.
Does anyone know how Home Assistant compares to node-red? As far as I can tell Home Assistant is better at building a dashboard of things, but node-red is better at actually automating specific actions (unless you want to grapple with HA's YAML based scripting language).
Is this about right? Is it possible to combine Home Assistant and node-red, or has Home Assistant outgrown the need for something like node-red?
For me node-red is more of an addition to homeassistant. There is node-red-contrib-home-assistant-websocket [1] to let node-red speak with homeassistant, you can watch state changes etc. I use ha for like two years now and at the start there was no inbuild automation-backend like it is now so we did everything with node-red.
It is possible, and a few years ago seemed quite a trend (just going by /r/homeassistant spectating) - https://github.com/hassio-addons/addon-node-red - but I gather HA's own automation scripting has at least visualisation if not GUI programming now.
IME: For simple automations, HA UI is far easier. For complex ones, if you want a GUI, node-red, if you want to handwrite scripts, HA. If you don’t care it really depends, in some cases coding them is easier, in some cases node-red is easier.
But then this is about Node-Red integrated into HA.
I've settled on homebridge to talk to the Home app on my iPhone (which is where all the automation is done, and then executed on an Apple TV) and a little Node-RED for custom flows.
I have zero YAML to maintain, and I'm happier for it.
They are not comparable because node-red is a generic low-code programming tool while HA is home automation software.
If you mean how they compare when you plug in node-red to HA and use it for automations, rather than use HA's "native" automations... I believe the consensus is that node-red is more user-friendly in that way, and because it can plug to other things on its own it can provide more flexibility. On the other hand, HA has recently worked on the UI of their automations, and it is already pretty good for simple flows. In the end both of these approaches are great, each with its own pros and cons.
+1 for Home Assistant. Have it running on RPi4 for almost two years already without single glitch. It has tons of integrations, and besides that you can always build your own Arduino modules to connect with it over MQTT if you have your special use case. Web UI is really nice, you can even access and edit config files with it without going ssh. Android/IOS app also works pretty well.
i have a NUC i plan to dedicate. i seem to recall that there were some features missing from the installs that were not managed (or whatever the Whole OS thing was called?
I'm aiming to use a supervised installation to free up the pi for other tasks, but concerned about performance. Will be booting from SATA to help to this end, and to avoid degradation from writing to microSD often (this may be the case when I start recording security footage)
You will probably not have a great experience running cctv from a pi. You may want something a bit beefier unless your cams are low res, mostly due to transcode/copying costs.
In terms of disk, you want multiple disks as either raid/zfs/btrfs/lvm. It will probably be better to do this on a separate machine and mount it via nfs.
For your boot volume, you can also use a usb thumb drive to help avoid sd card fatigue.
I haven't had any issues just using the stock image on an SD card - it seems to be fast enough. YMMV especially if you plan to run other things on the Pi, though in this case I'd recommend going for a mini-PC instead and just installing a standard Linux distribution on it.
I see. Really it was just going to be something like pihole which is supposedly light. Maybe better not to use the supervised if this is focused on running HA?
In the future I'll consider the mini-PC, thanks. This will have to do for now.
Does anyone have some experience to share with integrating home assistant with Homematic devices? I'm stuck with those for controlling my heating but really dislike the interface.
Better yet, run your home on Dell/HP micro computers that get decommissioned by enterprises/hospitals/big-orgs every 3 years. Find them on Offerup or FB or Craigslist market place for $100-$200.
These are built solid with metal enclosures, fan cooled, UL certified PSU, ~7k CPUbenchmark.net. Usually i5 6thGen Intel CPU, 8GB-16GB RAM and have REAL storage(SSD) and M.2 NVMe. If you are lucky you might find vPro version with remote management.
Currently running a HP G2 bought on Offerup for $70 with i5-6600T, 16GB RAM, 256GB SSD running Ubuntu server with HomeAssistant VM and few containers. It idles at 7w, with <2% CPU for my work load.
I tried Raspberry Pi with HomeAssistant with it's piss poor antique storage solution, it idled at 5W, it went straight back to store.
Putting a server in a corner of my home and forgetting about it is an application where I want the machine to be powerful enough to work without active cooling, as small as possible, and with no moving parts. None of these constraints is difficult to accomodate in this day and age.
Ya it's possible to build a fanless system, but Apple or any OEM with those parts is not going to sell you that system <$100. Not with out of the box compatibility with every x86 software out there.
The fan ramps based on usage, as I said, Ubuntu server with HASS OS, VS Code Server, Netdata monitoring, NodeRed, Sonos Airplay, InfluxDB and few other containers, it's barely gets to 2% CPU usage and CPU temp at 25C.
Look at Pi, they have been at it for SO many years and that shit is slow with no real storage interface, they are only faster compared to their own previous generation.
> and that shit is slow with no real storage interface
We were talking about using it for a home automation system, which is just a very low traffic web server with some odds and ends. We are not talking about competing in the TOP500 or something. It's easily doable with a newer Pi.
I do like some of the thin clients you can get cheap off eBay. A used Dell Wyse 3040 only costs a bit more than a Pi, and gives me full x86. Still no fan. If I didn't have a stack of Raspberry Pis, that would be my choice with the shortages.
I would probably move my Home Assistant setup to one if I hadn't preordered their Home Assistant Yellow board which has built-in Zigbee and other features.
I did that too but those PCs are just stupidly massive for the purpose. Raspberry Pi 4s don’t require SD cards any more so the storage issue is moot. My RPi sits next to my router now, you wouldn’t even know it’s there unless you looked closely.
Did you read the part which says "micro", these are about 2-3x the size of Pi with those cheap plastic cases. I had mine beside the router before I moved it inside TV console.
Pi will be single purpose device, the "micro" can be full blown server and comes with ALL parts required, no need to hunt for good PSU, case, cooling, storage. Some of them come with 1 Sata and 2x M.2 NVMe, so you can literally pass those NVMe to TrueNAS in proxmox and run full blown NAS with encryption/encoding/decoding in VM.
May be you mean 2-3x in every linear dimension but that still seem to be smaller than those in my mind. (While Mac mini is not known to be the smallest, it definitely exceeds 2-3x in some linear dimension already.) What kind you have in mind?
I just migrated all of my "production" home things to an HP Elitedesk 800 G3. It's great! Maxed out the ram and threw a 1TB SSD in. I'm using Proxmox and have:
* home assistant VM
* pi-hole primary LXC
* UniFi controller LXC
* media stack LXC (jellyfin is installed directly on the host to minimize hardware transcoding issues)
* a VM running various docker containers with dokku, including AppDaemon
It's barely ticking over most of the time and using very little energy.
We use zwave devices around the house and due to issues making zwave complex (RF + brick = bad, plus it's a very long house), I actually have a handful of Raspberry Pis with zwave sticks running zwavejs2mqtt scattered around. Yesterday I decided that I'm tired of dealing with the rPi-zero-W being so slow so I ordered a bunch of Dell Wyse 3040 thin clients off of eBay to replace them. Still trying to decide how I'm going to run them but probably it'll look something like Alpine diskless running docker and zwavejs2mqtt in a container.
Yeah. I haven't set that up yet after getting Proxmox up and running, but it's something that I should probably work on.
I've had such a good experience with this one that I keep trying to figure out if I can replace my Dell T20 minitower router running VyOS with one. The blocker is that the Dell actually has standard PCIe slots, whereas if I were to use a USFF I'd have to either go way up market to something like a Lenovo M920Q or accept USB ethernet adapters.
I have similar services running on my synology nas w/ docker.
Where do you get the LXC container images from? I build most of mine from scratch because people on the Internet have no sense of propriety or taste, but now that means I'm maintaining a pile of Dockerfiles, which I never update in practice.
WHAT STORE DID YOU BRING THE PI TO? IS IT STILL THERE?
I'll second the recommendation for real PC hardware. Tinkering is fun up until it's not. When it becomes not fun is when something breaks, takes down a system you have come to rely on, and now it becomes work to recreate the functionality you used to take for granted. Seeing people go for the RPi for something they're going to put a lot of hours into makes me cringe a bit, having had several SD card filesystems get corrupt. If that's what it takes to get you started with self hosting, by all means go for it. But there is much nicer hardware for running servers on.
Having said that, I still use RPis for edge devices and the like.
PS I really like that idle power figure on your server. I wish there were a list of power consumption figures for integrated hardware, it seems quite hit or miss.
I want to use a Pi4 8GB as an arm64 build host so I'm going the USB SSD route there. But adding on additional hardware ultimately seems like the "stone soup" (https://en.wikipedia.org/wiki/Stone_Soup) version of setting up a server.
Furthermore I'm hesitant to rely too much on the arm ecosystem in general. If one pops a drive with a Linux installation into any amd64 machine, it straightforwardly boots. Whereas just making an arm device boot can be its own tinkering session of figuring out its own bespoke partitioning scheme, firmware files, and U-boot image. Many times you're stuck using a non-mainline kernel since that's what the embedded developers hacked on to get something running. Devicetree compiler what? And sure, using a purpose-built distro can hide much of the complexity, until that distro decides your device is too old and drops support.
Heck, I've even moved my wifi AP's over to low power amd64 machines instead of the popular answer of OpenWRT/DD-WRT. When I want to upgrade to a new wifi standard, I'll just pop in a new wireless card. The same machines also run Kodi for the TVs (which used to be separate RPis).
I've gone this route as well. I had good luck with certain SD cards, and earlier versions of the Pi for small things. But I've tried to use them as wireless APs, and they fold under any real use. Wireless firmware hangs requiring a full reboot to fix. Pi4s have given me a ton of trouble, and I've been on the losing side of the SD card roulette and had USB drives fail under load. There is always some kind of IO or sagging voltage problem with a power supply wigging out or I got a bad one.
Moved to using low power thin clients on liquidation for similar pricing, and it's all been a breeze. I get real disk IO. I get a real wireless card which runs an AP night and day better. I get a real USB power budget with a much more trustworthy OEM power supply. Power profile and size is about double, but it's a very worthwhile trade. I like the 1L system profiles and lean to the HPs.
I don't mind ARM in the cloud, but at home I just want real hardware.
I currently run a home server on a librem laptop where I broke the lid and can't be opened anymore without breaking it further (I tried to repair it, but since some parts broke mechanically I couldn't really fix it)
Works kinda nice and is completely silent most of the time.
I'm still running an 8th gen HP micro server, but I would run something else if I could get >=8GB memory and 4 SATA ports for relatively cheap. I haven't seen anything that gets me there yet though.
Nope, no OEM wants to make that, not enough market for it. However you can buy NVMe to 6xSATA adapter, but your case might not fit those drives,
I planned on buying NVMe extension cable, cut a slit in case, run the cable outsidde, plug in NVMe to 6xSATA adapter outside of the case and keep the SSD's outside, this would need reliable external power supply for SSD (Amazon sells them for ~$30, but I don't trust them).
I believe you will get to ~10-15w with this setup, at which rate you could build regular ITX/MATX case to hold all SSD's with i3-10100 and be at ~16w idle.
Yeah there are some nice enough ITX cases with HDD bays [0], but any comparable replacement ends up relatively expensive, so it just hasn't made sense.
It sounds like you want a Synology, at least hardware-wise. This one supports up to 6 GB of RAM, four SATA ports, and idles with low wattage and low noise.
Built-in virtualization and docker. It's $500 at Newegg.
I'm running a kubernetes cluster on a pair of raspberry pi and a 4-bay NAS as storage. What kind of lunatic would store data on a single, non-redondant, disk or an SD card ?
Something like 30MB up/down, drives are mounted directly inside kubernetes pods with NFS or iscsi depending on the application. Not noticeable, everything else is loaded from memory.
I have an SFF Dell Optiplex I bought for $100 many years ago that’s still my main “homelab” machine. It’s fairly quiet besides the two 12TB spinning hard drives inside but the fan isn’t noticing.
A rack mount server would be out of the question as I live in a small condo, both for size and noise.
When my G3 (newer version of parent's machine) is booting up all of the VMs it's running it can get kind of noisy but it settles down quickly to basically silent.
Unless it magically cleans and tidies everything I really don't see the appeal of home automation. Pressing a switch is far more convenient to me when walking into a room than fishing out a smartphone - whether it be through a raspberry pi or otherwise. Same with central heating, I rarely need to adjust it and I don't see the appeal of messing around with some integrated interface, i'd rather an old fashioned radiator dial or an independent reliable digital central heating timer I might have to tweak for 2 mins once a year.
None of these things are analogous to the TV remote control, there's a reason why we haven't seen similar precursors such as radiator switches on extension cords, we don't need to constantly adjust these things at high frequency... and when we do, we are literally walking past it (light switch).
All my lights have a physical switch or button. The smarts come in subtle ways:
1. Turn lights on 45 minutes BEFORE sunset at any time of year
2. Turn lights on and off at specific times, automatically adjust for Daylight Savings Time (I'll never have to listen to my wife complain about the lights needing adjustment again)
3. Start warming up/cooling off my house BEFORE I get home from vacation
4. Double click a button in my Family Room to put my whole house into Movie mode
5. Some of my buttons are also temperature sensors. This gives me the ability to monitor the temperature of parts of my home very closely. I've had some struggles with our boiler this winter and we also had the insulation redone in our attic. Instead of relying on anecdotes, I have actual concrete data that I can use to make decisions and gauge the effectiveness of various mitigation strategies.
6. Put hard stops on devices (such as TVs or connected network devices) (i.e. turn the TV power off at my children's bed time).
7. Monitor my garage door and notify if it's open too long or if it's been left open accidentally at night.
8. Fans!! I can turn my ceiling fans on and off based on a variety of triggers (temperature, time, etc). I hate having the fans off but my wife can't sleep with the fan on. Now the problem is solved and we don't even think about it anymore, the fan runs on a schedule.
One of my next big projects will be to automate the opening and closing of our front window blinds. They are in an inconvenient place and are difficult to get to, but I really like to have light coming in during the day but have them closed at night.
There's more but I tire writing this. I hope I made the point. None of this is revolutionary or changes any existing behavior (all the buttons/switches we had previously work same as they always did). But boy does it make some things a lot more convenient.
I’m happy it all works for you and it sounds almost like a bit of a hobby for you too which is great. But from someone less interested in it from that angle it still just sounds a lot like a bunch of work and expense for marginal levels of convenience. I also just find interiors more aesthetically pleasing and relaxing with the least amount of technology possible, maybe a side effect of working in this industry.
Maybe it's just the UK, but I struggle to find any suitable physical switches that both look like a normal light switch but also function with smart devices.
I understand the benefits of some automation, but if there isn't a 'normal' switch that works completely 'as expected' for the family to use, it's not worth the trouble.
I'm in the US so I don't know what's viable for you, but I'm using Lutron Caseta switches for my main lights/fans. I've had zero issues with them and they even work without the hub (which I only have so that I can link it to home assistant).
They aren't a traditional toggle switch, they have two buttons (on and off). Personally I find that to be an acceptable compromise and in terms of looks they are some of the best looking switches on the market smart or not. A lot of generic smart switches look like mass produced cheap junk. Lutron actually put a little bit of effort into the design which is nice.
What do you consider 'as expected'? Personally I want switches that will turn off when flicked down, and turn on when flicked up. On the other hand, 3 way switches have already broken that rule so instead, toggling a switch will always toggle the light regardless of its position.
If that toggling is okay with you, I've heard good things about Shelly which you can use to convert a regular switch to be smart. Otherwise I'm looking at GE Enbrighten switches which looks to keep the switch itself in a neutral position and have two buttons on the up and down rather than being an actual switchable switch.
It would have to be 'toggling' I think as otherwise it is not possible to gain complete automation. Consider a light that has been switched (physically) off and then turns on via automation - the switch cannot be physically turned off, as it is already in that position.
This is why the conundrum exists and as such I would need to replace all switches with toggles, or distinct on/off buttons, so the family could easily manage.
However, they also need to work as non-smart/traditional lights if the network has failed, and I suspect this is a much harder problem to solve.
Lutron Caseta switches (which I use for my mains) have two buttons. One on top for on and one on bottom for off. They also have a little dimple so you can feel which is which without looking. It gets a little more complicated if you have a fan or dimmer control but the user experience is still pretty good. Is it as quick and easy as a physical switch? No, but it's about as close as you can get without having a physical switch that actively toggles when it's triggered remotely. I don't know if anybody makes a switch like that. It would be nice but for me personally the Caseta switches are more than sufficient. I've had zero usability issues and no complaints from the wife.
100% agree on this. There are very few things I want to do on a daily basis from my phone instead of from a physical switch. One exception is controlling my bedroom lights which I do with Hue lights from my bed.
The other thing I want to automate is music to start playing when I enter the bathroom, to give me privacy in there when my girlfriend is home. And that’s the startup I’m working on: https://Loodio.com since there are no other solutions for it out there (that aren’t crap)
A well-designed home automation system will still work with manual controls and/or if the system is offline.
I have smart devices everywhere yet I can still adjust the thermostat manually, turn on lights using a normal switch, set the house alarm using the keypad etc. However, I also have a 'Night' scene in Home Assistant which sets my alarm to night mode, turns off all the lights in the house and sets the thermostat to night temperature all with a single tap and all over the local network.
^ this. You need to have a high WAF - wife acceptance factor. Another analogy is that grandma needs to be able to stay in the house and not need a computer science degree.
Your home automation system is a complimentary enhancement to a traditional, switch-on-the-wall and physical TV remote control set up.
Here is what led me to some home automation (scared of water leak damage)
- I had a home security system using some external monitoring the old way
- wanted to be able to see what’s going on when I’m away (alarm would trigger the security company to call me but not a water leak)
- bought Aqara water leak sensors, camera etc, plugged everything and currently use Apple’s HomeKit (Apple TV) with iCloud (2TB cloud with “secure video”) to get notified when I’m on my cell if something goes wrong
- this basic “infrastructure” gave me the basics for home automation where I can configure a bunch of conditions (if this, then that) and buy additional sensors/camera and just add them if I want to…
That seems like a genuine application of the technology. It's adding something uniquely useful that wasn't there before; rather than replacing something simple and reliable with something more complex, equivalently functional and marginally increasing convenience (if at all).
A functional low tech, un-networked equivalent for remote sensor notification to identify a specific event cannot exist. So fair enough.
We just built a dumb house (intentionally). The electricians put in Lutron switches everywhere.
Each one seems to contain a computer, as do some of the LED fixtures.
You can turn on the lights in the bathroom, and start your business before the lights come on. Some of the other switches simply don't work at all (motion detector in the garage? Bypassed! Auto humidity fan switches? Always the wrong policy, haven't figured out how the override button works 6 months in.)
Don't get me started on the compatibility issues between the dimmers or the LEDs, or manufactures endeavors to "minimize nuisance trips" because the new AFCI breakers hate high torque motors.
If humanity ever masters time travel, I'm going back to the 80s to buy light switches!
Yes - I bought two triple-switch wifi-connected switches and only installed one of those. I rarely use it - the only automation that I could leave on for years was "turn the living room led lighting at sunset". This one works because it doesn't make a diference if other lights are already on, etc. Its "robust".
Every other idea failed in some "corner case" I had not thought about (eg turn off all lights as midnight - until one day you have guests at midnight etc).
You would have to add more and more sensors and logic for that to be robust enough for cases like this. (Or Alexa would get smart enough to confirm with you before applying that logic.)
You're missing the point. It's home automation, not home connect everything to your phone. The whole idea is to create custom automations using network connected devices; the ability to use your phone as a switch is just a bonus IMO.
I have to ask others, have people had good luck with this?
I have tried to do this multiple times, with multiple models of Pi and every time after 6ish months it just seems to completely stop working. I can't get it to boot even off of a new sd card.
Am I doing something wrong or does the hardware really not want to be run consistently in this manner?
I have since decided to just run some home automation VM's on my media server which has worked flawlessly (minus networking being weird sometimes).
Have had the same problems, though it's always the Pi chewing up the SD card, a new card solves it. These guides really need a warning saying "If you find this useful, switch to using a server that doesn't rely on an SD card after ~6 months"
The Pi is fantastic for getting a taste of self-hosting / home automation as it's so cheap but by no means should it be relied on.
I moved our Home Assistant install to a docker container on basic PC and it's been rock solid after several SD cards were eaten by the Pi. Though I still have DNS (via PiHole) running on a Pi and that's starting to act up.
Only time I've found Pis reliable is when they either don't write much or are running a read-only disk most of the time. We've a network audio player using piCorePlayer[1] on a Pi running for years and never had an issue, it's running tiny core linux and only makes the disk r/w when you're upgrading or changing settings.
I'm tempted to move my MQTT server to a Pi using tiny core as that'd keep messaging up while I'm rebooting or doing some other task on my single server. Something like http://akeil.net/posts/mosquitto-mqtt-on-tinycore.html
I used to boot my fileserver (not a Pi) like that and it worked fine for a number of years until the USB-SATA adapter died. Not sure if you meant an actual purpose-built external SSD but I don't know that I'd trust a home-brewed solution in the long term.
Make sure you use the right adapter, I got an enclosure at first that wasn’t on the list, and it ended up not working. I switched to the recommended StarTech 2.5″ and everything worked.
I have a TV PI that seems to so far be running just fine after about 6 months. But I also have it reboot once a day so maybe that may be some magic.
I have PiHole running myself, my original plan was to use a Pi but the last thing I wanted for that thing to die and my internet stop working. An easy enough fix, but a frustrating one.
Mine is also running on the my media server. Which... is fun. When I reboot it I have to start things VM's in a certain order or things get really unhappy.
I agree that a warning like that should be in place, I am surprised though that a new card worked for you. I have a drawer of Pi's that I never managed to salvage.
Mix of things mean I haven't changed it (yet)
- As you point out having on a shared server isn't great. I do want a separate device for DNS, I don't want the network down because the 'everything server' needs a reboot.
- aand when it breaks I don't tend to be in the mood to fix it. Usually late in the evening when watching telly. It usually needs a reboot or `pihole-FTL.db` has gotten to several GB in size and needs deleting. But I'm sure it's getting closer to being unrecoverable.
I've also got it doing DHCP so every device gets a <hostname>.<network>.uk domain, plus it handles some static records as well. Means it's not just a quick swapout, need to find and migrate the custom stuff I've done to it. Most of this should be in a git repo but unsure if all of it is.
It might be the Pi wearing out the SD card. The Raspberry Pi Compute Module 4 [1] actually has a PCIe Gen 2 x1 socket.
Jeff Geerling (who's active on HN) has actually gotten SATA working [2] through the Compute Module 4 and the Compute Module 4 IO Board [3].
If you go his route you could potentially set up a Pi with more durable storage. Although, if you watch Jeff's video its a PITA getting SATA working as he had to recompile the kernel with SATA support. Also its pretty hard to get the Compute Module 4 and the IO board at the moment.
Alternatively you go PXE boot and network storage but then that assumes that you've got something else running to provide it. It's a nice solution when you've got half a dozen Pis around though (we use it in labs for our hardware test setups).
The new Home Assistant Yellow board even has a built-in M.2 M-key slot for NVMe storage; there are some other boards out there with built-in slots too (I especially like the BitPiRat I recently tested: https://pipci.jeffgeerling.com/boards_cm/mirkos-bitpirat-com...
I've got "TV Pis" that idle LibreELEC/Kodi most of the time and have been running for 4+ years on the same SD cards (updated them a couple times in that time, not recently).
My "server Pi" is >3yo old, only turns off when the power fails, and doesn't do a lot with its SD card but the OS, but has never had an issue.
What kind of power are you using? I've got these on little 2.5a wall warts; the "server" has a scrounged 20 year old PSU pulled from a Cisco 2500 router running it and some other stuff. No UPS tho and we get power outages and weirdness as usual for a rural area.
Make sure to check the USB cable you use. I have a few which has ~1 Ohm resistance, and given that the Pi can use 1-1.5A under load that translates to a 1-1.5V cable drop.
This can cause the Pi to shut down, and SD cards are not happy losing power while writing AFAIK.
I've had multiple Pi's run for years off the same SD card without issue. My Home Assistant install has a 3GB database with updates every few seconds due to some chatty Z-Wave modules. Been running just fine since 2018.
I also had a few with issues, and all of them were down to cables with too high resistance. Some were sold as charging cables, yet were rubbish.
I got a USB cable tester from AliExpress, alternatively buy some known good ones.
Not only are SD cards not happy with losing power during writing, but they are very sensitive to "brownout" undervoltage conditions. In this case writes may appear to succeed but actually fail to commit, which is obviously problematic for the integrity of your data.
(if people were using a checksumming filesystem like ZFS, this of course would be immediately apparent when it was occurring!)
Samsung sells "high endurance" SD cards - I would strongly recommend these for RPi usage, and they are also very useful for dash cams since those are always continuously writing as well. Sandisk sells high-endurance cards as well but frankly Samsung is a cut above the rest of the SD card market - my SD card failures have essentially gone away since I stopped buying other brands. I think I have had one SD card failure since then and it wasn't related to write endurance, just didn't use it for a couple years and it was dead when I tried it again.
But yes, in general, power quality is a massive problem for RPis, and people don't really consider it because it's one of those "it works 99.9% of the time" situations. It's like a race condition that you only rarely ever hit, it looks correct and people will die on the hill of "it's worked perfectly fine for months now, the power can't be a problem" and then you hit a weak flash cell when the CPU is heavily loaded and the voltage is starting to droop and it happens to be a critical file rather than just some log or a chunk in an audiovisual file somewhere, and then you notice it.
I'd say >95% of all Pi failures come down to either power problems or SD card wearout. They're certainly not otherwise flawless, it's a janky cheapass SOC in general, but that is the overwhelming cause of Pi system failures.
If you can swing it, network booting from a fileserver is a much more reliable option in the long run. I haven't really experimented with it, and performance will probably be worse, but it gets the SD card out of the equation entirely, which mitigates both of these problems. You're not writing to flash, so brownout doesn't matter in terms of the potential for failed writes, and you aren't writing to a physical SD card so there's no wear.
You're looking for a 0x0 result (not now and never under-volted nor throttled). The result is a bitfield. A script to turn them into human-readable format is here but I just look at the bits: https://gist.github.com/aallan/0b03f5dcc65756dde6045c6e96c26...
For this kind of application, my own experience and others using home assistant seems to be your much better of plugging an SSD into the pi’s USB 3 and using that. It’s also really important to use a good power supply.
Pis are horrendously unreliable. Even if you pull out all the stops to make them work reliably (like disabling SD card writes), they don't.
The broadcom chips they use are complete garbage. I was hopeful I could use the watchdog timer to make a pi more reliable (by restarting it when it inevitably crashed). Guess what? The internal watchdog timer doesn't f*cking work! You'll notice that every project using pi hardware that needs a WDT uses an external one.
Total junk! I don't use them for anything anymore, and I used to have like 6 automating various things.
I played around with Pis a lot in the OG Pi 1B era and yeah, I got tired of fixing it every couple months (I attribute this to power quality and SD card failures) and just moved to using various x86 based systems. I used surplus SFF optiplexes from surplus sales (paid as little as $5 for some), AM1-based mITX (mobo+CPU for $50!), or Atom-based NUCs (used to get "surplus" J5005 NUCs with the plastic still on for $125).
I'm looking to get back into it with some Pi4s for a few things - but I'm planning to PXE boot this time around to try and sidestep the SD card/power problems. Basically just stuff like LibreElec to free up some of my J5005 NUCs for actual stuff.
I've used the internal watchdog without any issue. I mean the one you enable from systemd with RuntimeWatchdogSec=10 (where 10 iirc is a number less than 14 which is the number of seconds where the timer overflows). Not really much to configure or customize but it works reliably.
Interesting. Perhaps they fixed the driver/hardware since I last looked at this a couple years ago. At the time, no one had got it working successfully (the special device was there; it just didn't seem to do anything).
Modern Pis have effectively zero (pun intended) relationship to the original ones. The kernel issues were fixed, and as long as you keep them cool, you can go a long way even on SD cards.
I’ve had good luck running pi’s for years between (power fail) reboots.
One thing though - I don’t think any of my pi’s run x windows, I think they’re all console mode. I get the feeling that they do fewer writes in general this way, but I don’t really know. Anecdotally, I had a pi that ran x that I was experimenting with, and after a couple of months found it locked up/dead to the network. A reboot brought it back fine, but I unplugged it a few days after that and haven’t been using it.
I ran 20 pi's at work for years and had one failure. I've run my irrigation system off a pi for 4 years without a problem. I may be the anomaly and I don't live in Death Valley.
When you have an image that you're happy with, back it up. Using a USB<->SD card reader on a linux machine, run gparted to slim the image down to the minimum, then run something like this
The only time I've bricked a Pi ever was when I shorted 5V and 3.3V rails in a 3B+. A friend always talks about one bricked with an apt upgrade by a broken firmware or something, never seen anything like that. Saw plenty of dead or corrupted sdcards but that can be fixed with a new one, which doesn't seem to be your case.
Maybe a bad power supply? Pi is a bit picky, you need one with 5.1V to prevent low voltage warnings.
You have bad luck or are doing something wrong. I've had a Pi2 running for years and a Pi4 running since they were released in 2019. I had to replace a SD card once (and now am using a SATA SSD on a USB bridge instead). Otherwise, no hardware failures.
> I can't get it to boot even off of a new sd card.
Best guess: power adapter problems. Either the power adapter itself is dead or it misbehaved, killing the Pi.
I have 2 and they've been running for years on the same SD cards. Maybe power or cooling is an issue? Mine use reputable USB-C adapters (the official Raspberry Pi one just because it's small, but alternatively I'd go for an Apple or Anker one) and cases with a little heatsink you can stick onto the processor.
I'm with the others who have been running pi's for years with no issue. I did however go back into the office awhile back and stumbled across a pi display setup that had gone into read only at some point - most likely an sd card failure.
I would think if you want to take it up a step, you could look at something like an Intel NUC.
Yeah it's better to spend $80 or so on an off lease small form factor desktop. They don't really use that much more energy than a Pi but are much faster and more stable with real SATA storage.
It's super doable, especially if you store your root in a HDD partition. Using an USB flash drive would be the same as relying on a MicroSD, not the ideal for 24/7/365 usage
It's basically an enclosure + some extra integrated hardware for zigbee (and supposedly "easy-to-add" z-wave).
The ecosystem is heavily "run it on a rasb-pi" anyway, so you're 99% of the way there just purchasing or repurposing a PI, but it wasn't that expensive to back their crowd-funding and I'm happy to support an attempt at an "official" configuration that can be purchased.
I use it slightly for its own capabilities, but mostly as a bridge to bring more devices to HomeKit compatibility.
eg: I was able to add my wifi-controllable pool pump to homeASSISTANT and it creates corresponding devices in homeKIT so I can make an iOS shortcut accessible to siri: "Hey Siri, clean the pool" which turns on the pool cleaner. It's kindof like living in the future. :-P
I also found recently a "chromecast => airplay" extension/addon which... bam... now the few chromecast devices I have show up as airplay sinks for audio (not airplay2, but airplay1 is convenient for where I've got them put).
...and I just got some Samba-mount thing so I can rip my DVD's and copy them over to `/Media/*.m4v` which I can then blast out via chromecast (nice b/c both my chromecast and home-assistant PI are wired, so it shouldn't be sucking up wifi bandwidth, and keeps me from having to set up a real ).
It _really_ needs a complete overhaul of how it's thought about (ie: configuration via yaml, confusing organization of extensions, addons, configs, better update management) ... I'm a developer and I still get lost in the mental model they expose via the UI, but it gets the job done and it's got a good ecosystem and community to help figure out how to get done what you're trying to do.
I've been looking for a Pi4 for ages to use with PiHole but sold out everywhere, some have back orders for 2023 O_O
Anyone got any recommendations of other devices that work well with PiHole? I have a Mac Mini acting as a home server but from what I've read, PiHole does not run great on it via Docker...
You don't need any in fact. It's Python so it runs on any powerful enough Linux platform, and there's a boatload of them out there; the Raspberry Pi is just the 1st one to become popular and it's the most advertised. Hoarders depleted all stocks to sell them overpriced? Well, screw them,
we're going to the competition instead.
Also worth visiting is the Linux-sunxi site, where you can find a huge load of open documentation about hardware and software for Allwinner CPUs used in some of these boards.
It's probably not worth going much past something that's also super popular like an ODroid. Otherwise you'll just be spending an inordinate amount of hours trying to figure out obscure bugs that the non-existent community hasn't yet fixed for you.
I once made the mistake of buying a Banana Pi M2 Zero. Specs were great, support looked decent, I thought I'll be getting a Pi 3 in a Zero form factor. What I instead got was a thing nobody uses on which barely anything runs properly with such flaky wifi that it's practically useless. Never again.
I tried running PiHole on a Windows box using Docker Desktop. It worked for a bit but it stopped working at some point... and I didn't bother fixing. It had been working fine for so long on my RPi.
You can buy fanless celeron mini-computers from Aliexpress for $100-$150. You can also get i5/i7 if you are willing to pay a lot more. Most of them come with pirated Windows (some sellers even say that they'll send it with Ubuntu pre-installed) but you can simply remove it and install Ubuntu Server and then install PiHole, Home Assistant or whatever you want.
Pihole will run on anything and works fine in Docker. I'm currently running it via docker on a Pi but will eventually move it to a more standard server with other services.
They only made 150k in all of 2021, so we're seeing the result of that right now. Good news is they're expecting to make 250k in Q1 2022 with 500k planned for Q2.
I approach it like this. As you go through the next week, try and see if you can identify something that you do every day, almost like a habit, and think if automation could remove that task from your life. A few of my favorite examples:
1. Checking front/backdoor is locked every single night, now I have an automation that turns on the front hall light and sends my phone a notification that the door is unlocked.
2. The "did I close the garage thoughts", just being able to check your phone from the airport or wherever you are and see "yep its closed" is "nice" but not necessary of course
3. Everytime I open the garage door, the light turns on with a timer and some logic so that I never touch that light switch
4. auto turn off air purifier in bedroom in the morning (and on at night too)
5. Anytime the doorbell rings and I am not home, I get a phone notification with a picture of who is at the door. Helpful when worried about missing packages, deliveries, etc. Doesn't necessarily "help" solve the problem, but does do something mentally
6. Recently I added one that nudges the volume on the tv up a few clicks and down a few clicks based on the HVAC turning on/off, it was something I found myself doing habitually and now that it works it is extremely seamless and has been great
I have way more complicated scenarios of course, all of which I find compelling, but I really like the automations where you dont know they are doing something necessarily, your brain just forgets they are even automations.
What's kept me from trying this stuff out is a few interrelated factors:
1) There's no possible way setting up and managing this stuff manually is going to be worth it if it only controls a couple things.
2) Doing enough to overcome point 1 seems to begin with "step 1: spend lots of money and time to replace tons of stuff that already works completely OK" and/or a bunch of research (I've used enough AirBnB IoT "actuate the existing deadbolt" add-ons to know that a bunch of them are time-wasting crap that barely works, plus I've never seen one that didn't look bad)
3) Taking a "just do it as you replace things" approach still results in spending more money (IoT will be more expensive than dumb, just about every time), plus lots of things will probably never need to be replaced while I own this house, plus that means potentially years before I hit any kind of reasonable pay-off period.
4) I have a feeling I could solve several of the problems faster and cheaper with a dumb approach of low-voltage LEDs hooked to the right things and run to the right places, or outlet timers, or whatever, if I were so inclined—which I'm clearly not, because I haven't.
Every time I get the urge, I think back to that automation effort/payoff chart from XKCD and then... don't, because I can't see how I'll ever get on the good side of the line. Doubly so if any part of it can't go years without any kind of attention or maintenance related to the IoT aspect of it.
[EDIT] The calculation would change if I enjoyed that kind of thing as a hobby, of course.
Your math checks out. For me I came to different sums for each bullet point primarily starting at the place of "I do/did enjoy setting it up as a hobby". I also really liked maturing my solutions from a this is fun to hack together, to things my partner actually has said "wow this is actually pretty nice I like it".
I have also been very impressed with some of the infrastructure/solutions that the open source community has available, it is a pretty rich infrastructure that is not unbearably-brittle if you know what you are doing, but definitely is not user friendly enough to be easy to recommend to everyone.
Thanks for the ideas - I think that's how I would have to approach it as well in that I would really want to further explore the possibilities of IoT. I've done some in the past for home energy systems but past energy I haven't seen a lot of high value opportunities. Certainly some things that could be smoother (like your projects).
Yeah, I totally get that it works out differently for different people. Maybe I'm just getting old enough that I can smell when I'm looking at something that's gonna become an excuse to spend money, more than anything else ("well now that I have an X, if I just had an Y...") But if I truly enjoyed it as a hobby, different story.
I concur. I take a Battlestar Galactica inspired approach to appliances in my house. It's fine if they have computers in them, I just don't want them to be able to communicate.
Yeah, I've installed a bunch of PIR controlled LED strips around the apartment to light up as people walk into rooms automatically, and aside from that there really isn't anything else I see I would need.
The "smart" approach would be having all of them wifi enabled, and then being able to change colour, intensity, duration, etc. via web or app. You know what also works just as well for that though? One or two push buttons on each detector lol.
The starter for me was that my solar inverter (Fronious) wanted $199 per year for more than three days history, but exposed a very readable JSON over http interface. I had low watt PC handy (HP Microserver G7) doing nothing, so installed Home Assistant for long term tracking instead. After that, it kind of just snowballed.
I can't believe they would charge that amount for data. I was was pulling solar data in the past as well - pretty straightforward. To be fair it probably took you more time to set up but now that the work is done and maintenance is probably pretty low it should pay off after a couple of years.
$199 was a bit steep. If it was a one off, may be I just would have paid it, but since it's an annual fee, it's totally worth it.
Honestly, most of the solar / power tracking, graphing etc for home-assistant now works out the box[0][1]. I literally installed a plugin, and changed a few check boxes in the config. Not saying it's trivial for anybody, but for me it was. The real time data has proven invaluable in reducing power bills by knowing when we're importing or exporting power (1:3 price ratio here), and used that to load shift the expensive power users.
Since then I've added some temperature & humidity tracking to work out which rooms are letting the heat/cold in and out, done some basic automation to know when the washing machines and dryer out in the garage have finished, and a sensor to tell me when the garage door is opened or closed. I used a combination of BLE sensors and ZWave.
I wanted to be able to install water leak sensors in the bathrooms/kitchen and get rid of the external security monitoring company by having direct access to my home (and what’s happening there when I’m away).
I started small with the basic stuff and went into more home automation thingies…
The Book "Control Your Home with Raspberry Pi" by Koen Vervloesem is excellent imho. His approach centers around mqtt and he shows how to connect ZigBee, Z-Wave, Bluetooth and 433.92 MHz devices. He is very into security, showing how all communication is secured via TLS. He also shows how to setup VPN so you can access your home automation system securely from outside.
I just implemented the foundation of my home automation.
I installed NodeRED on a raspberry pi and it communicates with arduino through MQTT.
The good thing with this is that i control the hardware parts woth arduino and the automation logic with NodeRED.
NodeRED is much easier to use than HomeAssistant because of the extensibility and the use of NodeJs.
I have implemented an arduino library that makes the development on that an easy task.
I wanted to have everything on the local network and dont depend on the internet. We had some internet outage the past years and i dont want to lose the control of my house for that. Also, everything is much faster.
Every arduino can run on its own, that way even if the raspberry pi is out of order, i can open and close the lights using a button or a switch.
I second this approach. My home automation is 98% HomeKit (via homebridge) and 2% Node-RED (for a few custom MQTT message translations and a dinky little dashboard with metrics).
I find running my home servers on USFF x86_64 systems to be much more reliable. You can find them used for the same price as a kitted out raspberry pi, and they are faster, have more ram, and more durable disks.
I like USFF because it's essentially laptop components in a mini desktop case. With power saving features turned on (currently I force all power saving features on using powertop) they don't use many watts.
I don't rely on raspberry pi for anything that needs to run 24x7 because of sd card wear and the hassle of downtime and replacing/reflashing/reconfiguring a new card causes.
> I don't rely on raspberry pi for anything that needs to run 24x7 because of sd card wear and the hassle of downtime and replacing/reflashing/reconfiguring a new card causes.
It's like that scene out of Brazil. The wall panel is removed to reveal a greasy maze of tightly-packed houseguts. A dizzying complexity of convolving and cryptic organs for... turning on the lamp.
One of the scary things (for me) when it comes to home automation and DIY smart home tech is that there's so many options that I feel totally overwhelmed. For the last few months I've wanted to do a small setup with sensors around my apartment that measure temp, humidity, air quality - and a few more for my upcoming chili pepper growing setup in the kitchen. There's so many options each with their own advantages & disadvantages - even on the air filter side there's pages of discussions about the various filters possible and how some don't collect particles below a specific size. I certainly miss the geeky/tech circle of friends I used to live near which would always have an friend (or a friend of friend) who was an expert on basically any matter!
I still have this dream that some day everyone will have a little stack of SBCs running home services from tracking energy use to running pinhole to running persistent game servers (eg, Minecraft).
Every year you pop out the oldest one and pop in a newer one and speed the whole thing up.
I have a home system and tried RaspberryPi several times but in the end I went with an older Mac Mini. Old ones are quite cheap on the used market and also significantly more powerful than the RaspberryPi, that was one of the main reason in the end I rather chose that.
I'm curious about other people's experiences with this stuff, especially if you're someone who likes simple scripting and found Home Assistant to be too clunky.
Basically, I've bought a lot of these little devices, and am much more interested in doing this in a more "Unix Way" way; give me text streams/web APIs and let me figure it out. I think there are things like Huginn and there's a like a Perl one or something?
Frankly, I'd really like to do it in Bash or similar? Anyone gone their own way like this?
(I end up not getting far because the use cases aren't compelling enough for me to put the time in? I don't know.)
I use home assistant to create my list of "entities", then I do all logic/automation in nodered, but this could easily be bash if you really wanted. This allows me to leverage the communities work on integrations for stuff (python libraries interfacing with xyz hardware/api/cloud/local device), and then put all of my automations in one place (nodered for me)
If I add new "stuff" I add it to home assistant, if I want to do things with my "stuff" I do it in nodered.
Also, if the "stuff" I want to add is a little sensor or arduino or esp8266, I can just have it send MQTT messages, and then all of a sudden without touching anything in home assistant, I have access all the way in nodered to use these messages to do anything I want.
I did. I haven't sorted out proper ventilation for my laundry room yet. Using a pi zero with a dht22 to control a dehumidifier via a tradfri power switch when needed was quick, fun, functional, and offline with no vms or GUIs needed.
Logging the data elsewhere to influxdb was a fun little addition for a few pretty graphs.
I'm a fan of nats.io but it isn't necessary. A private network via zerotier makes life easy but is also completely optional.
You can do both! I run HA for the web gui, but I have a few shell scripts for more automated things. Actually, now that I'm looking at them, they call to an HA api to make changes. So you could run HA to provide a web API for things and script around that?
In ye olden days, you could run your home automation on MisterHouse on any old computer and use a parasitically-powered DB9 serial RS-232 wireless adapter to control X10 devices.
It's very handy to get off work, grab a mug of barley coffee, sit on the couch, say "Siri, let's watch Movies" to your watch, and have the TV turn itself on, the audio routed to the AV system, and the Apple TV set as input.
All the above is done by Siri talking to the Apple TV (which is the home hub) and homebridge nudging the TV and inputs via plugins.
And _nearly_ all the above happens solely inside your LAN, except for the speech recognition (which may or may not happen on-device if you have a recent iPhone nearby).
100%. I only touch it to do updates, maybe once every three months. It’s completely hands-free once you set up all of your devices, and for HomeKit, ZigBee or Tasmania devices, I don’t have to do _anything_.
I think a Pi is great to start with. Then you graduate to a real computer when your HA gets too big and laggy for the Pi to keep up with. I use a NUC. But my brother repurposed an old laptop with good results (built in UPS, and quiet!).
Nope, nothing heavy like that. I don't know what the threshold is for every use case, but for me once I got more than maybe 50 devices, HA started to struggle on the Pi. The NUC is lightning fast, however, and I've grown my HA considerably since. Plus it's nice to be able to run ESXi on the NUC and have VMs for other things too, e.g. Grafana, etc.
Did you try changing the CPU governor settings? I've found that light workloads don't always push the governor enough to push the CPU into higher frequencies. I have mine set to 100% CPU, always, for lowest latency. See: https://github.com/home-assistant/operating-system/issues/19...
I've been wanting to create something for awhile that monitors my driveway, and when there's motion detected (through camera or IR) near my vehicle, it turns on a red light in my room, and possibly include a sound.
Am I the only one who uses Fhem? From my point of view it‘s extremely robust which increases the wife acceptance factor . I‘m running Hue, Z-Wave sensors, Shelly‘s, Homematic and an Echo connection. Sooooo much potential.
Getting into home automation beyond Google/Apple/Smart things can be a boon, you really have to devote a chunk of time to it. Running useful services at home is one of the fastest ways to learn.
I am a fanatic of embedded stuff and home automation. I love the RPi to death but am wary of wanting it to run its programs from an SD card. Any pointers to alternatives?
Boot pi from SSD. Give it pi specific ac power supply. It will be fine. It has for me for years. Or go with a used low power G3, g2, etc. As mentioned above.
From the contents page this looks like a very cool book indeed. Is there a webpage somewhere that explains what the projects are? For example, what do Photo Hook and Hue Fan do?
Anyone have success with remote cameras? Services like Arlo and Ring seem straight up awful but also rolling my own seems like more effort and headache than it's worth.
I have had success integrate Ring and Wyze cameras through Home Assistant. If you'd rather not go that route, there is a project that publishes Ring cameras over MQTT and RTSP links: https://github.com/tsightler/ring-mqtt
I have ESP32 cameras that (mostly) work with HomeKit (there are a few memory issues on the ESP32 side), and a full blown HomeKit camera that "just works".
I've been doing a bunch of home automation work recently. Here are some observations I've made:
1. Most extant home automation software is totally unreliable garbage. Buggy, slow, poorly designed. You want to reduce the amount of code you pull in as much as possible.
2. Raspberry pis are totally unreliable garbage. The hardware is just bad. There's no way to make them reliably operate for long periods of time without crashing, and there's no way to make them automatically recover from a crash (because the watchdog timer, or its driver, in the crappy broadcom chips they use is broken).
3. Most of this home automation software is optimized for users who don't know how to code at all, and is extremely poorly optimized for people who can write basic scripts or whatever.
4. Most of the radio protocols for wireless IoT devices are totally unreliable garbage.
After messing around with this on and off for a few years, the best setup I've found is:
1. Use a normal computer or server, not a raspberry pi. It doesn't need to be expensive or anything; just use a decent machine with an x86 processor. There are tons of great industrial PCs for cheap on ebay.
2. Use some kind of containerization setup for separating components. I strongly recommend using jails on FreeBSD, since you can treat jails as physically separate machines for networking/firewalling purposes (using vnets).
3. Make liberal use of VLANs to isolate IoT things from the internet and from each other.
4. Don't use any IoT devices that operate over WiFi. Ethernet is great, and dedicated wireless protocols (zigbee, z-wave, etc.) can be OK with a bunch of work.
5. If you want non-trivial automations, don't try to use some existing "user-friendly" or "low-code" or whatever home automation software. Just write a normal computer program that does the automations.
The specific stack I have for most of my automations in my current house is zigbee2mqtt running in one jail (this is by far the most problematic component), mosquitto running in another, and then my program which does the automations running in a third. My program can also do things like send me push messages via pushover.
I've managed to get a seemingly stable zigbee network with a not-totally-trivial number of devices (around 25 now) by using CC2652 radios and plenty of zigbee routers, and also turning off all lightbulbs before adding more devices to the network.
It's a shit show, and I can't really recommend getting into it right now unless you want to spend a bunch of time doing sysadmin tasks, or you only want very simple automations (in which case maybe use HomeKit or something).
I would like to challenge that assertion that Pis are garbage. If you use a recent hardware revision you should not have any issues, mine have outlasted the ODROID I had running homebridge for a few years off an EMMC.
Also, I git push my automation services to a server using https://github.com/piku, which isolates them in (simple) sandboxes. Zero issues for a long time now, and with Docker pull restrictions coming up soon, I can certainly put up with npm taking a while to update Node-RED...
I'm sorry but this is just BS. Raspberry Pi are not garbage.
With good software, and by that I don't mean barely tested Java or Nodejs HomeKit wannabe services, Pis are rock solid.
I'm into SDR (software defined radio.) I have written software which processes 8MB of samples per second, and runs for months without reboot. It means 90-100% CPU load, 100% of the time.
Oh, and the same Pi also hosts AirPlay server and manages ~10 Arduino devices in spare cycles.
The hardware is crap. The broadcom chips they use are cheap pieces of crap. The watchdog timer doesn't work. They have a tendency to corrupt microsd cards. Power filtering is bad. Remember this? https://www.theverge.com/2015/2/10/8010457/raspberry-pi-2-cr... There have been any number of issues, over and over again. Maybe they've improved some in the last couple of years but my time is too valuable to continue to waste dealing with these pieces of junk.
I agree with all of your recommendations. I use Home Assistant running in a VM on a normal machine as a sensor fusion platform and for dashboards. Every non-trivial thing is then an AppDaemon[1] app running in a separate VM on the same machine. I have zwave devices rather than zignee so I use zwavejs2mqtt on raspberry pi's in various locations around the house (soon to be replaced with tiny Atom machines).
It expresses everything from every integration as an entity in various device classes so as long as some random tech is supported by an integration you can group them seamlessly. For example - I have Z-wave based motion detectors but I can control Hue lights and Wemo switches for motion detection - but only if I'm home as reported by the zone feature in their iOS companion application.
Their supervisor/docker based install for Raspberry Pi is pretty slick[4]. Write an image to disk as you usually would and you get a bare-bones OS with Home Assistant running in docker. Then there are additional docker-based add-ons you can install for stuff like MQTT, Z-Wave, Let's Encrypt, SSH access, nginx, and more. All managed in the web ui. Pretty cool.
[1] https://www.home-assistant.io/
[2] https://www.home-assistant.io/integrations/
[3] https://hacs.xyz/
[4] https://www.home-assistant.io/installation/raspberrypi