>I recently took some time off of work to spend time with family. During that time I worked on a few side projects like getting our backups in order and getting a Raspberry Pi 3 running Pi Hole.
I know what the original intent is, but still- I found this quite humorous.
Still, this seems like a very nice noob-friendly guide, and that's a very good thing when VPNs are concerned.
The funny thins is that I’ve been thinking something similar during Christmas. I now know that I’m not the only one thinking about taking days off work to sort out my backups and network :-)
awesome! glad you enjoyed it :) trying to write up the things I find super helpful (code-related or not) and share them. especially when I can't find any good docs on how to get something working lol
I run a pi-hole equivalent on a Raspberry Pi at home (dnsmasq with pi-hole's block lists). One word of warning, its list tends toward over-zealous, blocking sites / links you might want (e.g. Google Shopping links). You can add manual domain exceptions but it's tedious and takes a long time to restart dnsmasq due to the enormous list size and the RPI's relatively slow performance.
Blocker extensions are nice becuase you can selectively disable them or open an incognito tab in cases when you need to bypass the list. Easier than temporarily changing your DNS server to 1.1.1.1, etc.
Still, nice to have automatic adblocking for all devices in the house.
I had the same problem with overzealous blocking. I had to disable temporarily about once per day, and my wife found it bothersome also (especially because she didn't know how to manually disable it). We turned it off last month and I've just been using Brave instead. Works great on my MBP (fan doesn't run nearly as much as with FF & Chrome), though of course our mobile devices aren't protected anymore.
If anyone has a good solution for the overprotectiveness of the Pi-Hole, I'd love to hear it!
I have no idea about how Pi Hole internals work being myself happy with Ublock Origin and other browsers extensions, but anyway a database storing the danger level based on IP nature (plain malicious, NSFW, advertising, etc.) and associated risks into multiple fields could make things easier for the user who for example could want to surf gambling or porn sites while keeping the guard high against phishing, advertising etc. A series of switch would be more immediate to use than editing a long list.
You could just disable the pi-hole dns server in your routers DHCP server (so it will not be used by all network devices by default.)
Then, only on your own phone/laptop/etc, manually configure your pi-hole as the DNS server for the corresponding home Wifi network. So only the devices you manually configure will use the pi-hole.
Isn't the overprotectiveness of Pi-Hole in relation to the blocklists not Pi-Hole itself. I'd suggest findings different blocklists.
There is one particular website that is extremely popular, I can't remember what it is now but it has blocklists showing which are likely to cause false positives.
I run pinhole on a Debian Vm on my FreeNas Mini so the perf is not s problem. Could also run fine on any 2008ish era computer you May have faster than the Raspberry Pi if that’s the concern.
Ideally, one would hope a Pi-Hole would just block Google altogether. ;) Iunno why you'd want Google Shopping links to work.
Jokes aside, it'd be nice if there were easier ways to mark certain groups of ads as okay. For instance, if I'm paying for a streaming service at the "comes with ads" tier, I'd actually prefer to allow those ads, without having to figure out which ad domains Hulu is using.
The biggest reason to go for the network level block though is convenience across many devices, as you said, as well as blocking tracking and ad domains that aren't running in your web browser: Apps themselves now come with ads that tend to be a lot harder to block on the device.
The problem is not the links you click, it's the links that load nasty stuff during page load.
This is why a rewrite proxy is better than a simple DNS blackholer. Unfortunately a side effect of https ubiquity is that rewrite proxies are much more complex to deploy, even on your own local machines.
I do. I use a hosts blocklist that blocks some inline trackers, like affiliate redirects you find on SlickDeals. I accidentally click those frequently so I want them blocked.
Pihole is great! Great revelation was that over 50% of the requests generated by devices in my network are "malicious", ad or tracking related.
The greatest offenders were phone apps and roku TV. Also roku tv scans your HDMI data stream (if you connect your laptop to the Roku TV) which is purely evil!
Also worth to note that to have pi-hole to recognize and block over 50% of the requests, constant gravity updates are required (i have over 3.5M unique domains in block-list)
Also get VPN for your phone. All traffic on my iPhone goes though pi-hole (check OpenVPN and DNS Override iPhone apps in store)
"When you use a Roku TV with the Smart TV experience enabled, we also receive information about what you watch via the Roku’s TV’s antenna, and devices connected to your Roku TV, including cable and satellite set top boxes. Using technology such as Automatic Content Recognition (ACR) technology, we receive information such as the programs, commercials, and channels you view, the date, time and duration of the viewing, and how you use the on-screen TV guide. When you use a Roku TV with the Smart TV experience enabled, we use this information to personalize your TV viewing experience and advertisements. Please visit Settings > Privacy > Smart TV experience on your Roku TV to learn how you can enable or disable the Smart TV experience. To understand your choices, see Part IV, Section F below."
interesting, i haven't heard about dnscloak. On iPhone i had issues with DNS overrides, since there's no such option in settings. DNS override option only available for app devs which is why i needed and additional app for iPhone DNS settings besides OpenVPN.
yes, dnscloak acts as a local vpn that forwards dns traffic only, i believe. however i also just remembered it was removed from the app store for some technicality recently (apparently vpn apps must have a company as the dev and not an individual? i assume to crack down on shady vpns). hopefully it will be restored soon
1) pi-hole
2) steve blacks hosts file
3) ublock origin
3 = most conservative filtering configuration that can easily be tweaked from the browser
2 = use modules (-e gambling -e porn etc)
1 = most basic blocking configuration
this way you don't have to do much fidgeting on the router. this comes at a tradeoff for putting some of the burden on the hosts #2 & #3 but with the advantage of better usability for non-tech users.
protip: if you can live with not accessing unicode domains at all (counter measure to avoid domain squatters and some phishermen) patch[1] your dnsmasq and add this in dnsmasq.conf:
EDIT: for mobile I used to have a VPS running openvpn. Make Android connect to the vpn by default routing all traffic through it. Run something like opensnitch[1] to MiTM and whitelist the mobile traffic and sinkhole shit you want to get rid off. This isn't for the fainthearted since new versions of apps might make different API calls and break your rules. Apps will just stop working. If you only have 2 or 3 apps and want to kill traffic from built-in carrier spyware it works nicely and is well worth the effort. Nice way to study what your device does. https://github.com/evilsocket/opensnitch
I agree this is the best setup. Locally using uBlock Origin and Steve Blacks host file is all you really need for a computer but doesn't protect devices which aren't as configurable (eg Roku, gaming systems, phones). By using all three you maximize your protection on your local network and also on your computer when outside that network.
For those that prefer algo, there is an open issue to add Pi-Hole (https://github.com/trailofbits/algo/issues/1258) and in my fork I've added Pi-Hole support: https://github.com/dan-v/algo. It's a really nice setup especially on mobile devices where you typically have to choose between using a VPN or using adblock as they typically require a local VPN hack to function.
Algo user here -- it's really great for ISP-oriented privacy protection. I have a $5/month droplet running Algo in DigitalOcean, and an auto-on mobile config on my iPhone that uses it when I'm not on WiFi.
I also set up my home pfSense router to route all traffic through it (via an IPSec/IKEv2 tunnel). This works great, except when we're using Netflix or Amazon video -- I have to turn it off since those services block DigitalOcean IPs. Still haven't figured out how to configure the pfSense tunnel to route requests to those services through my ISP instead.
I'm using a similar setup. I have my ISP's router in bridge modus, an ERLite‑3 router by Ubiquiti, and a NAS server by Synology. It could've been a router by PC Engines, a Router7, an unRAID, or a Pi too. Ever since I got the Synology NAS (which can run Docker) my Pi is gathering dust.
The NAS runs Docker and uses PiHole(dnsmasq)/Unbound to take the main traffic. The ERLite‑3 router is second choice and uses PiHole/Unbound as well. The setup provides redundancy. All DNS traffic not generated by the router and not coming from the NAS is redirected to the NAS. Both PiHoles have each other as redundant server as well. Both utilize Quad9's DNSSEC servers with DNS over TLS [1]. Though I suppose ideally you want DNSCrypt [2] this should also be possible with Quad9. The advantage this is going to work on any client. Since the ERLite‑3 allows WireGuard (and only that; no OpenVPN or OpenSSH or anything) clients such as my smartphone or laptop have secure and ad-free internet. Since I'm also using Quad9's blocklist, things such as porn are also blocked. The only caveat is that these clients have access to my LAN. While that's intentional, it increases the attack surface of my devices. I shouldn't trust my wife's devices on the LAN remotely. Then again, all internal LAN services are behind strong passwords.
Do you lose any bandwidth in bridge mode? I’m about buy a new Ubiquiti Edge router and a few APs.
With AT&T I can’t do a true bridge mode or use my own modem. Their gateway has to do the authentication to the network and I’m stuck with it if I want gigabit fiber. People on dslreports lost a good bit of bandwidth on a gigbit connection writing rules on the edge router to authenticate back through the gateway.
Author of that guide chiming in! None; Google Cloud Services is not a consumer offering. The monetization approach is different from the model wherein you give up privacy in exchange for a service. More info at: https://cloud.google.com/security/
For those running pfSense: pfBlockerNG is a great package. It allows for IP and DNS based block lists. Between that and the built in OpenVPN server, I have ad (and other malicious traffic) blocking even while remote. Pretty crazy how much it catches on both levels.
Yeah, this is a big problem. I set up a Streisand instance last year just to play around with, and while the set-up was pleasantly simple and the results were great as far as the VPN functionality, it shut down Netflix entirely for me. Definitely a deal-breaker.
interesting! Haven't had any issues w/ netflix yet; wondering if it's an eventually-consistent thing or a static config on their part (i.e. will I eventually get blocked? not sure lol)
From the stuff I was reading/folks I was chatting with when I ran into the problem, it sounds like restarting your instance could sometimes fix it - I assume Netflix is basically playing whack-a-mole with AWS-based scrapers.
One of the more annoying things, while traveling is when I forget to turn off my (Linode) VPN and try going to Delta's website https://imgur.com/a/5lllN6J
I just setup the Pi Hole yesterday on the raspberry pi I had sitting on my shelf and I must say it is a god send.
The bonus is my girlfriend noticed the coloring app on iOS she uses doesn't have banner ads and after she's done with a picture, there is no longer popup advertisements.
I only had to make one whitelist so far and it was graph.instagram.com otherwise the app completely doesn't work.
I have basically the same implementation except it's running on my dedicated openbsd router using openvpn/unbound/pf and scripts. It doesn't support wireguard though but openvpn should be equivalent? I know the wireguard developers claim it's more secure, however I think a properly secured openvpn with certificate based authentication both ways should be fine for a home setup.
For the lazy, someone other than me made scripts to both pi-hole (dns) setup[1] and pf (firewall) setup[2] to block on network level too.
The scripts themselves are very easy to read and vet for bad stuff.
The main selling point of wireguard is elimination of insecure ciphers and less chance of shooting yourself in the foot with configuration. the protocol is more lightweight too. openbsd has a port btw:
Yeah I know it's supposed to be more efficient too.
But thanks for the link to the openbsd port, I'll definitively look into that! For some reason I thought it was only implemented as a linux kernel module.
What do the failure cases look like with Pi Hole? E.g. broken websites, missing (legitimate) content, apps that crash when they can reach their ad server?
And what kind of workarounds/remediations are available to deal with this, ideally in a user-friendly way for non-technical users?
It really comes down to how many sites you block. It comes with some default blocklists, but wally3k hosts a site tells you the status of different blocklists: https://firebog.net/
As for what is seen when a site fails - well, it could vary. If a JS file fails to load, the site will either look bad or not at all. ANd yeah, some apps may have issues, but pihole comes wiht a built in query log so you can see which devices just made which DNS queries, along with an easy-to-click whitelist button.
There are different blocking modes you can choose from that return different response codes, so you can choose what works best for your environment: https://docs.pi-hole.net/ftldns/blockingmode/
I would recommend checking out the pihole discourse(https://discourse.pi-hole.net/) or subreddit. The community is very active
The best way around this is to pick the right block lists - I can count on one hand the number of times it's blocked legitimate content (and I run rather a long list of block lists). wally3k's list is good for this - https://firebog.net/ - providing you still stick to the ticked lists, you are unlikely to have a problem.
For working around it when it does happen, the UI is easy to use - go to http://pi.hole, login, and then you can disable it for 10s/30s/5m, or whitelist it if it is something you are going to use a lot.
Yes. It's often pretty easy to look at the query list and just whitelist some obvious domains though, rather than letting everything through temporarily.
I know it's somewhat irrelevant, but, I would like to note that I've never had this happen. I am running an updated raspberry pi 3 b+ and pi-hole + dnscrypt. It blocks something like 30-50% of my requests daily (top domains are windows and my 2 roku tv's). It allows me to visit sites that detect ublock but also block ads.
It depends on how closely your definition of legitimate matches the block list you’re using, which is why it’s important to choose the right one. My wife and I occasionally disagree about whether this or that link should be blocked but otherwise I’ve had no problem with legit sites working.
I'm running pihole in a Docker container on an Intel NUC, performance is great. Client (browser) performance has also improved remarkably. One problem is getting ipv6 working between host, container and the outside world is non-trivial and non-obvious compared to ipv4 - I still haven't figured it out.
Typically you go into your your router's DHCP settings and populate DNS with the pihole DNS, or disable DHCP on your router and let pihole do DHCP. The gotcha is if you have an ISP supplied router which has no DHCP interface at all: no way to disable it, no way to customize DNS. All of Xfinity's hardware now does this for residential, you have to pay for business service to set DNS servers.
> The gotcha is if you have an ISP supplied router which has no DHCP interface at all: no way to disable it, no way to customize DNS.
The way I get around this is by putting all of my devices behind another router, and treating the ISP router as if it is part of an external network. It introduces a second layer of NAT (ISP router gives my router a 192.168.0.x address, which it treats as its WAN IP; devices get a 192.168.1.x address), but in practice it's caused me no problems. I've been running a similar setup for close to a decade now.
I had this same problem with Xfinity and just switched to using my own modem & router. Switching to your own hardware should save you $10/month anyway unless you're on some kind of building-wide plan.
For me it was worth it to spend the money and have full control over my hardware.
Looks like we're running Streisand as a VPN in AWS, and pihole at home on a Raspberry Pi. Why not just run PiHole on the AWS instance alongside streisand and save on the added complexity?
I think he actually had pi-hole running on his AWS instance as well. He mentioned using the "connect" button in the AWS console during the pi-hole setup portion.
ah sorry if that's not clear; I started by running Pi Hole on my RBP, then added it to a VPN setup. Nice to have it running at home to block our other devices though
Wouldn't you need some sort of intercepting MITM HTTPS proxy to be able to inspect the traffic for ad content? Thought Google was using https and legit domains to drive ads instead of just doing lookups to youtube-ads.google.com etc.
Presumably you'd need a way to inject a cert into the Apple TVs trust store as well, and I'm unclear if that is possible. Perhaps with a developer license?
This is network level, and it's taking the name lookups, which are done before picking the connecting endpoints, so SSL doesn't matter. It's not rewriting the stream, it's just saying, someone is trying to get an endpoint at this particular name and I am going to give it a different IP than the "world" would. Nothing of that is happening inside the https link. Eg, you get https://whatever.com/whateverelse... it will see the DNS lookup for whatever.com regardless of procotol.
I believe rhexs's point is that it's using the same domains for video data and ad data - i.e. you're seeing hxxps://youtube.com/video_data, hxxps://youtube.com/ad_data, hxxps://youtube.com/video_data..., so filtering at the DNS level doesn't work.
> I even started noticing some network calls our ISP appears to be making
I'm curious what this means. AFAIK the only way your ISP could make network requests that appear in Pi Hole is if they're running some kind of software within your network.
I think the OP means modem plus router combo that most ISPs offer. The modem is most definitely patched OTA, has been true for most modems since DOCSIS 3.0 was introduced.
Yeah I get it. I always run my own router behind the modem provided by the ISP, if any, so there's no way my ISP would be able to make requests that Pi Hole would see.
With all the people running piholes at home, could a subscription pihole service work? A small vps could run with authentication of some sort (mac-whitelisting seems simple and easy to implement). Allow customization of blocklists per MAC.
That just sounds like AdGuard. Plenty of consumer routers offer VPN server capability for free. That combined with DuckDNS or a similar free service would be a better option I think
Haha, I don't really know but the name cracks me up. You should take a shot at it and the report back any problems you hit so they can improve it for n00bs.
I know what the original intent is, but still- I found this quite humorous.
Still, this seems like a very nice noob-friendly guide, and that's a very good thing when VPNs are concerned.