HOSTS is useful but limited. For example, it does not allow for wildcards like DNS.
Unbound is included in many distributions nowadays and it has plenty of features now that can make it act like a HOSTS file or authoritative server. These work well for ad blocking.
Blocking ads is like blocking traffic using a firewall. Firewall rulesets often block everything by default and then lines are added to whitelist desired traffic. This can be easier to manage than allowing every domain by default and trying to come up with a list of all undesired domains. The same firewall-like approach has worked well for me in blocking ads. All domains blocked by default; desired domains are whitelisted.
If you use Chrome browser, it will even help you formulate your whitelist. Go to chrome://site-engagement after some routine browsing.
You might find there are some shocking entries in those massive blocking HOSTS files popular on the internet if you ever choose to read one. Sites you will never, ever visit in your lifetime online. Grossly inefficient.
It also appears sections have been cut and pasted from a variety of disparate sources without any sort of verification.
I tried to read through one of these massive HOSTS files once and had to stop as I found it too repulsive. There were far too many dark corners of the web listed that the average web user will never visit. Makes one wonder how the authors even know about these domains.
People's browsing habits are not all the same. A "one-size fits all" HOSTS file seems inappropriate.
Occasionally something like a download link, where the webmaster has chosen to use cloudfront for that specific resource, might require that I whitelist a cloudfront domain temporarily. If the domain has a unique subdomain and I am confident no ads are ever served from that subdomain, I might whitelist it permanently.
Every user is different and visits different websites. Each user's needs are to some extent unique. I think you have to find what works for you. No one can do this for you.
The more engaged you become in blocking ads, when you stop relying 100% on a third party to try take care of it for you, I think the more familiar you become in exactly what domains you need to access to accomplish whatever it is you are doing on the web. That knowledge allows you to make yoiur whitelist.
Meanwhile anyone using Chrome can tap into the built-in diagnostics via chrome://chrome-urls to get a very quick and easy analysis of what domains they are requesting and the ones they actually need:
Personally I like to know if websites are changing their IP address. I think there can be good and bad reasons for changing IP address. When one is using whitelisting instead of unrestricted recursive queries to a DNS cache then it becomes easy to identify websites that are changing IP address and to monitor the changes.
Thanks for sharing!
"This repository consolidates several reputable hosts files, and merges them into a unified hosts file with duplicates removed. A variety of tailored hosts files are provided."
I wonder if you could circumvent the hosts method by rotating through unique subdomains as your ads server. My understanding is that you can't wildcard the hosts file.
On MacOS, try this in a terminal window to flush your cache.
sudo dscacheutil -flushcache;sudo killall -HUP mDNSResponder
They come up and I spend a few seconds deciding if it's important to me to read what is behind it, and 95% of the time that answer is "no". Saves me a TON of time. :-)
I presume that the 421 cookies are tracking something, only a hundred or so go to the Liverpool Echo, the others go to 20 or so other places. Nonetheless there are not many people reading local papers online, it is too much effort wading through the junk that gets downloaded. 6 megabytes to display 15 sentences and a video embed is a bit much.
In the olden days the newspapers were read by many people. Nowadays the newspaper readers are 'read' by many people. It has gone back to front.
How often does anyone here see a link to a newspaper and think to jump straight to the comments in order to see if the article is worth reading? For me this does not happen if the link is to a blog or other site likely to be sensible with the inline spam.
The sooner this ad-spam business dies off the better.
Oddly they put every store on Twitter. And email me about it. God knows where they get the money from.
I do weep for the lack of coverage of local democracy though. Where journalism dies, political manipulation and blatant lies run rife. All we have left is private eye to cover the most egregious cases
Perhaps you listened to the same Hidden Brain episode?
Starving The Watchdog: Who Foots The Bill When Newspapers Disappear?
they have not yet. the interenet/computers does give a lot more control to the user though. ads are not going anywhere though unfortunately.
It's realy neat to get autoprotection for all your devices at the same time with the Pi-hole.
Just ad uBlock to the browser to remove the rest ads and get a much smother web experience without distractions :-)
+1 for pihole; rPis / odroids / SBCs / NUCs / home servers are easy enough to run that it's worth it.
I would have just made it an image with the text, from a random url.
The ability to track and monitor internet users is very powerful and lucrative. They won't give it up so easily.
That's apparently a windows-centric statement. In Linux,
For example, you might be in for a nasty surprise if you assume that "nc -l 0.0.0.0 1234" is equivalent to "nc -l 127.0.0.1 1234".
It does it for :: as well...
When you are setting up a socket to listen for connections on a particular port you would specify 0.0.0.0 so then things can connect from anywhere like localhost or on any of the many possible IP addresses assigned to the machine, or you can specify a particular IP address and only be able to get traffic from that. For example if you wanted a program only reachable from the same machine you could listen on localhost (127.0.0.1) and then nothing external could directly connect to that particular service.
The bigger issue comes from the likes of Google/YouTube/Facebook who host their ads on the same domain as their main website, ergo, if you want to block the ad domains, you'll be blocked the main domains as a whole. In this case, the only way to block ads is through an in-browser addon.
A PiHole could do wildcard blocking for the subdomain - but as in the ticket where the content for the site is also served from the same encrypted subdomains - nothing can be done. uBlock origin filters also fail at blocking these requests. After some research, I found a potential solution is to block off of request headers, since the ad tool is using headers as a way to send data. Unfortunately I'm unaware of any browser based tool that is able to block requests based on header content.
Its very interesting that this encrypted subdomain tool is only enabled in chrome and not Firefox. It will also detect if the developer tools are open or not. WebMD is a good example where this tool is being used.
I am using the Unified hosts file (mentioned in the article), it is a great way to combine many other hosts including Dan Pollock's list.
It would be possible to make one like that by hosting your content and your ads on the same domain, that would trip up naive hostfile blockers, but of course if companies were doing this quite a lot of people who habitually block ads wouldn't mind them doing so, since one of the key complaints against ads is data harvesting by third party ad providers.
But like I said, the back button is effortless and if your content is not rare, I'm going elsewhere.
DNS over HTTPs isn't just a Chrome issue; firefox are the ones who are actually shoving it down your throat.
I know most users here on HN are firefox users, but come on... It's an issue with all browsers, not individual.
Privoxy can disable host requests, but for HTTPS traffic will no longer disable specific page elements.
Admittedly, I do occasionally have to turn the adblocker off to get a site to work, but this is maybe once a month.
This is the reason I haven't installed Pi-hole. I understand that a broken site may be because of the adblocker and can turn it off in my browser, but a less tech savvy user may not know this. And if they are on a Pi-hole network they won't know or be able to turn it off (I understand there is a whitelist but I believe this is only configured by an admin - could be wrong here).
I'd probably happily pay for a commercial VPN which had similar and better functionality.
Does anybody have experience in this regard?
What about a basic version with ~100 entries?
I wonder if there'd be something that allowed you to allow ads on the current page and just removed it from the hsots file.
If you block using a hosts file, the ads will be requested from an IP address, thus skipping a DNS lookup.
If certain IP addresses start getting blocked, they'll move to IPv6 and have an infinite dynamic supply, which are randomly picked as the web page is served.
It is an arms race.
Also: advertising ruins every medium it ever touches. There is no self policing or sense of restraint or any line that could be crossed leading to a feeling of shame.
I'd advocate for encouraging adoption of more flexible browsers, if it comes to that. Firefox is far from perfect (see Looking Glass), but for less tech-savvy users it's probably the best option (and you'll get better results than a hosts file or DNS server since the ad blocker can actually fix the page layout and modify elements, too!).
The annoying thing is that it blocks direct links on deal sites etc.
I already use Firefox + ublock origin and it is enough for my browsing.
However I am looking for something to block ads also on apps.
It blocks ads in news and other apps for me.
DNS66 has been working great for me but for some reason it misses a few ads unlike Blokada which never missed a single ad for me.
In my case: 10.14.2. Mojave