To me this looks that for ~90% (eyballed) you just need to tell your browser somehow to stay away from any port other than 80 or 443. If some script/link-rel/src attrib points to a non-http(s) port just pop a warning to confirm you're ok with it. Is there a browser today with this feature?
Or go wild to neuter your browser and configure your firewall to allow only dst ports 80 and 443 for mozilla/chrome/edge/etc.
Also curious what are the performance implications for adding to your firewall
about 2 million IPs or maybe a couple 100k if you're brave to do ranges.
This feels like the wrong conclusion in an adversarial game. Such a heuristic works only because it's not done at any reasonable rate. As soon as it gets applied nontrivially you'll see scammers adapt back to ports 80/443. And you've got a broken browser to boot.
I agree that is the likely outcome. It is interesting to me though that they're not concentrated on ports 80/443. There must be a reason why and one answer, from a comment below, could be that boxes hosting this already serve legit http content on those ports. Having this kind of traffic show up in http monitoring tools would make the hack obvious.
A lot of stuff is hosted on compromised sites or devices - which are often running some insecure admin interface on an odd port. Very rare in my experience that someone would spin up a new web server on a box they’ve popped.
I recently got a pop up in Firefox doing exactly that. I was messing around with my homelab and entered a URL/Port that Firefox deemed suspicious and warned me that “this port is usually not used for web browsing. Are you sure you want to visit that?”
I'm hearing more and more nice things about firefox lately. Been on chrome for the last 10 years or so but I might have to switch soon if they break extensions the way the plan to.
Where do you get those 2 million IPs? The plaintext url list [0] only contains 90k entries and after filtering it to ips only and de duplication it's just 39k.
I've just added it to my firewall that does around 160Mbit/s right now using an ipset and the only increase in CPU I can see is a small blip from the ipset restore. And that's just an APU2 with a AMD GX-412TC (1GHz Quad core from 2014) and not a beefy box.
yea, it should not have any performance implications.
be aware that blocking stuff in your infrastructure will have hard to diagnose fallout and you're generally better of if you police content on the client (ad-blocker)
If you're running a stateful firewall, those generally don't evaluate firewall rules for established states, and most of your traffic is to established states, so no big deal.
If you're not running a stateful firewall, it's not totally unreasonable to skip the firewall for tcp packets with ACK and not SYN, so again no big deal on those. But http/3 is udp, so no shortcuts there.
Afaik, most firewalls have a lookup table available, you'd want to use that, rather than 2 million rules. On FreeBSD, ipfw and pf have lookup tables, ipf calls them pools, but it looks like the same thing. A lookup table for IP addresses is pretty fast, even with 2M entries.
> Also curious what are the performance implications for adding to your firewall about 2 million IPs or maybe a couple 100k if you're brave to do ranges.
OpenBSD's pf firewall supports tables of IP address which can be black or whitelisted. From their FAQ:
> A table is ideal for holding a large group of addresses as the lookup time on a table holding 50,000 addresses is only slightly more than for one holding 50 addresses.
> Also curious what are the performance implications for adding to your firewall about 2 million IPs or maybe a couple 100k if you're brave to do ranges.
The problem here is, that the "bad guys" move around, and sooner or later you ban most of the eg. digitaloceans IPs, amazon IPs, azures IPs, etc., and you break conectivity for other, "good" uses.
This is a big part of it. There are companies that scan the internet in search of malware, and if you run on a different port it makes the search space so much bigger.
In my day-to-day work, we analyze millions of files every day, and it's well-known and well-utilized detection evasion techniques to host and serve malware from "trusted" websites. It's so widespread that I did extensive research on that issue. There are well-known apps with $Ms in funding and revenue with a plethora of malware hosted on their servers. Some are even used as C2 servers for data exfiltration. I see an increasing number of companies proactively blocking all traffic to those notorious sites to increase overall network security.
The outcome of my research was the following:
- Disjointed content moderation and cybersecurity departments: Not many companies have content moderation teams equipped to perform malware analysis or make cybersecurity-related decisions (the only company that does an exceptional job in this regard is Meta).
- If hosting malware doesn't impact the company's revenue and reputation, the content moderation team has other priorities.
- Section 230: Companies will refer to Section 230 when asked about hosting malicious content or scanning the content for potential malware.
I see a few false positives. It appears that unsigned software is being labeled as malware, and as grayware on some pages.
Unsigned software is not malware or 'grayware'. It's not inherently malicious.
I'm also seeing coin miners being labeled as malware. They often are, but I'm sure there are misclassificatons along those lines as well in this dataset.
Sometimes popular domains like drive.google.com get added, and at other times some domains are just that popular to reach a TRANCO rank of 20,000, so I generally advice against blocking using sources like URLhaus.
They seem to be partners of spamhaus. From the recent feed [0] it looks like it's often just IP addresses so you would need to add it to your firewall.
abuse.ch is a non-profit, initially private. Working on cyber security issues for 15 years. Mainly focused on botnets and malware. Since 2021, abuse.ch is under the Institute for Cybersecurity and Engineering ICE at Bern University of Applied Sciences. To date the project has been funded entirely from private-sector donations.
They have mainly two goals:
1 Research: Research into malware and botnets
2 Open source threat intelligence: indicator of compromise – IOC for the public to prevent threats
Or go wild to neuter your browser and configure your firewall to allow only dst ports 80 and 443 for mozilla/chrome/edge/etc.
Also curious what are the performance implications for adding to your firewall about 2 million IPs or maybe a couple 100k if you're brave to do ranges.