That particular domain is sluggish from the UK, but other domains, but my route53 hosted domains - including ones never before used (wildcard subdomain) - are all fine - around 5ms.
I think the idea is that you take a list of all domains and then count the number of DNS lookups that are done for each over the course of some time period (e.g. 1 year). Then sort them from high to low number of lookups. At the start of the list you'll probably find only a few domains with many billions of lookups. As you go down the list you'll find many domains with very few lookups, the "long tail".
It's a bit confusing because normally "long tail" refers to a histogram or probability distribution where a large portion of the population is far from the central part. I don't think that works in this case, unless I'm confused about what to put on the x-axis. (Because if the x-axis is lookup frequency, the domains being referred to here would be in a peak close to 0, not in the tail.)
edit: maybe the x-axis of a histogram could be "mean time between lookups" instead of frequency? That would put the popular domains in the peak near 0, and the unpopular ones farther out in the "tail".
I think in this case about DNS caching, "long tail" is better than "infrequent". In the wikipedia graph, some of the domain lookups in yellow may be "frequent" (absolute sense) but simultaneously but much less popular (long tail) such that they don't stay in DNS lookup caches.
DNS is essentially a cache. I've never once in my life heard of infrequently accessed cache items as "long-tail". This is definitely a dumb phrase that should be avoided.
You also used the word "domains" when you asked about "infrequently used domains" and that's the context I was responding to. I didn't say that cache items are labeled "long tail".
I’d urge everyone to run a dns bench tool at home. Cloudflare isn’t always the right choice and for some ISPs with routing issues it can sometimes be a bad choice.
Here are results for my custom edited list of domains (first three are popular domains, rest are "long-tail" domains):
test1 test2 test3 test4 test5 test6 test7 test8 test9 Average
2001:558:feed::1 18 ms 18 ms 16 ms 30 ms 202 ms 377 ms 90 ms 87 ms 485 ms 147.00
2001:558:feed::2 47 ms 31 ms 32 ms 154 ms 436 ms 343 ms 102 ms 76 ms 254 ms 163.88
75.75.75.75 20 ms 16 ms 17 ms 78 ms 191 ms 293 ms 68 ms 75 ms 203 ms 106.77
75.75.76.76 35 ms 33 ms 34 ms 149 ms 437 ms 283 ms 123 ms 102 ms 464 ms 184.44
cloudflare 17 ms 19 ms 19 ms 103 ms 1135 ms 427 ms 69 ms 293 ms 191 ms 252.55
level3 18 ms 17 ms 17 ms 45 ms 209 ms 231 ms 73 ms 49 ms 358 ms 113.00
google 21 ms 17 ms 16 ms 37 ms 381 ms 124 ms 79 ms 28 ms 183 ms 98.44
quad9 18 ms 19 ms 17 ms 42 ms 211 ms 127 ms 71 ms 73 ms 181 ms 84.33
freenom 36 ms 49 ms 59 ms 88 ms 534 ms 342 ms 219 ms 82 ms 204 ms 179.22
opendns 16 ms 19 ms 27 ms 23 ms 1514 ms 325 ms 85 ms 69 ms 488 ms 285.11
norton 25 ms 27 ms 26 ms 134 ms 389 ms 243 ms 277 ms 273 ms 354 ms 194.22
cleanbrowsing 22 ms 24 ms 27 ms 105 ms 533 ms 142 ms 70 ms 289 ms 199 ms 156.77
yandex 192 ms 197 ms 191 ms 293 ms 378 ms 803 ms 287 ms 603 ms 232 ms 352.88
adguard 84 ms 75 ms 74 ms 144 ms 240 ms 257 ms 72 ms 292 ms 170 ms 156.44
neustar 18 ms 21 ms 16 ms 29 ms 389 ms 222 ms 276 ms 285 ms 315 ms 174.55
comodo 65 ms 65 ms 82 ms 119 ms 458 ms 417 ms 236 ms 267 ms 290 ms 222.11
Also note that DNS queries might be overridden by your ISP. I've seen a few ISPs override DNS queries to 8.8.8.8 and respond with their own stuff. It might not be the case for 1.1.1.1 since it's not that popular.
The one thing Cloudflare DNS is missing is providing something like NextDNS.
Choose your own filter lists (that are constantly updated), create multiple profiles to use according to the target device/location and enjoy as blocking at the DNS level. It’s not a complete match for something like uBlock Origin, but a lot of stuff still gets blocked with DNS filters.
Thanks for pointing that out. I hadn't known about Cloudflare Gateway and am exploring at now. Preliminary thoughts: it seems a lot more complex than configuring and setting up NextDNS. Had a look at setting up policies, and it doesn't seem to support adding ad-blocking lists (like the ones used in uBlock Origin) easily. In NextDNS these are just checkboxes for each filter list.
> Unlike most DNS resolvers, 1.1.1.1 does not sell user data to advertisers.
Putting aside the question of whether they actually honour that commitment, has your ISP even published a similar statement to put their reputation on the line?
I think Cloudflare's commitment is plausible. They have a financial incentive to maintain their free DNS resolver's reputation and popularity, because they are selling points for their commercial authoritative DNS service; https://www.cloudflare.com/en-gb/dns/. Does your ISP have a similar financial incentive to behave?
"If it's free, you are the product" is not always true. Sometimes, if it's free, you are the marketing funnel.
why? arent we already using pi-hole for blocking all the stuff?
that said, i have a query about a simple way to force all dns in a local network to pass through pi-hole. i only have access to the iSP router and pi-hole and cannot use third party router
Pihole comes with a list of ads and trackers by default, but not with a maintained list of porn domains. There are more people working on getting trackers blacklisted than there are people scouring the web for new porn sites for free.
Pointing pihole at a porn blocker seems like a good combination of the best of both worlds to me.
Double NATting is underrated. I have zero problems with it and I like the buffer zone (subnet) between the ISP's Gateway/Router and my home network. Should the ISP's device have a known Zero Day exploit then it won't affect my home's subnet. Then there's all the additional stuff that can be done on your own router and also use DoH to ensure that a compromised ISP-router can't rewrite your DNS queries. Plus your ISP's router won't be able to gather statistics about the devices in your home, in case it would do that. I don't trust ISP-provided devices at all.
It's extremely unlikely that any consumer router would support it.
If you really want to force clients on your LAN to always use a specified DNS server you're looking at a more enterprise-y router solution, probably something running pfSense or OPNsense.
I wonder how much ICMP is going to those IPs. I ping 1.0.0.1 ("ping 1.1") as a quick check to ensure my internet is working a lot, far quicker and less stretching than typing ping 8.8.8.8. When I'm tracing a fault I'll ping 1.1.1.x as I can then tcpdump on a spanport against that IP and be fairly confident any traffic is from my test point and not from another device.
funny that you mention it but most technically minded Germans I know (maybe outside of people spending their days with datacenter stuff) habitually use `ping heise.de` (of c't and ix print magazine fame), which seems to have been a thing since the 90s. It's usually fast, you can really count on it being up and still around.
I even remember them once writing about having such an unusually high volume of ICMP traffic that they had to divert that traffic to a dedicated box at some point.
I still think this is a business that Cloudflare shouldn't be involved in. There are very legitimate reasons for parents to filter Internet content. But Cloudflare is in a unique position here, they have a brand as a company that cares about free speech, and specifically because of who they are, they really shouldn't be making determinations about what is and isn't inappropriate content for kids.
When 1.1.1.1 for Families launched, it blocked access to GLADD's site because Cloudflare didn't do a good enough job testing any of this stuff and they just pulled in filters from other parental companies, some of which turned out to be anti-gay. Cloudflare apologized, pushed a couple of fixes, but never actually took a step back and asked how this happened. In the meantime, 1.1.1.1 for Families launched without blocking access to sites like Stormfront. Cloudlfare didn't think it was appropriate for them to make a determination over whether that site was safe for kids.
I think that our society is just generally a lot less thoughtful about filtering adult content than it is about filtering other forms of content like political speech, and we don't think about adult content filters as having a downside, or being real censorship. So when 1.1.1.1 for Families was released, I came up with a challenge: https://danshumway.com/blog/sex-censorship-is-censorship/
I do think there are scenarios where it's completely appropriate to block content for children, and I do think families should always able to make these kinds of determinations. People and communities have a fundamental Right to Filter (https://anewdigitalmanifesto.com/#right-to-filter). However, adult content isn't the only content that falls into the category of being harmful to children. It is utter hypocrisy for Cloudflare to launch a service that blocks adult content but not hate speech; both forms of content are legitimate for parents to want off of their networks.
My challenge is, if Cloudflare is frightened of the implications of being the company that decides what is and isn't hate speech, then why isn't it also frightened of being the company that decides what is and isn't adult material? Why do we view accidental censorship of LGBTQ+ informational materials as less of an existential free speech risk than accidental censorship of political ideas or extremist groups? Cloudflare still, over a year later, doesn't really have clear documentation I can find anywhere about what specific criteria they use to make filtering decisions on 1.1.1.3 beyond that they "aim to imitate" Google Safe Search. Would people tolerate that kind of fuzziness if they were filtering hate speech or political extremism?
There is a reasonable debate people can have about whether or not it's appropriate for Cloudflare to be the company that carves out sections of the Internet that are inappropriate, even as an opt-in filter. I think both sides of that debate can make some good points, and reasonable people could go in either direction. But for me, the biggest question isn't really whether Cloudflare is the right company to build and maintain Internet filters. For me, the biggest question is about which subjects Cloudflare views as OK to moderate, and which communities Cloudflare is OK offloading the externalities of their moderation onto.
Because frankly, in free speech communities we do have a lot of hypocrisy about this. There's no argument to be made that extremist hate sites aren't just as dangerous to kids as pornography is. We should try to have more consistency about stuff like this. Are we OK with content moderation or not?
I think it’s up to the network owner to decide what should be blocked or allowed in their network.
1.1.1.3 (or 2) is a tool in the tool chest. Some people may find it too aggressive and don’t need to implement it, some may find it too conservative and implement more. No tool will be perfect for everyone, and if you don’t find it hits the right balance you don’t have to use it. No one has to use it, and cloudflare can literally release any free block list they want and call it parental blocking. It’s free, it’s a best effort product that doesn’t drive revenue, and it is up to each network owner to determine which blocks they want.
It would be a totally different story if the company was determining blocking for the US or people were forced to use it. But they aren’t.
I agree that for an optional tool, Cloudflare can make any blocklist they like. People have a fundamental Right to Filter. I personally don't think it's consistent with Cloudflare's brand or stated purpose to go down this route, but that's just my opinion, people can have other opinions.
I do want to kind of question how egalitarian we are inside free speech communities about this stuff though in reality. I am fairly confident that if Cloudflare added hate speech to 1.1.1.3 or started adding misinformation to their filtering list, that is something that would show up on HN and see debate. I think a lot of people on this site wouldn't see that as a neutral act, I think a lot of people would be on here arguing that it was a dangerous value judgment, or at the very least a dangerous behavior for Cloudflare to normalize.
We all have the right to filter content, and we all have the right to choose which filter lists we'll use. But is that actually our philosophy? Would we collectively as a community be applying those same standards if Cloudflare started blocking Covid misinformation or conversion-therapy sites from 1.1.1.3? The way society debates filter lists can sometimes betray our collective ideas about what kinds of information needs more or less protection.
> or people were forced to use it
There's a separate conversation to be had here about the fact that children are forced to use filter lists. This is exactly why Cloudflare reacted so quickly to stop blocking sites like GLADD and why if it ever does offer the ability to choose custom categories, it's probably never going to offer an "LGBTQ+ information" category to block.
Cloudflare (to its credit) does at least recognize that child filters are often only semi-consensual and can be (and regularly are) abused at the network level.
That doesn't change the overall debate, it doesn't mean that making a filter list is always evil, communities still have a Right to Filter. But it is important to bring up, kids at schools don't get to choose whether or not the filters on those networks are too conservative or too liberal with what they block.
Kids (necessarily by virtue of being kids) do not have agency to decide what networks they're a part of. There are good reasons for that, but it still puts kids into a somewhat more vulnerable position, and it means there are more dangerous implications for network-wide filters than there are for user-controlled filters. This is also something that kind of gets glossed over in these debates sometimes.
Parents have a responsibility to teach, guide, and educate their children to prepare them for adulthood. Today a vast amount of your "life" is online (much more than a decade ago). It only makes sense for parents to "parent" their children online.
> "Horrendously invasive"
Children do not have a right to privacy from their parents. Privacy (from parents) is a privileged that is earned and can be taken away. If you found your child off {insert worst thing you can think of} would you crack down on their privacy? Most parents would.
Parents also have the right to decide for themselves what really is "bad", and then try to raise their child according to those beliefs.
Don't confuse privacy from parents as privacy overall--children absolutely have a right to privacy from companies/3rd parties.
Simply because the internet doesn't physically harm you in an immediately noticeable way doesn't mean it's not dangerous or that harm isn't being done. It's good for parents to be aware of potential dangers (of which there are plenty) and to help their child navigate them.
Also, as other's have pointed out, the internet from decades ago is much different than the internet of today.
> Children do not have a right to privacy from their parents.
They do everywhere outside the US, under article 16 of Unicef's convention on rights of the child. Of course this right is not absolute and many will say that the right to a child's safety comes before the right of a child's privacy, but children do inherently have a right to privacy. The convention does not exclude parents from this right for good reason.
There are parents who will demand their 17 year old child to hand over their private conversations, search history, you name it, and there are those that give 3 year olds unrestricted access to the internet. Neither extremes are healthy for children, but this "guidance" for the internet can last into children's late teens for certain parents.
The US signed the convention but did not ratify it, so you're correct that children don't have this right specifically in America. Legally speaking, the UN convention should ensure the right in all other recognised countries, though.
I don't believe article 16 gives a child right to privacy in all places and at all times. I believe a loving parent should be able to rightfully demand that online privacy be limited for the child in some ways with respect to the parent. I failed to point this out, but I was generally referring to online privacy with my comments. Thanks to your comment I've thought this through a bit more and did a little digging/research.
> There are parents who will demand their 17 year old child to hand over their private conversations, search history, you name it
Under certain circumstance this is justified, even according to unisef and the resources they link to.
"Governments, companies and others should support parents with
appropriate guidance and tools, including how to respond to and, if needed, report harmful contacts,
conduct and content." [1] -> "guidance and tools" links to [2], which explicitly states parents can "Monitor a teen’s social media sites, apps, and browsing history, if you have concerns that cyberbullying may be occurring" as well as "Know your child’s user names and passwords for email and social media" and "Establish rules about appropriate digital behavior, content, and apps".
I'd also add that unicef has called out pornography as potentially harmful for children [3], as that's likely a topic many children may debate. And that parents may use parental controls [1] (point #4) to help them make online platforms safe for children.
We use the Google Families setup to provide some safety features (location), and have device schedules to limit constant use. Families also shows which apps are used, and for how long. We don't track websites, or filter them other than for ads. I also use the similar setup on the google wifi (now nest) mesh devices, to have schedules so that time limits are enabled for all children's/media devices. They often ask for overrides, or extra time, and that's fine - interactivity over health boundaries. We don't collect any data on content, contacts, etc. Just apps and how long per day/week/month, so we can share that with them.
We have to teach our children to be good people, and how to process the world, and what we've already managed to process out of what we've seen throughout our own lives. If your strategy depends on censorship to provide a healthy path, I don't think it's going to be that healthy of an outcome.
I'm nearly 40, the internet that I grew up with, is vastly different to the internet my kids are growing up with. Heck, the internet the 15 year old grew up with is vastly different to his 4 year old sister's experience.
I see nothing wrong with blocking access to certain sites by default - protects us as well - if anyone of them have a problem with it, they can come and ask why it is blocked. Simples.
I believe the biggest difference is the expansion of what's available online. When I was growing up online, I didn't have to worry about the same set of issues children have to navigate today, or even the same set of bad things online. There weren't as many attack vectors, and there weren't as many people to target. With more people online and accessible, it's safe to assume more issues will rise with having unrestricted access.
> I had unrestricted internet access as a child and turned out fine.
The statistical power of an n=1 study applied to a population many orders of magnitude larger is not very strong.
That aside, many, if not most children below a certain age lack the requisite ability to discern danger/non-danger with a fidelity that would satisfy their parents who have moral and legal responsibility in that domain. I admit there's a tension between privacy and the duty to protect.
As humans, we can share our experiences on important human questions, such as what is an appropriate way to raise a child, without being told that a sample size of one is insufficient to found our opinions.
Also, with respect, you could also have made the point that my experience might not be the experience of everyone without dressing it up in statistical speak. I can assure you that I understand basic statistical principles.
Point of clarification, was this before or after stormfront explicitly started using the site as a recruitment and training ground for neo-nazis? Because I also spent a fair bit of my middle and high-school time on 4chan, and can attest to the fact that the 4chan of 2007-2012 and the chansites of the present day are very different beasts.
Where are you getting this from? Of course discussion on 4chan wasn't as politicized back then because politics was a much smaller part of the internet a decade ago in general, but people back then are still what you'd call "nazis" or whatever even today. If anything the /n/ (the first iteration of /pol/) board had existed since close to the start of the site and only in 2008 was it removed and replaced with the Transportation board.
Using 1.1.1.3 blocks adult content on search engines like Google - it’s obviously not hard to find adult content through other means, but it avoids accidents.
Our networks have always been open and unmonitored for our children.
We figured it would be better to train and guide them around the "search for pussy pictures" results than to let them grow up in a sheltered internet at home and get confronted with the "less desirable results" when connected to the open networks of friends & neighbours.
Are you telling us this for some reason other than to make yourself feel better about yourself? Because it comes off as sanctimonious and self-serving.
Different parents. Different families. Different cultures. They will all make different decisions. Your choice for your children is likely not the right choice for other people and their children. So I'm not sure what it is that you're trying to brag about.
I bascially ran a 'dig' with multiple DNS providers and CloudFlare was slowest among the bunch for long-tail domains.
Here are the details: https://twitter.com/vladquant/status/1428761979808669704
CloudFlare never responded to this tweet.