I didn't know HE.net was using Network Solutions, maybe they will switch to someone else soon. My personal experience with Network Solutions wasn't so great (lots of trouble and human intervention needed to transfer a domain out of their service).
I can't believe that NetworkSolutions doesn't have a list of significant ISPs by domain name or something. I would proactively work with significant ISPs before suspending the entire domain and interrupting their services.
idk. aren't they tier 1? what if it breaks peering? is all their management capability strictly "inside" their network? what would happen to the rest of the internet if they dropped off?
There's a reason people were practically partying in the streets when Network Solutions lost their monopoly.
The fact that their netops / support staff have no way to escalate an issue like this is the bigger issue. IMHO the giant ISP shouldn't get a better response time or procedure; everyone should be able to get help in an emergency like this.
Really, NANOG have themselves to blame. Half the reason NANOG was started was to help high level techies bypass low-level operations/support staff.
Everyone should have worked together to make sure their own orgs could appropriately handle calls/issues like this.
Instead, with the issue mostly fixed for them, they stopped caring, and all the peons and techies who don't work for Fortune 500 or famous tech companies are still fucked...
First off, Hurricane Electric has always been one of the "cool" companies on the Internet since the early days (of public Internet, at least). So, my goodwill with them tells me that Network Solutions, a company that most certainly has NOT been one of the cool companies, is at fault. I feel and know this to be the case now, as well.
Having said that, I am also greatly pleased that the NANOG mailing list is public and plebes like us can see these kinds of discussions, which I feel are becoming harder to find as time goes on as services get consolidated into larger, government-friendly corporate private digital gardens.
Status in whois response is “active” for me, and he.net resolves from DNS for me, so I guess it’s been fixed?
But why did they list a @he.net email address as one of the emails to contact them on? Wasn’t the problem that he.net would not resolve because of that very client hold?
Verified he.net is correctly listed in the root servers, though I did not check prior.
dig +nocookie @i.gtld-servers.net he.net
;; AUTHORITY SECTION:
he.net. 172800 IN NS ns1.he.net.
he.net. 172800 IN NS ns2.he.net.
he.net. 172800 IN NS ns3.he.net.
he.net. 172800 IN NS ns4.he.net.
he.net. 172800 IN NS ns5.he.net.
;; ADDITIONAL SECTION:
ns1.he.net. 172800 IN AAAA 2001:470:100::2
ns1.he.net. 172800 IN A 216.218.130.2
ns2.he.net. 172800 IN AAAA 2001:470:200::2
ns2.he.net. 172800 IN A 216.218.131.2
ns3.he.net. 172800 IN AAAA 2001:470:300::2
ns3.he.net. 172800 IN A 216.218.132.2
ns4.he.net. 172800 IN AAAA 2001:470:400::2
ns4.he.net. 172800 IN A 216.66.1.2
ns5.he.net. 172800 IN AAAA 2001:470:500::2
ns5.he.net. 172800 IN A 216.66.80.18
I don't have a paste, but a similar query gave me NXDOMAIN during the incident. I did get the A/AAAA glue records for nsX.he.net when asking gtld-servers[1] about my .net, though.
[1] note that these are tld servers run by Verisign, not root servers organized by ICANN or IANA and run by multiple organizations. A few decades ago, I think root-servers.net did do root and also the three letter tlds, but that was a long time ago, before everything got split up into multiple roles.
I have old habits I am trying to break doing things from memory as it gets me into trouble here for saying outdated things, so I just did 'dig -t ns net' and then did a dig @ the first one in the list. I considered checking them all but I don't know what the current convergence time in the anycast clusters are for net.
I'm glad you caught the NXDOMAIN. It's really one more epoxy coated screw in the coffin proving people should move off that registrar. It should take a great deal more than an abuse complaint to get a domain on hold, especially one belonging to a large network provider. Maybe it would make sense at this time for Hurricane to become it's own registrar. I printed out the book of requirements. For a retired hobbyist like me it would be insane, but for a business I think they could do it without too much overhead.
I read a bit more in the linked email thread. Some other message said that caching would likely make it still resolve in most places and that checking the root servers would show the problem and that elsewhere it would show up around 48 hours after, when the caches expire.
Still not sure if the status it shows in Whois means that it’s resolved or not.
Looking to see what escalation numbers I may still have...
[Edit]: Sent them what I have but it isn't much. Most human contacts have been replaced by menu's. I hope the last one I sent helps.
It appears someone managed to get NetworkSolutions (now web.com) to put the domain he.net on clientHold for one Phishing complaint, unrelated to he.net meaning someone did a lookup on bgp.he.net and reported he.net instead of the domain they were looking up. If true it means web.com did not do any due diligence, investigation, etc... and just put a very popular domain with many shared services on hold. The domain he.net is used by MANY services and those services are likely going to start breaking. I have no idea what's going on over at web.com.
As a side note, I recently transferred all my domains away from NetSol/web.com as they lost the ability to define custom name servers (glue in the root servers) which is bizarre to me, given many of the meme-style resellers of resellers can still do this. I had many other strange experiences with them, including having to teach their escalation contacts how to use dig outside of their own name servers.
I was with Network Solutions from 1997 to 2023. They were bought by web.com in 2018. They keep the name on the old domain and don't redirect to web.com because they have some non-API automation in place on the old domain.
Wikipedia has it incorrect then, as they list it as "formerly web.com" ("Network Solutions, LLC, formerly Web.com is an American-based technology company"). Thanks for the clarification!
> As a side note, I recently transferred all my domains away from NetSol/web.com as they lost the ability to define custom name servers
FFS is that what happened? They recently bought the registrar I use and a domain of mine with custom nameservers isn't working and I haven't had a chance to troubleshoot.
I doubt they have fixed it and I doubt they have plans to, but in the off chance they have make sure it is prefixed with dns(number) or ns(number) as they will not give you a useful error message if you have not done so. e.g. ns0.yourdomain.tld but that probably won't work either. I used to use bare apex domains for a couple decades but they and several other registrars stopped supporting that. They do not give useful error messages to the clients or to their support. AFAIK there is nobody left that knows how to debug the back-end.
This is just a suggestion but I would consider moving the domains somewhere else. It seems web.com is just hoovering up registrars but not migrating everything to a centralized, modern, maintained system. In my experience and opinion they do not know how the back-end of the old Network Solutions systems work.
Hurricane Electric (HE) provides, among other things, DNS services. When a domain is placed in `clientHold`, as has happened to HE due to a spurious phishing report, it causes the domain to no longer resolve. So the DNS records for HE and all of HE's customers are gradually becoming unresolvable as caches expire.
They offer a nice (free) secondary DNS service, which I use for all my domains. From my limited interaction with them, they seem like a great company. They also provide some nice IPv6 tutorials as well, and offer free IPv6 tunnels for those who can't get native IPv6. All in all they seem like a model company that believes in giving back to the internet community in meaningful ways.
Unfortunately, I use them to host secondary DNS for all my domains, so this Network Solutions stupidity is hitting home for me right now. Then again, I had enough issues with NS back in the 1990s that as soon as that monopoly was broken I switched and never looked back. TBH I'm a little surprised that HE would use NS, but perhaps they did thinking that NS would provide proper enterprise-level support for a major backbone provider and not cut them off on a holiday with no recourse - behavior you might expect or fear from a cut-rate provider.
Edit: since the domain is fixed, this isn't reproducable anymore.
An interesting thing is that if you have a .net domain with he.net nameservers, the .net authoritative servers will give you full glue records with the A/AAAA for nsX.he.net. But if you ask for he.net, you get back NXDOMAIN.
Example domain removed; no longer relevant.
NetworkSolutions is a truly terrible registrar, and anyone who still has a domain there should use this occasion to switch to somewhere better, which is almost anywhere. IMHO, it may be worthwhile for a real business where their domain is important to move to an expensive corporate registrar like MarkMonitor or CSC, and looking into registry lock, etc.
For anyone not well versed in DNS who needs an ELI5 level:
This impacts anyone using HE.net for their domain name service because a domain name specifies its nameservers (in an NS record) using fully qualified domain names. I believe it's ns1.he.net and ns2.he.net.
User at WildISP tries to go to a site. Resolving nameserver at WildISP.net fetches the NS records for samplewhamplesite.com from a root server and gets pointed to ns1.he.net...which no longer exists because the entire domain disappeared from the root servers.
> This status code tells your domain's registry to not activate your domain in the DNS and as a consequence, it will not resolve. It is an uncommon status that is usually enacted during legal disputes, non-payment, or when your domain is subject to deletion.
> Often, this status indicates an issue with your domain that needs resolution. If so, you should contact your registrar to resolve the issue. If your domain does not have any issues, but you need it to resolve, you must first contact your registrar and request that they remove this status code.
They provide same functionality as DNS[sec]. You are not at whim of a corpo, nobody can suspend you, corrupting or hijacking records is pretty much impossible...
There are already many anti-phishing solutions: Chrome has a built-in phishing database, it shows a warning in place of a phish web site. Many VPN and anti-virus software packages come with bad site blockers, it can be done on DNS resolver level, etc.
This is a free market solution: different vendors can compete in phish detection, and users can choose one which works the best.
There's really no need for TLD provider to do this.
And we see why it's a bad idea: a mistake can lead to thousands of legitimate sites being kicked off.
Why are you advocating a solution which is clearly inferior?
If we treat Blockchain as a database with specific properties, and use that to back DNS, then we have a DNS system with those specific properties, instead of the system we currently have, which has different specific properties. Some of those properties are desirable, but as we see here, some of them are not.
https://mailman.nanog.org/pipermail/nanog/2024-July/225903.h...
I didn't know HE.net was using Network Solutions, maybe they will switch to someone else soon. My personal experience with Network Solutions wasn't so great (lots of trouble and human intervention needed to transfer a domain out of their service).
reply