The results page should always show what your search term was. Currently it only shows this when the search finds something. If nothing is found, all it does is tell you that nothing was found.
That leaves you with no way to check to make sure you entered the right term, other than typing it again. If you are typing it right but auto-correct is kicking in when you hit enter, even trying again might fail because your site responds fast enough that you might not have time to notice the change before the results page comes up.
Also, it might be helpful when someone searches for a domain at a TLD that you do not support to say on the results page that the TLD is not supported.
+1 on search term! And if you enter a domain that doesn't exist, it throws you back to the homepage with no way to fix it :(
Another feature request: When entering an IP, there's very little information available on it (only the hosting provider). Would be nice to get latlong, country etc (a la https://www.iplocation.net/).
Because seriously, that is an insanely cool database. I can even find all the sites that share my domain's Cloudflare IPs.
Awesome ideas. Will implement. If you'd like updates in the future drop me a line at chris at securitytrails.com - would love to have you keep testing.
I didn't look into what this is doing but it's not finding 1/2 of my domains (they are not private). I'm guessing it's not "Find every domain someone owns" it's "Find every domain that meets X criteria" which may more may not be every domain someone owns.
Yeah, I was fairly unimpressed when it didn't even manage to lookup the first domain I entered (it's not new and it's not private). I guess it might be useful for some domains, but there's no way I'd rely on it.
Same about this one. Any further info so we can enhance the user experience would be awesome. We have a tough time still with ccTLDs because registries locking down the zone files. Any more info or suggestions appreciated.
Hi! Would love to know more about the domains it's not finding. Are they Country Code domains or gTLDs? Any other info so we can debug would be awesome. Thanks for trying the service!
Must say, I'm not a fan of yours or similar services, or whois databases for that matter. Privacy should be easier on the internet for people owning domain names.
Hi! The only way on the current Internet to remain anonymous is to obscure the domain registrant from right when the domain is initially registered. It's also important to use a service that pools a lot of domains together or have diversity in the hosting providers so they can't be easily correlated. We try and help people understand and make the problem visible. You bring up a good point that it should be easier.
A good option for people living in the EU is to register a .eu domain. For private individuals (like private non-commercial websites), the whois data is by default restricted to show an e-mail address only. (see section 2.4. in https://eurid.eu/en/other-infomation/whois-policy/)
Yeah, it's hard. If one has multiple domains from multiple tld's pointing to the same server, it's really easy to find out other domains and tlds with more detailed and easier to access records that will reveal the owner's identity.
Privately registered eu domains are quite nice. I couldn't even find two of such that I own in your database and whois only reveals email (which I made sure from the beginning to be of the same domain that I just registered, so that's useless for anything). People can probably pay for more info though.
Why do you think privacy should be easier for domain owners? Shouldn't it instead be easy for a visitor to find out who owns the domain and is responsible (content, technical or legal) for a certain site?
I bought an internet domain in order for people to be able to contact me via some stable handle on the internet, that only I control. That's all. I don't want my phone, my home address, my all other domains, etc. to be easilly accessible for no reason whatsoever by nosy types.
Also your question may apply to the other end too. Shouldn't it be easy for an owner of the server to find out who is behind an IP address that is connecting to his server - and I mean easy access to his/her phone number, home address and a complete history of those, too?
It's easy to concot legal, technical and content reasons why this should be.
How do you plan on handling the EU’s “right to be forgotten” (it’s pretty straightforward to make the argument you’re a search engine) and other components of the GDPR?
"The right to erasure" is not an absolute right for anyone to get all their data deleted. If the data owner (read: the registrars) still have a legal right to collect and maintain the data public and it has not been revoked one could argue that they (security trails) don't have to remove the data.
It's my understanding that the registrars are the ones with the burden here. They need to inform everyone of the data erasure and/or data updates on private information.
Fun times when you have public information for anyone to gather on the internet. It could be that there are exemptions for these kind of services, I do not know, but would the exemption not also include the services that aggregate/collect historic information as well?
Disclaimer; I am not a lawyer. I am not well versed in GDPR.
Anyone finding this interesting should go read up on GDPR.
Interesting! From a quick google the following wikipedia citation seems to what you are referring to:
Grounds for removal include cases where the search result(s) "appear to be inadequate, irrelevant or no longer relevant or excessive in the light of the time that had elapsed."[1]
Under GDPR, Security trails (company or person that operates it) could be classified as a "Data controller" [2] and then would of course be liable to delete information gathered about a person upon request and when the data is deemed to be "inadequate, irrelevant or no longer relevant or excessive".
So for example, John Doe wants to remove the historic information that he used to own porn.com which he doesn't anymore.
However, I do not think it's clear that you have to delete the data for the current owner of porn.com due to his or hers need for privacy as long as they have collect the information lawfully.
As an actual advice to the people at security trails I would recommend they put up clear instructions on how to request a data erasure from their database. Like "Email erasure@securitytrails.com to request removal of your personal information" and what information they need to delete it.
Actually the first thing Security Trails have to do is to figure out under which legal basis they think they have the right to process personal data. This is fundamental to figuring out their duties. I strongly suspect they don't have a legal basis in GDPR terms and therefore would need to rely on consent. The much publicised "right to be forgotten" is the very least of their worries.
You should read up on GDPR urgently. It applies to anyone anywhere in the world processing data of EU citizens and has some massive penalties for things like not asking for specific informed consent and not offering an opt out. Using the excuse that you're just aggregating public data does not cut it.
It says I own 86 domains but is using my given names and not my email address. My name is not unique. Hardly a valuable service. I own less than 10 for the record.
Is this just a reverse indexed WHOIS database? If so, it's no surprise that my domains don't appear in yhe results: I signed up for Whois anonymization through my DNS provider. I was recently considering unsubscribing, so thank you to the creators of this for reminding me that my privacy is under attack at all times and I should do whatever I can to protect it.
I'm finding that sometimes it'll turn up the data correctly and other times it won't match what I can find by manually typing in addresses into domains.google
As in this service will claim all data is private when google is able to return the actual registrant email address and/or name. As well as valid phone numbers which don't match what dnstrails is outputting.
So has Domaintools (http://domaintools.com) but both are expensive (Domaintools in particular is very expensive) whereas DNSTrails appears to be free unless you want API access.
Hi ohashi, we do not only track the current and historical whois records, but also current and historical DNS records, even for subdomain, which technologies the website uses and even more data which we are currently working on. Maybe try our WHOIS aggregation feature and let us know what you think!
Just tried it with some of my own details. Found a domain I forgot I owned on an old reseller I havent used in a while - lol.
On a more serious note - I'm very curious how you get such a long history of domains. i.e. I can see every DNS change and ownership for any domain - I didnt realise that was always available?
Hi! We acquired 4 companies that have been doing lots of cool data work. We also license and collect our own data to mix in. There's different granularity depending on the data (WHOIS history, Name Server history, DNS record history, technologies used) etc. We're constantly improving it.
The thesis we have is that if you get hacked, it often times is through an old server or satellite domain. We're building tools to help you find the extended surface area where you can be hacked or have downtime. The example of you finding an old domain is a prefect use case.
Just out of curiosity, was one of those companies originally named DeletedDomains.com? They had the full root zone (of the ones now managed by Verisign GRS anyway) since approximately 2001 (my involvement with them was 2003).
I'm pretty sure this goes against European (or at least Dutch) privacy laws because it's not just company domains being searched. There isn't any privacy-overriding reason to keep a database with this kind of identifying information. Since these laws are currently barely enforced, nothing will happen of course.
So, for .de it just returns either NULL, undefined, or empty for everything, for .eu it errors out entirely, and for the domains of me it does find, it has wrong data.
kuschku.de has, for the past 2 years, always pointed at 51.15.1.223 or 163.172.217.134, never at 204.236.227.242. Funnily, for other domains pointed at the same IPs, it has correct data – e.g. quasseldroid.info correctly shows the IP history.
The datasets used here seem of questionable quality, souring the taste of this awesome feature.
hey- would love it if you can drop me an email (in profile) so we can debug/improve.
.DE is particularly hard because they lock down the zone file. GTLDs like .INFO are easier to get because the zone files are open. We have 9 years daily granularity for the gTLDs.
Would love to clean up the U/X so it's clear what we have data for and what we don't to be completely transparent.
For example, WHOIS info https://i.imgur.com/WNpyvcl.png should maybe show something like "none available", or "no WHOIS info is available for .de", or "go to denic.de to see WHOIS info" (DENIC offers the WHOIS info, if you enter the captcha). On the other hand, http://whois.domaintools.com/kuschku.de (a competitor) correctly shows the WHOIS.
Second, with the domain – I have no idea how the wrong value ended up on there.
Hi kuschku, thanks for providing us with the samples with the outdated/missing data. We will definitely look into the case and continue to enhance the data we collect.
The UI enhancements you mention are excellent - we will implement this shortly (not only for .de but for any case where we can not output any values).
Feel free to contact us at the e-mail address given at the bottom on every dnstrails page!
What’s interesting is that it correctly finds Namecheap whois-guarded domains too. If I search for the guarded domain directly it shows the correct record as the owner’s address/etc being WhoisGuard, but then if I search for a non-guarded domain and click through from the identified name, it does list the guarded domains as well (!)
It said I owned 45 domains, and listed 10 that I actually own along with 35 I've never owned, heard of, or ever been associated with.
I have a very common english name.
Annoyingly, some of the domains it inaccurately says I own are NSFW. They need to put a big disclaimer on the results page pointing out that the results aren't necessarily accurate.
Nice find bananamansion. Actually you will see this with many big cooperations which also register domains such as <companyname>.sucks, <companyname>.adult etc. ;)
Is it possible to include WHOIS data from way back when InterNIC was the only registrar?
I own a domain I registered way back when you would send an email to InterNIC and registration was free. The WHOIS data returned for the domain only starts in 2008 and skips about 12-14 years.
Hmm is there a technical write up of how you are pulling this data?
I tried one of our companies for fun and it’s only pulling 1.4million records in one place and then 65,000 in another. Doesn’t seem to have all our nameservers or relays either.
Based on the supported TLDs, I’m guessing they are pulling down the root zone tables from Verisign GRS. Verisign licenses the Whois data in bulk out to companies like theirs.
<sigh> these country code domains. LOL. We bought and built up http://www.domainlists.io so we can add some more stuff like this in. I'll make sure we add .RS to the action items. Thanks for the feedback and enjoy!
We have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.
I'll take a guess : customers are given unique whois protected email addresses allowing you to find all the domains owned by the person. Eg- If I own abc.com and xyz.com, both have the same public email address. The problem with such bugs is that there's no way to undo the damage since historical whois records are archived.
you nailed it. It goes a little further with what you can see and how easy it is but right on point! If you notice anything else people would find interesting, feel free to post it or email me (in profile).
You say fun, but in the era of swatting and doxxing, private registrations are often the first line of defense for those who don't have a USPS or ups mail address if they don't have a physical office/address. I would be very careful about how you responsibly disclose any such bugs.
Good point. Poor word choice on my end. Thanks for the note. The stuff we found is not quite that abrasive but very interesting. How would you recommend disclosing?
1. for the current record displayed on the domain results page, it's live but then has a short cache after the first time it's pulled on the public site.
2. For the whois registrant search, it's around 90 days old right now, but we're working on techniques to make it more current.
It's pretty much missing or empty data for the entire .au TLD (e.g. even google.com.au is not present).
Living here, I know that their whois server rate limits heavily and the ccTLD zonefile is not available, so I'm guessing those are probably contributing reasons. Do you guys do any crawling at all?
I think this is a fantastic concept though. Knowing that whois database and zonefile access is often protected for commercial motivations, it really irks me. Open it all up.
right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.
It worked exactly as I hoped. I couldn't figure out why some domains I were expecting to show up weren't, but searching by email reminded me that I need up update some of my whois info due to a name change.
That leaves you with no way to check to make sure you entered the right term, other than typing it again. If you are typing it right but auto-correct is kicking in when you hit enter, even trying again might fail because your site responds fast enough that you might not have time to notice the change before the results page comes up.
Also, it might be helpful when someone searches for a domain at a TLD that you do not support to say on the results page that the TLD is not supported.