
A penetration tester’s guide to sub-domain enumeration - adamnemecek
https://blog.appsecco.com/a-penetration-testers-guide-to-sub-domain-enumeration-7d842d5570f6
======
belorn
For those on the defending side, remember that the purpose of DNS is to be
public. Relying on DNS information to stay secret in order to protect
vulnerable application is a bad idea. A service with a SQL injection is
unlikely to last long, and as the article shows, it is not hard to enumerate
DNS records if you are under a targeted attack.

~~~
feelin_googley
"... remember that the purpose of DNS is to be public."

DNS records, e.g. zone files, are supposed to be public information.

As such, ICANN requires gTLD registries to provide users with access to
them.[FN1]

However, various ccTLDs at various times refused to comply, and most still try
to pretend their zone files are private.

Obviously, as the blog post shows, users can still get a large quantity of the
data freely from other sources.

Google search results for ccTLD zone files are dominated by individuals who
have done so and are selling this data. IMO, this sale of public data is
unnecessary, not to mention annoying.

Last year .se and .nu started making their zone files public.

[https://www.iis.se/english/press/pressreleases/iis-
releases-...](https://www.iis.se/english/press/pressreleases/iis-releases-
zone-files/)

[https://zonedata.iis.se](https://zonedata.iis.se)

They were also the first to try their hand at DNSSEC. As the blog post shows,
DNSSEC makes it _easier_ to get contents of a zone.

FN1. "About Zone File Access

Registry operators must provide to ICANN bulk access to the zone files of the
Generic Top Level Domain (gTLD) at least on a daily basis. For gTLDs, a zone
file contains information about domain names that are active in that gTLD. In
general, Internet users may be able to access and download zone file data at
no cost for certain purposes.

This contractual obligation does not apply to any ccTLD (such as .us, .de, or
.uk). If you have a complaint about gTLD zone file access, please submit a
Zone File Access Complaint Form."

source:
[https://www.icann.org/resources/pages/zfa-2013-06-28-en](https://www.icann.org/resources/pages/zfa-2013-06-28-en)

------
jcims
The CommonCrawl dataset is another good source of host and subdomain
information. The index server is here:
[http://index.commoncrawl.org](http://index.commoncrawl.org) There's a client
referenced on that page.

    
    
      [ec2-user@x cdx-index-client]$ ./cdx-index-client.py -c CC-MAIN-2017-43 '*.ycombinator.com'
      2017-11-11 22:50:08,381: [INFO]: Getting Index From http://index.commoncrawl.org/CC-MAIN-2017-43-index
      2017-11-11 22:50:08,387: [INFO]: Starting new HTTP connection (1): index.commoncrawl.org
      2017-11-11 22:50:08,468: [INFO]: Fetching 2 pages of *.ycombinator.com
      2017-11-11 22:50:08,477: [INFO]: Starting new HTTP connection (1): index.commoncrawl.org
      2017-11-11 22:50:08,478: [INFO]: Starting new HTTP connection (1): index.commoncrawl.org
      2017-11-11 22:50:10,409: [INFO]: 1 page(s) of 2 finished
      2017-11-11 22:50:11,711: [INFO]: 2 page(s) of 2 finished
      [ec2-user@x cdx-index-client]$ awk -F')' '{print $1}' domain-y* | sort | uniq -c
           66 com,ycombinator
           38 com,ycombinator,apply
          500 com,ycombinator,blog
            2 com,ycombinator,fellowship
            2 com,ycombinator,macro
        24036 com,ycombinator,news
           25 com,ycombinator,old
            2 com,ycombinator,ycblog

------
polote
Great list !

Also you can look at the SSL certificate of a page of the doamin and see if
many domains appears inside it.

And currently if the website uses letsencrypt and need wildcard DNS, you may
find hundreds of domains. Fortunately Letsencrypt wildcard certificates come
in January

~~~
nsgi
If the website is using Let's Encrypt then any subdomains using it will be in
certificate transparency logs anyway (until it supports wildcard
certificates).

~~~
throwaway613834
Do you know how to find out which certificate issuers DON'T submit certs to
transparency logs? Is there a list somewhere?

~~~
regecks
Symantec presently give you the option to redact the labels on
precertificates.

I think they are the only ones because RFC 6962 does not actually support
redaction.

Symantec say they are only submitting these entries to the
deneb.ws.symantec.com CT log, but it is possible to cross-post redacted log
entries to other CT logs, which actually is happening and is kind of annoying.

You can see an example of a redaction (for nsw.gov.au) here:
[https://crt.sh/?id=49265178](https://crt.sh/?id=49265178)

I think the best way to protect your privacy currently is to purchase
wildcards, though LE will support wildcards next year :) !

Kind of related, I also wrote
[https://ausdomainledger.net/](https://ausdomainledger.net/) a couple of weeks
ago that is crawling CT logs and Common Crawl as well to enumerate domains in
the .au zone, because the zone file is not accessible for the ccTLD.

~~~
throwaway613834
Ah, thanks! Yeah, currently I'm just waiting for LE's wildcard support, but
then it hit me there might be others who don't even submit the 2nd-level
domain to CT, which I'd prefer if possible.

------
coderholic
Great guide.

On point 6, find the asn, my own service
[https://ipinfo.io](https://ipinfo.io) can help you there. Curl
ipinfo.io/ANYIP/org and you'll get the ASN info. Also see ipinfo.io/developers
for more options such as geolocation, hostname, hosted domains, and more.

~~~
_synster_
Thanks. I'm the author of the blog post. I have used your service before and
it has come in handy many times.

------
dzhiurgis
WolframAlpha can display subdomains, not sure how and the format is not easy
to use. But it works.

How one would one go about finding A domains pointing to specific AS. The
target web app in question allows either using their subdomain or pointing to
your own. Any ideas how to find those?

------
mschuster91
CT is going to have way more impact given that people switch from paid certs
to Lets Encrypt which cannot do wildcard certs.

I'm looking forward to wildcard certs, anything that makes it harder for
attackers helps...

~~~
_synster_
Wildcard certificates are not really a solution especially in this scenario.
If you are using wildcard certs you have one more problem to deal with.

------
fencepost
If you're looking for physical site ip addresses rather than dns entries, SPF
records may be worth looking at (if there are distributed mail servers and
outbound mail doesn't all go through a single exit point).

Mail headers may be similarly informative if you can get one or more email
messages out from within the office.

~~~
_synster_
Looking at SPF records is worth it. I should have added that technique.

------
shabbyrobe
I love "AXFR". It lets me AX FR the whole zone in one go.

~~~
_synster_
I'm surprised at the number of domains that have mis-configured Zpne Transfer
even today especially given the fact the the defaults of most popular DNS
servers are secure enough.

------
Joyfield
I have a project called DNSDigger.com that could be used as well.

------
porfirium
A small question. For a CloudFlare-protected website, I see this in crt.sh:

>X509v3 Subject Alternative Name:

>DNS Name:sniXXXXXXX.cloudflaressl.com (X are numbers)

>DNS Name:(lots of domains, including the one I was researching)

>...

Does that mean all those domains belong to the same user/person?

~~~
breakingcups
No, Cloudflare batches together many unrelated domains from different
customers into a single SSL certificate.

~~~
programd
Which leads to amusing issues like the following:

[https://arstechnica.com/information-
technology/2015/03/ted-c...](https://arstechnica.com/information-
technology/2015/03/ted-cruz-for-presidents-ssl-certificate-nigerian-prince-
headache/)

Though to be fair, this incident is now fairly infamous inside Cloudflare and
these days they have ways to deal with it.

