
Ask HN: How are bots finding my (brand new) site? - hacky_n00b
I have just put a site live &amp; I can basically guarantee no back-links point to my site and I, the creator, cannot conjure up any google search to return it at all (it does not appear to be indexed - I have done no SEO whatsoever). It does have DNS records, just no way to find it from a search engine or crawl for links to it.<p>However, I can tell from the logs that bots are already probing for vulnerabilities.<p>How on earth do bots even find such a target?
======
dmlittle
They might be keying off SSL certs created in the past few days/hours. For
example, this cert[1] for www.krsnalite.com was created a few minutes ago.

[1] [https://crt.sh/?id=3120549998](https://crt.sh/?id=3120549998)

------
gregjor
How do ants find a piece of fruit I left on the table?

How do Jehovah’s Witnesses find me no matter where I live?

Finite address space, lots of time for bots to go door to door.

------
terrycody
There is a domain pool which contains all domains in the world registered and
many people/companies in the world will scrape the list 24/7 to filter sth to
meet their own needs.

Except this, there are a lot of possible reasons you may expose your website
to a bot:

1) if you choose wordpress, when your site is a alive, site will Ping some
services. 2) when your site is live, your site's IP is fixed, thus may get
scanned somehow. 3) when you register your domain, your provider know your
domain, the name can expose to somewhere. 4) if your domain name happen to be
a "reused name", your website is exposed.

There are many many possible ways I don't even know, but in a severe case,
people will receive spam email less than 1 hour after they bought their
domain, you guess it.

Nothing is hidden today.

------
gvb
The bots scan the entire public IP address space for open ports.

------
elliottinvent
Another possibility to throw into the mix is that the domain is now in a
public zone file, like the one ICANN[1], Nominet and others provide:

1\. [https://czds.icann.org/](https://czds.icann.org/)

------
thephyber
Brute force IP crawling (this is how Shodan.io works).

SSL Cert transparency logs.

Remember that bot nets and other malware allow the type of scale that you
wouldn't expect. There is a big advantage for the first mover in hacking. I
wouldn't be surprised if nation states or cyber mafias paid bounties to lots
of contractors for identified vulnerable boxes to use as intermediate proxies
(although there are so many I'm not sure if bounties would pay well).

------
achairapart
Also take a look at things like browser extensions. It's not uncommon that
they phone home every URL you visit, then crawl them for whatever reason.

------
mattmanser
They watch for new DNS entries and then ping it.

It's the reason why Google have their own DNS registrar, they get told about
all new websites.

------
ryanmccullagh
IP addresses are recycled.

