Show HN: CT Logs Search Engine

[Eikon]

https://www.merklemap.com/

I recently launched https://www.merklemap.com/, a tool designed to leverage Certificate Transparency logs for enhanced domain monitoring.

Example query

https://www.merklemap.com/search?query=example&page=1

Key features of MerkleMap:

- Near real-time ingestion of CT logs

- Full-string subdomain search capability

- Support for an arbitrary number of wildcards in searches

While the current functionality focuses on subdomain discovery, I'm planning to expand MerkleMap into a comprehensive CT log monitor over time.

I'd love to hear your feedback.

[conradklnspl]

Creative way to get domain name monitoring. Do you use this to keep a history of DNS records to, by any chance? If so, do you find it to be reliable?

I would suggest making the domain name clickable: https://www.merklemap.com/search?query=datafragment.com&page... Currently, I need to copy and paste in my browser.

What do I get by signing-up? Even with a very large domain like google.com, it looks like all results are available for free without limit: https://www.merklemap.com/search?query=google.com&page=1

I'm working on https://datafragment.com with a couple colleagues, and domain monitoring is definitely one of the things we'd like to try and sell to prospects.

[Eikon → conradklnspl]

> Do you use this to keep a history of DNS records to, by any chance? If so, do you find it to be reliable?

While that wasn't the initial goal, it's an interesting application I hadn't considered. The project actually started as an attempt to build an IPv6 database, which led me down this particular rabbit hole. DNS record history could be a valuable feature to add in the future.

> What do I get by signing-up? Even with a very large domain like google.com, it looks like all results are available for free without limit: https://www.merklemap.com/search?query=google.com&page=1

Currently, signing up doesn't provide additional features. It's mainly a way for me to gauge interest and identify potential power users. I have plans to add premium features in the future, which will be available to registered users.

> I'm working on https://datafragment.com with a couple colleagues, and domain monitoring is definitely one of the things we'd like to try and sell to prospects.

That's great to hear! I'm open to collaboration. For now, you can use my API at https://api.merklemap.com/search?query=example&page=1. The rate limits are quite generous, but please be mindful of the load, searching, with wildcards across that many entries, as you might guess is _quite_ expensive :)

You can also get the live ingestion data feed using

    curl -N 'https://api.merklemap.com/live-domains?no_throttle=true'

> I would suggest making the domain name clickable: https://www.merklemap.com/search?query=datafragment.com&page... Currently, I need to copy and paste in my browser.

Excellent suggestion, thank you! I'm actually working on implementing this feature. The plan is to make each domain name clickable, leading to a detailed page that will display subdomains, certificates, and other relevant data from the CT logs.

[conradklnspl → Eikon]

DNS history is really interesting for a variety of reasons.

I've seen it used to try and circumvent Cloudflare firewall rules. Some people don't replicate the firewall rules on the servers behind Cloudflare. If they've ever pointed their DNS to their servers directly before turning on Cloudflare proxy and you have that old IP address, then that IP address has value. White hat security firms for instance pay for that information when running audits.

I will sign up, curious to see where this project leads you.

Thanks for the API details, will discuss this with my teammates.