Unfortunately this was not the case. For ranking purposes Kojitsu had to crawl every unique link it found which ended up being very resource intensive. There were multiple servers dedicated to crawling, in addition to the app and db servers.
Running Kojitsu taught me a lot about building up technical infrastructure.
Running Kojitsu taught me a lot about building up technical infrastructure.