With only 17 hours caffeinated to go before a weekly meeting with my research advisor, and desperate to create some results to show, I accidentally reinvented the Rabin-Karp algorithm for string search:
My thought process at the time was: Shit, brute force string search takes over ten minutes on my data! There must be a way to speed this up. Hm. It probably involves hashing somehow...
Yes I realized that, my question was more like, "couldn't you just put the data in a relational database or a full text search engine and avoid reinventing?".
Depends on the scenario, of course, and judging by how you solved it, yours seems like an exception, and not rule.
http://stanford.edu/~jonr1/software-faire-poster.jpg (see center panel)
My thought process at the time was: Shit, brute force string search takes over ten minutes on my data! There must be a way to speed this up. Hm. It probably involves hashing somehow...