I hate the thought of worrying about the future ramifications of my searches. For example if someone asks me to check the time of a sports ball game for them will Google Now nag me with sports scores I don't care about in the future?
I'm constantly frustrated by this. I use incognito browsing way too often. Then when I come across something by accident that I do want in my history (either for searching through history later or because chrome occasionally crashes), I have to copy the link over to a normal-browsing chrome window.
It's not exactly convenient but it's possible to go to https://www.google.com/history/edit and edit your search history, in case this becomes more of an issue in the future.