Theoretically, yes. For that to work, you also need everything on your network to never have encountered a google cookie, adwords, Android, Chrome, YouTube, Maps, Google DNS servers, any of various "smart" devices, etc. And hope those Google Global Cache servers living at your ISP are trustworthy.
But I don't believe they can't do much from an IP alone. Unless the ISP starts ratting us out.