Only through sync, which you can encrypt with a separate password to your Google account.
My sync is encrypted so I can't test this for you but I believe if it's not, you can check (and clear) your history here: https://myactivity.google.com/myactivity
Ah I was wrong about this!!!
SafeBrowse uses hashes.
Edit: never be afraid to admit your were wrong folks :) So it only gets a “potential list” of sites you visited. Wonder if the operators could aggregate the data enough to deanonymize things
My sync is encrypted so I can't test this for you but I believe if it's not, you can check (and clear) your history here: https://myactivity.google.com/myactivity
Google's privacy policy seems to make it clear that this happens unless encryption is turned on: https://www.google.com/chrome/privacy/index.html#signed-in