I'm talking about a slightly different problem. I'm not suggesting that you might accidentally click to send a reply you didn't want to share, so you reviewing it is beside the point. I'm suggesting that by mining all our emails, it might make a suggestion to me based on something you didn't want to share.
E.g. Someone writes me an email about a rare kink you often talk about. You're the main data point on that kink, so it suggests I respond with something you often say when you talk about this topic, maybe including personal details. It's not a totally precise or realistic example, but with large numbers and complex models, unintended things are bound to happen on occasion. Will those things leak information?
As for your comment that the potential replies are limited in number and as structured as you say, I don't get that from the original post, and it doesn't quite fit with my understanding of the model.
You raise a very important point. I'd hope that there is an actual finite (and relatively small) corpus of approved, manually white listed answers that transcend through Google accounts. You might get personalized options based on what you write most but they would not show for other people. Would that be enough to satisfy this concern?
Yes, it would. If curation is too troublesome for gathering a large enough training set, it might be possible to train a smaller curated network with a higher false-negative rate that flags responses that aren't appropriate (personal info, insults, etc) and removes those from the training set.
You could also do something like googlebombing. Have lots of people send each other the same question, and all of them reply with the same/similar response.
E.g. Someone writes me an email about a rare kink you often talk about. You're the main data point on that kink, so it suggests I respond with something you often say when you talk about this topic, maybe including personal details. It's not a totally precise or realistic example, but with large numbers and complex models, unintended things are bound to happen on occasion. Will those things leak information?
As for your comment that the potential replies are limited in number and as structured as you say, I don't get that from the original post, and it doesn't quite fit with my understanding of the model.