Besides the cultural taboo that some non-agricultural societies had, there's the European fear of new world tomatoes that persisted from the 17th to 19th centuries. Wealthy folks ate the (acidic) fruit on pewter platters, the acid leached the lead from the pewter, and 10 generations of Europeans feared the fruit! If someone can explain why this fear didn't hit lemons, wine, and other acidic consumables, I'd love to know.
Less directly related, but more way more hilarious, is the tendency of babies to fear plants. It must be something hard wired. Even soft leaves that could not possibly irritate their skin elicit a strong response. If you have a baby in your life, take them out to a garden and watch, it's something to behold.
If you don't have a baby in your social circle, there are lots of videos online of infants (who can scarcely control their movement) contorting themselves to avoid contact with grass.
