Interesting. Could be because it is somewhat stale, maybe. (Was it bought from stores, what you tried?) Same happens in India. Fresh home-made buttermilk tastes good, from just after being made up to say 24 hours or so. After that it does get more acidic, particularly in warmer weather or if not kept in the fridge, although I might call it sour instead. Not found it tasting bitter though, as I remember. Could be you had bad luck and it was quite old stock.
A quick cooking hack when you're out of buttermilk at home is to add a tablespoon of vinegar or lemon juice to a cup of milk. So I always assumed the sourness/acidity was the defining feature of buttermilk. It would be interesting if there were types of buttermilk (and recipes requiring them) where this was not the case.
Interesting, must try that out sometime. In India paneer (a sort of Indian cottage cheese) is made that way, by putting lemon juice in hot milk, which makes it curdle. Then the curds are put into a cloth and hung up for a few hours to drain off the whey. What's left is paneer, which can be eaten raw, or used in dishes like palak paneer (spinach with paneer), mutter paneer (green peas with paneer), some Indian sweets, etc.