If you write your styles in the same preprocessor as the Bootstrap package you chose (Less by default, but optionally Sass), you can mix Bootstraps grid classes into your own semantically meaningful classes, and drop all the col-xs stuff from your markup. Your markup elements can have class names that are purely semenatic. Almost every modern grid system offers preprocessor mixins that can be used in this way.
There are methodologies like BEM that are great for providing a semantically meaningful layer of classes in your markup.
It's true that the complexity remains and is shifted. The important part is that it's no longer interlaced with the complexity that describes the fine-scale structure of the page.
There have been some huge deficiencies in CSS over the years. It is frustrating to have to use complex sets of counterintuitive rules to achieve seemingly simple effects. But there are techniques for helping to manage it in modern web development.