People's reactions may be "I like the final covers better". I get that, I think I do too. But I suspect that's because they're more like what we expect from book covers. They're safer and less experimental. A lot of times our aesthetic evaluations are just based on what seems more familiar—that 'mere exposure' effect—and we find post hoc reasons to justify a gut reaction. I'm not even sure that's a bad thing, from the publisher or the author's perspective: they just want a quick visual shorthand to tell people that a book might be for people like them. They're not trying for much more than that. But I want these designers to not be that cynical, and keep trying new things even if they just run up against brick walls, because that's the mechanism that slowly evolves visual design.
Some of the covers exhibit a fairly common problem with designer/artist work: That it's more made to showcase the mastery of the designer rather than fit the marketing for that author. For example "that author usually has some piano related cover. I know: I'll do the opposite." There is rarely a need for "experimental" (sometimes but rarely) - and for sure the artist shouldn't be too disappointed when their genius idea gets rejected (and they should have also submitted a more on point design.)
Then again the marketing brief in some projects is non-existent. The designer should prompt for some but at some point does what they can.
I think it is important that the designer gets a feel for the character and “vibe” of the book by actually reading the manuscript and gathering as much context information as they can, ideally in direct exchange with the author(s) to align with their artistic vision. Book cover design can be an art in itself and as a reader, I appreciate it if it is done well and communicates some of the subtleties and nuances of the content in an interesting way. Of course, as you mentioned, it should not be about the ego/self-marketing of the designer, but that does not exclude artistic expression.
I believe it is a disservice to the author and the reader to focus solely on the marketing side of things. After all, the design of the book is part of the product, which, as a whole, sets the frame for the reading experience. Physical books today have to compete with digital content and ebooks, so as a reader I would expect that more effort is being made to make a product that also looks and feels good and not only carries the content to my eyes. That also applies to the layout and typesetting of the inner pages, which is unfortunately often careless and not aligned with the cover design (let alone the choice of paper and binding, which is of course seldom in control of the designer).
Physical books compete with digital versions - that would be a reason for LESS budget for the paper cover. Some has to be devoted to the digital marketing.
Some authors have the luxury of going all in on the artistic. That does exist. For most though, the book needs to sell or there won't be a next book. Constraints like that are what a marketing or design brief is supposed to explain. It's stuff the publisher understands long before they call in the cover artist or layout designer.
I'm with you on the surprising choices we often encounter. But then these problems were common when paper was king. The distractions of the move to digital won't help. For example, far too little margin next to a stiff binding. For example, ebooks with ludicrous epub rendering (on book readers that don't sit on the publisher's desk). Puzzling.
