But that's an uninsightful distinction. There's no real difference between the experience of a bu'er sayer and an arabic speaker. If the sounds are noticeable to native speakers, calling them merely phonetic is simply a dubious scientific analysis not representative of the real world.
There's a real difference in that for a speaker of a language where those are two allophones of the same phoneme, butter and bu'er mean the same thing; they're the same word pronounced differently. For a speaker of a language where the difference is phonemic, they're different words pronounced similarly.
That matters if a speaker of the former language tries to learn the latter. It's hard to learn to reliably make a phonetic distinction that has never mattered for comprehension before. See also Japanese speakers struggling to distinguish "rock" and "lock" in English.