I'm generally not too terribly much of a stickler for accessibility, but a thousand times, yes. Unclickable labels are a GIANT pet peeve of mine, both as a user and a developer.
What's great about "accessibility problems" is that they're usually huge annoyances for everyone else too. Curb cuts are essential for bicyclists, closed captioning at sports bars, etc.