If you want to save the full format of the file, including layers and whatever other GIMP magic you've created, you'd best save it in GIMP-native format.
If you're just doing some quick tweaks, you'll get used to the export option and ignore the prompt to save on quit.
Since most of what I do are quick edits of screenshots, the latter works fine for me. If I were a graphic artist, I'd probably get more than slightly peeved if my afternoon's work were lost due to an incompatible file format save w/o prompt.
Not prompting the user leads to massive loss of user work and state. Prompting the user is a minimal overhead to avoid this risk.
Considering it's a relatively recent change, it certainly has come as a suprise to lot of users. Did to me. Did to my wife. Bit at least a couple of friends I know.
From a technical point of view I can certainly approve the change. Back in my time I've had couple of my images ruined by badly timed save (user error [tm]), so the non-destructive approach certainly has its merits. However, the new behaviour breaks a lot of ingrained casual use flows. So maybe the best approach would for Gimp to remember the original filename when it's opened and if the filetype is anything except XCF, simply do two saves: create a new $filename.xcf automatically with all the changes the user has made, and also auto-export with the old name.
That way the more casual users could still just do "open, edit, save" for their quick edits. As a bonus, since the XCF file is non-destructive, the original file would still be in there. This might give the best of both worlds: it wouldn't break existing workflows, and it would automatically prevent accidentally changing the original image. Plus it would make the save/export split less of a headache for new users.