There are still some old languages (e.g., legacy FORTRAN or COBOL; IBM JCL) that are all uppercase.
And I'd guess there are still some old data-entry applications that use uppercase only.
There are also headings in all-caps on certain legal documents.
There are even less useful keys, like the "Scroll Lock" key (and its associated LED) on the standard PC keyboard. This serves no function that I know of in any modern Windows application, but enough old programs use it that manufacturers don't want to get rid of it.
The button exists because people do use it. If I need to write a sentence in capitals - generally labeling/headers/outlineish stuff - it is Much easier to use than to hold the shift key for each and every letter.
Now, as far as the standard position, I agree. I'd be fine if the standard were in a more out of the way location, because I rarely use it in a hurry and happen to do enough accidentals to turn on a sound letting me know it was hit. I'd personally replace it either with symbols (non-english punctuation, perhaps) or use the space for one of the more common non-english letters (ø,æ,å,ô)
Optionally you can set your keyboard to use English - International to type Pokémon properly. Or português. Although it can get annoying trying to type an opening quote " because you need to press space afterwards (it's waiting for the following character to modify: ä)
And I'd guess there are still some old data-entry applications that use uppercase only.
There are also headings in all-caps on certain legal documents.
There are even less useful keys, like the "Scroll Lock" key (and its associated LED) on the standard PC keyboard. This serves no function that I know of in any modern Windows application, but enough old programs use it that manufacturers don't want to get rid of it.