The current[1] WAI-ARIA spec is to blame for not clearifying this intend. However, the W3C Editor's Draft of WAI-ARIA 1.3 of 26 October 2022 attempts to do so:
"In the cases where DOM content or a tooltip is undesirable, authors MAY set the accessible name of the element using aria-label, if the element does not prohibit use of the attribute. [...] Authors MUST NOT specify aria-label on an element which has an explicit or implicit WAI-ARIA role where aria-label is prohibited."[2]
That doc lists only a few elements where aria-label should not be used: caption, code, definition, deletion, emphasis, generic, insertion, mark, paragraph, presentation, strong, subscript, suggestion, superscript, term, time
I am particularly concerned if people interpret the parent article to mean that aria-label shouldn't be used on certain semantic containers that are exposed in the accessibility tree by screen readers, such as <section> and <nav>.
"In the cases where DOM content or a tooltip is undesirable, authors MAY set the accessible name of the element using aria-label, if the element does not prohibit use of the attribute. [...] Authors MUST NOT specify aria-label on an element which has an explicit or implicit WAI-ARIA role where aria-label is prohibited."[2]
[1] This is WAI-ARIA 1.2 (and not 1.1): https://www.w3.org/TR/wai-aria-1.2/#aria-label
[2] https://w3c.github.io/aria/#aria-label