This is a brilliant summary, thank you.
The best 'agile' experiences I've had are situations where the 'clients' are directly involved, often within the same organization. Instead of a hard scope or deadline, there's just a shared interest in producing a valuable product efficiently, and the users are on-hand throughout the process for feedback and reevaluation.
The worst experiences have been waterfall contracts, developed by an internal simulation of agile. The software team does frequent "releases" to business or management, who provide feedback and feature requests, but the actual recipients are uninvolved outside of occasional demos, or contacted only indirectly by non-programmers. The result is almost always thrashing, with time and effort spent pointlessly satisfying the forms of agile even though the real timeline and customer feedback are unyielding.
I'd say generally IME they are still preferable, but occasionally can be more painful.
We call this wagile
I often sit between the business and tech orgs, so the way I explain it to the business is this: I can get you detailed status reports and metrics, but they will slow progress and be expensive. So think about why you need that information: if there are legitimate business reasons with dollars attached, go for it. If it’s just to soothe your anxiety about the timeline, a therapist will be far cheaper and more effective.
That's just one example of the general value I like to ascribe to "AINO": it gives you a useful mental model of the delta between what you have and a known good process, making it quite easy to name the missing pieces. This usually doesn't make it easier to actually fix their absence, but at least you know where to start with damage mitigation. When waterfall fails, you just throw your hands in the air and say "more of the same".
> The Scrum Master serves the organization in several ways, including:
> - Helping employees and stakeholders understand and enact Scrum and empirical product development;
So, I am not saying that it is an easy job. Many managers just hear what they want to hear, so it can be quite difficult. But if your team has such issues, be sure to support your Scrum Master with good arguments to help him make managers understand what it means to use Scrum (and become Agile).