I've been in the engineering industry for more than 10 years. I am baffled by how many people there are who don't understand the importance of iterative development but would rather some version of getting the perfect plan out. I have to ask, when in your career did you realize the importance of iterative development
In high school with my first attempt at a significant project (a data analysis tool for a high school science fair, significant for me at the time but probably only about 10-20k SLOC of C++). I ran into a hitch in my design and had to scrap it a couple times, eventually arrived at bottom-up development (a term I didn't know yet). Relearned in college in my 3rd CS course (GT, CS 2130) when I had a grand design for my compiler and realized that it was a clusterfuck. Scrapped it, hand wrote a bunch of code over a day or so, and started coding with more regular testing (not quite TDD, but definitely in that vein) and significant edits (rather than commitments) to the plan.
Then every major project I've worked on as a professional. Iterative (or processes incorporating regular V&V feedback loops) projects have almost all been successes. Almost always on time, on budget; if they were delayed it was almost always an external factor (hardware unavailable, hardware was available but wrong, requirements changed mid-project). The BDUF with minimal feedback loops have almost all been late if they were of any significant size, sometimes with significant multi-year, multi-million (approaching billion) dollar overruns.
The main thing I've learned, plans are not commitments. They are useful and maybe even essential, but you must incorporate real-world results into the process and update the plans (potentially totally scrapping them) as you gain insights. Iterative development is just an acknowledgement of this.
On smaller projects, you can possibly create the perfect plan. In a domain you know well you may succeed with somewhat larger projects. But major projects rarely succeed by starting from a perfect plan and committing, because the plan will almost never be perfect. More likely, they succeed despite that "perfect" plan because unofficial feedback loops develop, even if they're not officially acknowledged in any discussions or write-ups about the effort.
Then every major project I've worked on as a professional. Iterative (or processes incorporating regular V&V feedback loops) projects have almost all been successes. Almost always on time, on budget; if they were delayed it was almost always an external factor (hardware unavailable, hardware was available but wrong, requirements changed mid-project). The BDUF with minimal feedback loops have almost all been late if they were of any significant size, sometimes with significant multi-year, multi-million (approaching billion) dollar overruns.
The main thing I've learned, plans are not commitments. They are useful and maybe even essential, but you must incorporate real-world results into the process and update the plans (potentially totally scrapping them) as you gain insights. Iterative development is just an acknowledgement of this.
On smaller projects, you can possibly create the perfect plan. In a domain you know well you may succeed with somewhat larger projects. But major projects rarely succeed by starting from a perfect plan and committing, because the plan will almost never be perfect. More likely, they succeed despite that "perfect" plan because unofficial feedback loops develop, even if they're not officially acknowledged in any discussions or write-ups about the effort.