My tests keep failing until I fix all of my code, then we deploy to production. If code fails in production than that's a problem.
We could say that rockets are not code. A test run of a Spaceship surely cost much more than a test run of any software on my laptop but tests are still tests. They are very likely to fail and there are things to learn from their failures.
You test components in isolation, you test integration of components, you run simulations of the entire rocket, and finally you test the rocket launch.
You’ll catch issues along the way, but you can’t catch all of them before a full launch test. That’s why there are launch tests.
This can get as far as the test plan is complete, multiply iterated under different interface conditions and thorough. And you are still relying upon the adherence of the simulated models to the physical reality.
Real tests do all of this at once with no option to escape reality.
Again, one thing is automating thorough software tests, another one is testing physical stuff.
That might still happen this year, it’s the next step in the development plan.
What makes these launches “non-production” tests is that they are not carrying any valuable payload. Blowing up rockets like this is exactly what gives the company it’s advantage over competitors who try to anticipate everything during design stages.
Testing their ability to deploy satellites is a short-term goal that will make them money now. Testing refuelling will be needed for Luna and Mars missions, but that’s a long way off anyway.
This has been SpaceX’s methodology for a long time now and has gotten them to the point where they have the most reliable western launch vehicles ever launching record amounts of mass to orbit each year at record low prices.
I truly hope that if you ever design a rocket yourself, that you will test it. I have no idea why you'd think testing is a terrible thing to do if it has to do with rockets.
I think we all agree that you need to test eventually. I do think most of us would already be double checking for leaks. It just seems one of the obvious things that may go wrong when putting it all together.
They likely did test it, and it passed. The leak was probably caused by the somewhat violent environment of the launch, and that can’t be entirely replicated on the ground.
My tests keep failing until I fix all of my code, then we deploy to production. If code fails in production than that's a problem.
We could say that rockets are not code. A test run of a Spaceship surely cost much more than a test run of any software on my laptop but tests are still tests. They are very likely to fail and there are things to learn from their failures.