
CRS-7 Launch Update - chiachun
http://www.spacex.com/news/2015/06/28/crs-7-launch-update
======
beltex
_" Cause still unknown after several thousand engineering-hours of review. Now
parsing data with a hex editor to recover final milliseconds."_

4:09 AM - 29 Jun 2015

[https://twitter.com/elonmusk/status/615431934345216001](https://twitter.com/elonmusk/status/615431934345216001)

~~~
colomon
Ugh, now imagining trying to debug a really sneaky bug that costs $100 million
every time it triggers...

~~~
drzaiusapelord
Stupid question perhaps, but don't we have the ability to 100% simulate the
rocket physics and the code it runs on in a simulation environment? Or are
they just punching out code, shoving it onto rocket controllers, and testing
in the real world?

I'm curious as to the QA system used here. I imagine with proper simulation
this should have been catchable. I wonder if SpaceX's low cost approach means
cutting certain corners and situations like these where catchable issues make
it into the wild because of the difficulty of rocketry in general with the
added difficulty of cheap spaceflight tacked on.

I really hope they didn't just find themselves in a STS-51-L moment where
it'll take months to truly iron out the root issues. Thank goodness there was
no loss of life and SpaceX's stack isn't man rated yet.

~~~
nkoren
Rocket physics are actually relatively trivial and, yes, can be fully
simulated (and are). Rocket _plumbing_ is always the hard part. Perhaps the
threads on some connector got stripped as it was screwed together. Perhaps
there was a manufacturing flaw or unexpected bit of corrosion in a
particularly vital bolt. Perhaps a bit of contamination in a fuel line caused
something catalytic to happen. Perhaps a bit of excess H20 condensation in a
LOX valve caused an ice dam to form and the supply line to over-pressurise.
These are the sort of problems which bring rockets down: really damned
complicated plumbing problems.

~~~
drzaiusapelord
I guess my point is, can we do a full simulation of every screw, material,
plumbing, liquid dynamics, weather dynamics, etc and augment those with known
fail scenarios and other fuzzy data to build out a real world KSP that
predicts fails reliably? We should understand how things like corrosion and
condensation work on a rocket engine. Considering the low cost of incredible
amounts of CPU power, granular level simulation is possible on a certain level
today if someone wanted to create it. We certainly see this kind of thing with
stealth technology, where we can simulate every permutation of near every
radar photon hitting the various surfaces of planes with various materials,
scenarios, temperatures, etc.

I imagine this level of simulation might not be entirely feasible yet. Maybe
for the lack of trying or budget. In a growth industry or one powered by both
commercial and technical pressures, it may be difficult to sit down and build
something like this out. From a more practical point of view, it may make
sense to just let things explode than spend years running expensive
simulations instead of building things, launching, and collecting paychecks.

~~~
HCIdivision17
I think a critical part of his point is that what you build and what you model
are neccessarily two different things. You can totally simulate it, and then
when the rocket inevitably fails, you can go "aha! This bit deviated from the
simulation!" But you can't feed that forward beforehand to prevent the
failure, since you can't _expect_ random acts of poor workmanship or crafting.
You can only prepare and hope (and you can do that with shockingly high
confidence - but rockets are more than equally shockingly complex.)

Part of what makes the idea of bringing the first stage back to the pad so
important is that we so rarely get to use the same engines multiple times in
the field (where all the really nasty reality checks are done). Being able to
reuse stages allows us to far better model how they will perform in the
future. Otherwise, we're using test beds to feed parameters into sims to
inform our launches; it's good practice, but more physical evidence is always
better.

~~~
ethbro
This.

Engineering is what hopefully guides reality up the correct branch of a
theoretically possible tree.

You can simulate most of each one of those branches. But what are you going to
do with a million simulation results? How does that guide your course of
action? What do you do differently?

If this was an engineering or assembly defect, the answer is always going to
be "Don't do that next time." If it was a design defect, then the part wasn't
simulated (unlikely) or our understanding of how it operated in this design
was incomplete (more likely).

The trick with rocket science is that the design tolerances are by necessity
very tight. Physics dictates this with chemical propulsion. Every part you
over-engineer in a weight-increasing way decreases the weight available for
payload. And there isn't very much weight there to start with...

------
sokrates
As a non-native English speaker, I find the omissions of "the" very prominent
and interesting in SpaceX speech. "shortly before first stage shutdown",
"resulting in loss of mission", "139 seconds into flight", "some period of
time following separation", "data to determine root cause" \-- is this a
general theme in engineering or journalism? I wonder what linguists have to
say about this.

~~~
DanielBMarkham
Not a linguist but a writer here who loves language.

I'm seeing a lot of folks dropping articles. It's also becoming quite common
to eliminate pronouns at the beginning of sentences, i.e., instead of "I went
down to the store" you write "Went to the store"

I use this purposefully to jar the reader. I have no idea what the underlying
linguistic reasons are.

~~~
rezistik
Implied pronouns, perhaps a trend from Facebook usage? On Facebook and Twitter
you have an implied "I" that makes saying you did something redundant. This
could transfer over to other formats of conversation.

~~~
nitrogen
Other languages are a bit ahead of English in that regard, but their
conjugations make it clear who the subject is.

------
dabeeeenster
"Preliminary analysis suggests the vehicle experienced an overpressure event
in the upper stage liquid oxygen tank approximately 139 seconds into flight.
Telemetry indicates first stage flight was nominal and that Dragon remained
healthy for some period of time following separation."

Sounds like they are trying to suggest that if there had been people in the
capsule they would have survived...

~~~
mrfusion
Also the first stage might have survived and returned to landing if it had
known about the trouble and detached earlier? Or am I misunderstanding?

~~~
lmm
If the second stage just hadn't been there then the first stage might have
survived and landed. But I'm not sure there is any way to "detach early", and
you'd have the problem of having too much fuel left over in the first stage. I
doubt it's designed to land in those circumstances (why would it be?), so it
probably couldn't.

------
arianvanp
Just after the pressure event you see Dragon being slung off the rocket... If
only it was possible to deploy the parachutes during launch we could've
hypothetically saved the payload.

I wonder if the launch abort mechanism on the Dragon V2 would've been of any
help here too to jettison it safely away from the rocket.

I read the rocket was around maximum dynamic pressure during the event (Or
just after?) and I'm not sure if it would stand such forces of a jettison
during such time.

~~~
VLM
WRT max-Q its a meme that won't go away because the most "interesting" thing
on the public PR timelines before the incident was max-q. However dynamic
pressure drops off extremely quickly due to altitude (low air pressure) and
max-Q was a good while before the incident, so it wasn't a direct issue. It
could obviously be an indirect issue if something started to buckle a minute
previous or shook loose and for whatever reason a minute later it burst,
perhaps as pressure built up or something.

The other meme is there are or have been rockets or overall systems with
flight profiles and designs that have unsurvivable portions of the flight. Or
only extremely theoretically survivable. Think of the old shuttle system, for
example. A RTLS abort was theoretically survivable, but lets be realistic
here... However the space-x guys are extremely proud that they designed an
overall system that has no unsurvivable by design flight portions, and also
very proud that they did a test flight with a separation near max-q
specifically to prove it would work just fine even at max-q...

One interesting problem with a structural failure at that speed is it could be
hard computationally to tell the difference between some irrelevant pogo-ing
or vibration vs 50 ms later half the rocket is flying sideways, at which point
it might be unsurvivable. Bad car analogy is I can jump out of an airplane
with a parachute at 100 MPH and all turns out just fine, but randomly getting
tossed out of a 100 MPH car isn't going to likely end very well even if under
ideal conditions its no big deal.

~~~
DavidSJ
> the space-x guys are ... very proud that they did a test flight with a
> separation near max-q specifically to prove it would work just fine even at
> max-q…

The in-flight abort test for Dragon v2 is scheduled to occur later this year.
It hasn't occurred yet.
[https://en.wikipedia.org/wiki/Dragon_V2#Flight_testing](https://en.wikipedia.org/wiki/Dragon_V2#Flight_testing)

Also note that Dragon v2 was not on this mission and won't fly to space for a
while. This was a Dragon v1 mission. Dragon v1 is unmanned and has no launch
abort capability.

~~~
VLM
You are completely correct that they haven't flown it. When I wrote that, I
must have been thinking about the successful pad abort a couple months ago.

------
phkahler
My thoughts on this. Overpressure in the LOX tank could be due to 1) failure
to vent properly or 2) something heating it up.

Option 1 is based on the fact that it's constantly evaporating and needs to
vent. This seems really unlikely given that it passed all test on the pad 2
minutes before. It also sounds like this was likely ruled out based on Elon's
tweet about a "counter intuitive cause".

Option 2 sounds like fire or flames due to fuel leakage or something, but then
realize that this is the second stage and all the action is going on 100 or
more feet down in the first stage. They also have a camera on the 2nd stage
engine which was shown shortly before the incident and nothing was going on in
there. I wonder what the in-tank camera showed.

It seem that to get the extra energy into the tank, something must have fired
up early. But if there's one thing Spacex seems to have a lot of it's data.
Aside from a breech letting external air in (like the last shuttle accident),
how do you get enough added energy into a tank to build pressure to the
breaking point? In 2 minutes.

~~~
JshWright
There are plenty of other scenarios...

The IDA is the heaviest thing they have ever carried in the trunk, AFAIK.
Maybe its mounting bracket failed and it impacted the top of the second stage,
buckling the LOX tank. A suddenly induced crack allowed some LOX to escape
(the initial 'puff'), and the suddenly reduced pressure allowed the rest of
the LOX to boil off and the tank to BLEVE, causing the catastrophic failure of
the second stage.

Obviously this is a completely theoretical scenario, but it's one of many I
could dream up...

EDIT: While is is a fairly technical discussion, I realize I got a little
heavy handed with the acronyms there...

    
    
      IDA: https://en.wikipedia.org/wiki/International_Docking_Adapter
      LOX: https://en.wikipedia.org/wiki/Liquid_oxygen
      BLEVE: https://en.wikipedia.org/wiki/Boiling_liquid_expanding_vapor_explosion

~~~
Serow225
Interpreting Musk's tweet about 'counter-intuitive' meaning BLEVE would make
sense to me.

~~~
JshWright
Hmm... perhaps, but that seems pretty 'intuitive' to me.

------
dlgeek
For those who are also having issues loading the site: Nothing that wasn't
already released via Twitter or yesterday's press conference.

------
chaostheory
... and people think debugging pure software problems are hard.

