Hacker News new | past | comments | ask | show | jobs | submit login
The Air Force is having to reverse engineer parts of its own stealth bomber (thedrive.com)
467 points by alrs 48 days ago | hide | past | favorite | 250 comments



I can't be certain, but I suspect the real problem was that the original contract was signed without requiring the technical data rights and/or delivery of the technical data. This is a common mistake made by the DoD, and can lead to extraordinarily high costs later.

Just look at the notice: "The deliverables will include all technical data related to the heat exchanger disassembly, all technical data related to the cores, all technical data related to the stack up, vacuum brazing, and installation of the cores on existing units. The deliverables will also include all technical data related to the tooling needed..." It's clear they don't have the technical data.

Here's how this can happen: the government asks for proposals, and the prospective contractors say "we'll create it for you for cheaper if you don't require the technical data & its related rights". The government program manager (PM) is under a lot of pressure to "get it cheaper", and this is cheaper in the short term, so the government PM agrees. The PM will be around for a few years, then leave. Later on, the contractor is the only one who can make the parts (because they're the only ones with the technical data & the technical data rights to do it). The government procurement system is all about competition, but now there is no possibility for real competition. Thus, the parts will cost 10x or more, and the government PM who approved this is long gone (probably with a promotion), and the contractor is happy with incredibly large profit margins.

I don't know if that's what happened in this case. But it's very very possible.


This reminds me of the following:

“In my view, shared by many blue-suiters, this marvelous airplane should still be operational but, alas, that was not to be. One of the most depressing moments in the history of the Skunk Works occurred on February 5, 1970, when we received a telegram from the Pentagon ordering us to destroy all the tooling for the Blackbird. All the molds, jigs, and forty thousand detail tools were cut up for scrap and sold off at seven cents a pound. Not only didn’t the government want to pay storage costs on the tooling, but it wanted to ensure that the Blackbird never would be built again. I thought at the time that this cost-cutting decision would be deeply regretted over the years by those responsible for the national security. That decision stopped production on the whole series of Mach 3 aircraft for the remainder of this century. It was just plain dumb.”

Ben R Rich and Leo Janos. “Skunk Works: A Personal Memoir of My Years at Lockheed”.


My heart hurts reading that.

I felt something similar walking through Weird Stuff Warehouse and seeing old SGI workstations that I used to covet sitting on shelves for a pittance. Even if you bought them, you probably didn't have a 13w3 monitor or software to get them up. and now weird stuff is gone itself.

It's that "it's gone and can't come back" feeling.


https://spectrum.ieee.org/view-from-the-valley/tech-history/...

When we look backwards in time, we see lots of tributaries that dried up for various reasons. That SGI workstation is still in your head, it is really up to us to copy and extend those concepts forwards in time.

That software is still sitting on a box of junk in someones basement somewhere, it just has to be found.


That software is sitting in an SGI in my basement :-)


That's a sad one too. it's hard to quantify the loss of an ecosystem.


Old SGI workstations and the things people make with them....I loved the surplus you could buy in the Vally in the early 90s.

https://www.youtube.com/watch?v=NPvIs0I-gmg


It’s risk reduction.

That was an innovative plane and the reality is that nobody is going to wake up and build it again once production stops. Once you break up a team, the team is gone — that’s why we always build submarines, even if unnecessary. You have to keep the people and plant working.

That tooling was going to be compromised and useful to the Soviets or some other adversary. There is a well established record of traitors selling plans or materials to the KGB/FSB/Chinese intelligence that have been caught. Also, from a PR perspective, a headline like “Air Force spends $50M/year to guard stealth bomber tooling untouched in 30 years” isn’t great either.


This was talking about destroying the physical tooling. Ain’t nobody sneaking 60,000 tools out the back door to the Soviets. Many/most of those tools are going to weigh many tons, would require rigging to lift and flatbeds to move. Could they easily sneak out tooling drawings and schematics? yes probably could and did. But not the physical tools themselves.


It was also keeping the manufacturing process and technologies, specifically stealth secret.


I don't understand this. I worked on the B-2 back in 1987-88, and two things about that program don't add up to having to reverse engineer anything.

First, the B-2 was the first large, complex product in history to be entirely developed digitally. Everything. was in CAD, and only the CAD geometry was authoritative. All drawings printed out were VERY clearly marked warning of this. IIRC, most of this was in Catia, so it should still be possible to read and use that data.

Second, if there's one thing Defense procurement Is good at, it's configuration management. This is doubly true of black programs like the B2, where not only do you want to make sure you never lose data or docs, but also that no bad guys ever have a way to get that data.

Either something went badly off the rails somewhere, or someone needs to rebuild something without the proper clearance. Otherwise, this makes no sense. BTW, the combo of the first complex CAD-only product multiplied by the hideous costs of any black program were largely responsible for the B-2's exorbitant costs.

Congress was stupid to cut B-2 production numbers, though, because the lion's share was fixed, sunk cost that remained the same regardless of whether one or a hundred airplanes were actually built.

(I think I've posted here before about how we spent many millions of dollars to develop a method to repair a major composite part in place. That part was pretty much what you started with when you put the airplane together, so even that was way cheaper than disassembling and reassembling the entire airplane!)


The same was done the Canadian Avro Arrow plane, and the British Black Knight rocket. What happens once you join government, they take away your brains?


The government doesn't have consistent communication between and even within departments, that's all really. There is no holistic understanding of things, it's just a conglomerate of BoundedContexts without good interfaces to talk to each other.

I mean if instead of discussing a topic freely on HN you were supposed to share your information using some I-1969 standard form with sections of one word answers, how many compelling and sensible posts would we usually see.


F-14 as well. In all of those cases, it was decided that the cost to secure and account for the tooling to prevent them from falling into the wrong hands was more than they wanted to spend.


How smart people behave depends a lot of context and incentives.

Many people are reasonable shrewd with their own money, and about stuff that directly impacts them. But people in charge of government organisations often have very different incentives and concerns.

(Other big organisations in the private sector can also go haywire. Especially when the threat from competition is low.)

Compare https://en.wikipedia.org/wiki/Public_choice


The Avro Arrow being destroyed is a meme in Canada, but there's a lot of questions at least in the historical community on whether that was a blessing in disguise. The state of the art was rapidly moving away from bombers coming over the North Pole (which an interceptor like the Avro Arrow) which would have rendered a fleet of Arrows expensive and useless.

That being said, it destroyed a large part of the Canadian aeronautical industry since Avro ceased operations soon after and the well trained workforce was scattered to the States. There's a long history of Canadian governments doing shortsighted things like killing industries and then paying out of the nose to try to restart the industry every generation/ See shipbuilding in Canada and how Canada can't build ships for its own navy now, despite being one of the largest shipbuilders in the world in the early 20th century.


With a globalized economy, does it make sense for every country to have a manufacturing base for every industry? Especially for a country with small/moderately sized population like Canada?

I agree though, the government has been short sighted and procurement of ships, planes has been a debacle for years. It seems that either way a lot of money is going to be burnt, either by propping up a non-competitive industry locally, or burnt buying something insanely expensive like F-35s from an ally.


For critical defense needs, likely. If you can't meet those needs you are beholden to the people who can, and they may not have your best interests in mind.


The Avro Arrow and Black Knight were just prototypes whose movement into production could only be justified for nationalistic reasons. They did not have a proven track record or offer cost savings. They aren’t comparable to the blackbird at all, and destroying their tooling isn’t comparable either.


I remember a trip from school in the late 70's to Cranfield university and they had a TSR2 airframe sitting sadly at the back of one of the hangers.


Probably someone was bribed by KGB to do it.


Robert McNamara as a deep-cover KGB "illegal", there's an 80s espionage potboiler I'd love to read. I'm honestly a little surprised no one actually wrote it.


Reading the article I immediately thought of the blackbird and wondered if it had suffered a similar fate.


China didn't steal a copy did they? Ah got the dates wrong.


That would be great because then you could just order the parts on Alibaba


Pls


s/China/URSS/


s/URSS/USSR/



Ah, the famous opponent of OTAN...


s/USSR/CCCP

I think this is why people like to refer to the Communinist Party of China (CPC [1]) as the CCP, because it evokes memories of the Soviet Union.

[1] cpc.people.com.cn


s/CCCP/СССР/ - Latin C and Cyrillic С (and P/Р) are different unicode characters, see also https://en.wikipedia.org/wiki/IDN_homograph_attack ;)


SSSR


> The government program manager (PM) is under a lot of pressure to "get it cheaper"

And let me guess, the 'pressure' comes from government decision makers that live in a perpetual revolving door with industry?

https://en.wikipedia.org/wiki/Revolving_door_(politics)


While the revolving door is always a potential issue, and I'm sure it happens, I don't think that's a key problem for the DoD. When government officials leave office they're required to not do work, for a period of time, on things they controlled, and the penalties can be steep. That reduces incentives for the worst of abuses. Also: a US government employee who retires after 20+ years has a guaranteed pension; it's nice to get more on top, but not by risking it.

No, I think the pressure primarily comes from the DoD procurement process itself. First, you have to develop the requirements. The Air Force wants a plane, the Navy wants a plane, the Army wants a plane. If you say "yes" to one & "no" the others you get a political battle. A bright person says, "I know, we'll make a plane for everyone, and the volume purchase will make it cheaper!" "Great!" everyone says. They each bring in their laundry lists of what they want the plane to do... if you say "no", then they probably won't fund it, so you have this HUGE list of requirements. Now the plane (or whatever) has to do everything.

Then it goes out for bid. Only the lowest bid will win, so contractors will do their best to make it a low price for a massive number of requirements. They're under great pressure to underbid the real costs, and when you do new things it's usually more expensive than you imagine. Yet those costs will still look eye-wateringly big (because it's trying to do everything). Contractors will then say "don't require that 'bunch of useless technical data and those technical data rights... it'll be cheaper!". In addition, every time you learn something, you'll need to make a change, and that will result in a change request with a remarkably high price tag. That's partly because the contractors are motivated by profit, but to be fair, they were often pressured into unrealistically low starting costs by the process.

Many of the DoD's problems aren't from "evil government officials" or "evil contractors". Many really are trying to do good things. Sometimes the problem is that the DoD procurement process has some perverse incentives that sometimes results in very strange outcomes.


The whole "Has to do everything to get funded" type problem, was very apparent with the Army's Bradley Fighting Vehicle[0]. The result was so misguided, if not comically bad, they made a movie about it, Pentagon Wars[1]

[0] https://en.wikipedia.org/wiki/Bradley_Fighting_Vehicle

[1] https://en.wikipedia.org/wiki/The_Pentagon_Wars


The "has to do everything to get funded" is definitely not a new problem, it's structural.

"The Best of the Journal of Irreproducible Results" (January 1, 1983) includes an article by William R. (Randy) Simpson about Murphy's Law, where he notes in a haha-only-serious way the many predictable problems that happen because of it. It's not online as far as I can tell, but the book can be purchased: https://www.amazon.com/Best-Journal-Irreproducible-Results/d...


Seems like if the US govt. can have a policy that disrupts the revolving door, they could also institute a policy that all tenders require the perpetual rights to technical data? Especially if this is a problem that even outsiders are aware of. So I wonder what's stopping such obvious and sensible policy from being enacted?

Edit:

I'll answer my own rhetorical question: because budget forecast time periods are aligned with election cycles.


> Seems like if the US govt. can have a policy that disrupts the revolving door, they could also institute a policy that all tenders require the perpetual rights to technical data? Especially if this is a problem that even outsiders are aware of. So I wonder what's stopping such obvious and sensible policy from being enacted?

I don't think that policy (as written) would make sense for commercially-sold components (which were developed by the company without government funds and are being sold to the general public). Also, I think there are special cases where requiring the technical data isn't warranted. But I agree that policies could be greatly improved.

I think it would be appropriate to require technical data and technical data rights for all custom components by default (e.g., where the government funded part or a majority of its development). Waivers would be allowed, but only after being personally approved by a Secretary or Department CIO. That would provide some actual oversight; if program managers can approve making themselves look good, by lowering short-term costs (while increasing long-term costs after those PMs are gone), the rule will have no teeth. But government personnel want maximum flexibility, and while the current policies increase long-term costs, the policies also make it possible to get promoted when people are asked to do lots with relatively little money. So many government personnel are not very interested in that kind of policy.

A bigger problem is that contractors like it when their customer is stuck in an endless monopoly. Some contractors have been waging a long term effort to keep the gravy train going, and talk endlessly about the DoD "stealing our (contractor) data rights". The "stealing" they're talking about is allowing the DoD to receive the data that the DoD paid to have developed (!). Many contractors think they should have an exclusive right to the things they created using the government's money (and thus took little risk to create). For an example of this kind of absurd thinking, see: https://govcon.mofo.com/cybersecurity-and-data-privacy/break... Don't get me wrong, some contractors are actually quite reasonable, and the DoD cannot be successful without contractors. But contractors have a lot of incentive to gain exclusive rights over what they were paid to develop, and that's a huge incentives problem.

At least one past Secretary of Defense has tried to get more data rights through contracts, but it's very much an uphill battle.

There are some efforts to get more technical data rights in the future. I don't know what their status is, but my understanding is that they are minor nudges towards the government getting more rights to what the DoD pays for. These minor changes are considered by some contractors as the apocalypse, which should make it clear why big policy changes are hard. Here's some information on what was proposed, I don't know what has happened to it (I don't pay as much attention to this stuff as I used to): https://www.jdsupra.com/legalnews/dod-s-new-ip-rules-may-imp... I think these are at best small half-hearted measures to deal with a serious problem.


Thank you for the insightful reply with actual domain knowledge. Here's hoping reform wins out in the longer term. I think there's renewed focus on military spending these days, and this seems like an easy win if the administration is determined and competent enough.


Because it isn’t obvious and sensible.

The whole point of hiring a contractor to sell a product is that the contractor is that it brings its own intellectual property to the table and “owns” the project.

Work performed as work for hire for the US government is public domain (subject to security restrictions). For some military products, the government “owns” the product. For example, artillery/naval gun tubes are manufactured by the Federally owned Watervliet Arsenal in New York, operated by a contractor.

The government owns other facilities relating to nuclear and chemical weapons that I’m aware of, and I’m not sure there is a material difference in competence related to the government “owning” all information.


Just having the technical data doesn't ensure that another contractor can actually build compatible parts.


It's probably a whole lot better than getting a contractor to reverse engineer the technical data and then building the compatible parts.


Mostly the CEOs who want to get their third home.


It's also the fact that they are likely to retire before any possible consequences arise.


It's also possible the government did get all the technical data and then lost it. It has to be somebody's job to archive and catalog that stuff, and the cost of archiving comes out of somebody's budget. The pressure to just toss old dusty paper files to cut costs can be enormous. It might have happened because "Joe Smith was the archivist and he retired 10 years ago and we needed the space."

If the data was classified (very likely) it costs a lot more to store it and audit it, and the pressure to "just burn it" is that much greater.


This is extremely implausible. Files (the non descript name for official records) are extremely hard to destroy even if you want to. Everything about the B2 is in a special TS compartment and will likely be retained until long after we are dust.


Slightly off-topic:

A few years ago I wanted to renovate an apartment. I dutifully applied for the plans from the local council. Only to be told they no longer existed.

I had this, "Well then, it was always like that" moment.

Didn't end up knocking out the wall, for other reasons, but if the plans no longer existed who was to argue that it wasn't always like that.


Configuration management, especially for black projects, is the one thing you can count on from the DoD and its contractors just as surely as taxes and waste.


Yeah, very well might be. Usually so, these contracts include decades worth of spares. Things usually turn ugly when this period runs out, yet 25 years after delivery seems a bit short.

Nice tid bit for the B-52. These old birds, old in the sense of designed pre CAD, are quite hard to understand for modern manufacturing. Even manufacturers rely on retires sometimes to understand these old drawings for their own planes. Then again, the B-2 shouldn't fall in that category.

I suspect that people in the armed forces, at least some, understand maintenance but have no idea of manufacturing. A couple of years ago I had spontaneous "workshop" session where we discussed how we could leverage BoMs and data collected during production and assembly from the manufacturer for version and build tracking of military hardware. I used to work for that manufacturer in question, so I knew what data they collected during production. The only we would have had to figure out was which BoM to use. That was immediately shot down by middle management, because it would cost 10s of millions to get the rights for that data. Nevermind that it was there from day one, even in SAP so our client wouldn't even had issues with compatibility.

And none of the people I had these discussions with had any idea how BoMs and such are used. In these environments it is quite easy for crucial knowledge, especially of highly specialised parts and processes, to be lost.


You entire comment is monstrous bigotry of the modern. The B-2 was in fact the first large, complex product ever built entirely from a CAD database - a factor in its huge cost.

As a CIM Manager for a major B-2 subcontractor, I can assure you they knew what they were doing. Yes, even 30-35 years ago. Yes, they knew as much or more then than we know today about BOMs, manufacturing, CAD, digital records and archiving, documentation and configuration management, and everything else. (BOMs weren't in SAP, but they were in mainframe DBs, ERP systems, and CAD systems, some of which are still sold and used today.) I have literally never worked on any other program in my entire career that was as well-run and orchestrated overall as the B-2 program. (We called it ADP-101 at the time, and for the first few years, we didn't even know what we were working on!)


I wouldn't be surprised. It has happened with a lot of different acquisition programs. That was actually one of my first thoughts on reading the article, as I know people who've had to do this kind of reverse engineering for other aircraft due to a lack of tech data in the initial contract and refusal of USAF to purchase it later.


There's a similar problem to the "we'll create it cheaper if" issue in a lot of government software acquisition. Contractors, especially big companies, often want a lot more money to hand over source code for software. Often the government takes the short-term discount, but then ends up paying massively over time by having no ability to maintain or upgrade the system without involvement from the same contractor.


Here in Canada, our air force takes out airworthy parts from planes, like emergency locator transmitter (ELT) batteries, before they're sent to museums. They have a stash of plane parts in a hangar somewhere for cases just like this.


During the Falklands war the RAF had to raid museum's planes for parts for their Vulcan bombers.


Why is it legal to do this? I don’t pay taxes so a bunch of morons can waste it on a consulting company trick


There are lots of different types of projects. Sometimes you're asking people to invent things for you, and you're the only conceivable customer for that thing, and (like here) it would be weird and perverse not to require all the data/specs/whatever for that thing. Other times you're buying something that's already been invented, with many existing customers, and the data/specs/whatever for that product line are worth much more than the units you actually want to buy, and it wouldn't make any sense to have to buy out that entire product line just to use the product. It might be hard to make simple rules that distinguish between these cases, and all the gray areas in between.


It would at least make sense for it to be mandatory for any vehicles or weapons used by the DoD, no?


I wonder if doing so would create a barrier to entry for the annual billions of $ of defense equipment the US sells to Saudi Arabia. For example, what if Saudi Arabia considers it a risk if another government has the technical plans? If that were the case, I expect defense contractors would be adverse to divulging those details.


And if the contractor were located in Australia or China the government could legally mandate the defense contractors give them access to the code


Keeping things is NOT free, especially for physical things like tooling/fixtures/equipment. It actually costs a surprising amount of money.


Definitely a real live possibility. It could also just be lost. Probably physical copies, tons of turnover over many years and sites. This seems less likely but there is a perverse incentive not to admit it is lost unless required by law.


I very much suspect this is the case in this instance.


How is that not a national security threat?


Reverse engineering seems like a good skillset for the DoD to nurture, anyway. Yes, the US is going to be at the cutting edge of weapons design at least through the medium term, but the leadership isn't arrogant enough to think that all engineering innovations will come domestically. If the US acquired a next-generation Chinese fighter, wouldn't it be handy if someone had the expertise to tell you exactly how its parts were made and machine replicas if needed?

So, the tinfoil hat interpretation of this story would be, of course the Pentagon has the original blueprints in quintuplicate, but it's identified reverse engineering as a key strategic advantage China has over the US (pretty reasonable so far, right?) and has announced this as a way to shuffle some money into building up domestic RE expertise in the defence industry.


As someone who works with the government a fair bit, this kind of forward thinking is extremely unlikely to be the case. This is the kind of government competence that only exists in the most wild conspiracy theories.

The idea that it was made by some long defunct subcontractor and nobody knows what happened to their blueprints and production notes is far more likely. Also, even if you had the blueprints you still need to be able to make the tooling, which is at least half of this project.

That said, if some contractor does come through with this and does a good job (on time, on budget, parts work, etc...) they could likely find plenty of other programs with similar needs. In some cases the original contractor does still exist, but they're asking way too much money to reconstitute an old production line so a scrappy startup like this could prove themselves valuable.


My dad was talking to an engineer at one of the major airlines manufactures apparently some critical part was foam injection or some such.

Anyway the process as such did not work as documented.

You had to add strings to the inside of the mold to get it to set correctly. This was not written down anywhere.

Part of it may have been job protection. The union was bad about that. New hires would not be able to complete the work.

Always stuck with me that at some point the ability to make that part would be los-tech.


It's also possible it was "Hey! The foam injection machine isn't working, I've tried everything", "There's this string trick I used once, try that", "Great, that fixed it, we should document that", "Sure, once we get all parts made that have backed up"

Or, maybe I'm reading bad practices in my industry to everywhere else.


I've seen that way too many times. "Why doesn't this link in the internal console work?" "Oh, after you click it and get the error, you have to remove the trailing dash character and refresh." "Weird, I wonder how this slipped by Jared in QA when he tested it. We better let him know." "Don't bother, Jared is the one who showed me how to remove the dash...."


It makes sense that new employees are trained on the job from actual line workers, rather than by reading documentation.

It is totally possible that the line workers have no idea what the documentation on the tooling says.


Makes sense in the short term, but not the long term, as the entire article makes clear


I would be very interested in reading case studies where a company both does documentation well and does not get bogged down in vast amounts of paperwork.

As yes, plenty of fixes for things are not written down anywhere.


If you haven't read it, you'd probably enjoy (and anyone interested in process) The Toyota Way by Liker.


Unfortunately, every company that I have been at that has tried to implant the Toyota Way just bolted additional processes onto their legacy ones, adding complexity.


Thanks!


Reproducibility.

Have others come in and follow the directions.


It is genuinely impossible to document every lityle nuance that makes everything work.

Every time i am dealing in new hires, i realise how much we just take for granted


No, the problem is that while you are writing documentation you are not making money for the company. I remember when I and Dave as the senior tech told our boss that we should put the most common repairs into a database so new hires would not have to ask us questions all the time.

He asked how long the job would take, and we answered about a week for all the common stuff. And his answer was, "That is a week you will not be making the company any money.".

We tried to explain the long term savings, but he was not interested.


I think the best way to think of this part of a wiki is as a cache for everybody's help. No need to pre-populate the cache, just start using it, and the most frequently used information will tend to be there.


Definitely, not perfect readme or documentation but at least that covers 70% of cases.


From the conversation, admittedly 20 years ago. Not documenting was highly intentional, and why I was being informed.


It might have been protectionism, but also life is just complicated. Documentation is always incomplete and much of it is out of date as soon as you write it down. Humans have so many cognitive biases, we don't even notice half the things we do.

Many scientific studies cannot be replicated either due to errors or ambiguous steps.

You only have what you test. If you want working documentation, you have to do a blind test. That can be pretty dang expensive, and most people don't think that way. Running production is heavily motivated to just keep running, and not worry too much about all of the potential problems - there are just too many potential problems.


Conversely, a lot of management have zero interest in knowing this kind of detail. It may not be possible to get the info fed back. People may be punished for not following the process, even if it's impossible to get the job done by following it exactly ( this is a very common dysfunction!)


Yeah this sort of "why didn't the lowest-paid employee do more work without permission so her boss's boss's boss wouldn't have to think too hard" suggestion surprises me a lot more often than it should surprise me. Sure, it would benefit shareholders, but companies are run by executives.


> Always stuck with me that at some point the ability to make that part would be los-tech.

https://www.sarna.net/wiki/Lostech


As some siblings have commented, it's probably the failure to get back and document it. But the desire for job protection is not uncommon. I saw it when I worked on testing satellite power converters at a large aerospace company. There were procedures and paperwork for everything, but the test procedure for one series of power converters did not work. Only one technician knew how to actually test them. He led me through it, and he had plenty of time to update the procedures, but refused to do it. It was gross and obvious what he was doing, but somehow he was allowed to get away with it. There was no union.


So many problems could be solved by giving every adult $1,000,000 in a one-time, tax-free, no strings attached, lump sum.

The technician refused to document the procedure because he was afraid of becoming replaceable. He was afraid of becoming replaceable because he was afraid of losing his job. He was afraid of losing his job because he would lose his sole source of income.

Let's just give people free money already.


I'm pretty sure this is sarcasm. But I never know these days because a lot of people actually avocate things like this. After seeing what happens to most people after they receive a lot of free stuff, especially large sums of money, I would never advocate something like this. There's also the rather obvious issue about how it would be paid for but, again, there a lots of people who think you can just print money out of thin air forever and nothing will ever happen.


Sorry, very often it is ego. Even is the job is secure, the worker thinks he is special because s/he knows something the others do not.


The return of guilds.


Part of it may have been job protection.

Management gets exactly the behaviours from employees that it rewards and incentivises.


Apparently that is a Russian proverb: "They pretend to pay us, and we pretend to work."


>but they're asking way too much money to reconstitute an old production line so a scrappy startup like this could prove themselves valuable.

The air force is currently going on an in-housing and "agile all the things" binge because they're sick of 30yr of getting screwed by contractors.

My money is on Northrop giving them a sky high but not without precedent price and someone saying "screw it, this is the perfect thing to learn how to do"


It sounds like it could be they have the original blueprints of the first ordered plane but by the time you received the 100th they were different and then it’s had overhauls and other modifications since. About as useful as a ten year old X-ray for a 25 year old.


Why does it seem to be that we always assume that our own government is incompetent and unable of forward thinking, but all other governments (China, etc.) are?


People assuming the worst mostly. US intelligence used to always overestimate Soviet capabilities too. Usually by a large margin.

It’s a bit terrifying when you realize that the people at the top don’t know what they are doing either. Much of the world runs on guesswork and inertia. This is also true of most companies.


"It’s a bit terrifying when you realize that the people at the top don’t know what they are doing either. Much of the world runs on guesswork and inertia."

In my early 20's this 'discovery' shocked me


Yeah, as an adult in late twenties, I am realizing this fact as well. Nobody has any idea on what they are doing, world is in chaos.


China doesn't have to deal with a lot of the "complications" that the Western governments have to deal with, like human rights, etc. This gives those in power the potential to achieve their goals more efficiently.

Not saying this is how it actually ends up, but they certainly are prepared to use a lot of leverage tools that are not available to more free countries.


“...[The Russians] have an immense advantage. I think it’s a dangerous advantage and a temporary advantage. Twelve men decide what to do. They don’t sit here like I do and have twelve others question me on why I’m doin’ it... In the end they destroy themselves.” —General George Marshall https://youtu.be/Y7eVaVUESCo


Because we can't see what's going on inside the Chinese government so they get benefit of the doubt. In contrast we know exactly how the local sausage is made.


Or that government should be, could be less incompetent than corporations?

People are just people. Organizational psychology is universal. Bit rot is inevitable.


>As someone who works with the government a fair bit, this kind of forward thinking is extremely unlikely to be the case. This is the kind of government competence that only exists in the most wild conspiracy theories.

No, but we'll totally sponsor/fund research into things like mind control and astral projection. Funding something totally useful like reverse engineering skills is totally anathema. Just really sad commentary


> No, but we'll totally sponsor/fund research into things like mind control and astral projection. Funding something totally useful like reverse engineering skills is totally anathema. Just really sad commentary

I don't get your point. The government does invest in reverse engineering, and much more than they invest into mind control and astral projection. It's nonsense to say that investing in useful skills is "totally anathema".

What GP was saying was not that they don't invest in reverse engineering, but they wouldn't try to backdoor it or secret the experience into the industry with some project like this. They wouldn't try to "trick" the industry into developing the experience by putting out smaller contracts like this. They do invest in it, but directly. That one program office has put out a contract for reverse engineering of one system does not mean that the general capability is ignored, within the government or the industry.


>This is the kind of government competence that only exists in the most wild conspiracy theories

And, possibly, before the 1960s.


Employees are either clock punchers or workers.

Small departments in poor cities occasionally wind up highly competent because they wind up staffed with a high enough ratio of workers to get things done and everybody above them has bigger problems than micro managing so they have the freedom to get things done. And it runs for awhile then comes under scrutiny and gets made back into junk because politics.


> politics

I agree. And there's a massive depth unwinding to that.


I don't think we need the tinfoil hat. This sort of thing happens sometimes. Like the time we forgot how to may part of our thermonuclear weapons and had to reverse engineer that [0].

0 - https://en.wikipedia.org/wiki/FOGBANK


> I don't think we need the tinfoil hat. This sort of thing happens sometimes. Like the time we forgot how to may part of our thermonuclear weapons and had to reverse engineer that [0].

The really interesting thing about that is one of the main things they had to reverse-engineer was an impurity in one of the ingredients that the original designers didn't even know they were depending on.


Reminds me of how NASA had to design completely new rocket engines for SLS because while they still had the plans for the Saturn-V rocket engines, back then plans weren't followed to such high accuracy as nowadays a 3D model would be followed. Each engine was custom tailored with small modifications here and there. The knowledge to custom tailor them was lost, so they went with designing new engines from scratch instead.

https://www.youtube.com/watch?v=ovD0aLdRUs0


> Each engine was custom tailored with small modifications here and there

This was true of lots of things outside NASA. This was one of the problems that killed the UK attempt to modernise the Nimrod maritime patrol aircraft [0]. They wanted to fit new wings to the old planes, but only discovered after the contract had been signed that the different airframes had not been built identically, but were different interpretations of a common plan. Each of the nine planes was in effect a completely new refit challenge.

[0] https://en.wikipedia.org/wiki/BAE_Systems_Nimrod_MRA4


I believe there is Murphy’s Law style saying for this

Interchangeable parts don’t.


It reminds you of a thing that never happened? That's a neat trick. The SLS uses the RS-25 engines, the Space Shuttle Main Engines which are in no way an engine "designed from scratch".

Additionally the F-1 engines were redesigned using more modern techniques around 2012 with the F-1B [0].

The SLS went with the RS-25s for a number of reasons. While the F-1s provide high thrust they're not as efficient. The RS-25s can burn longer and with the SRBs providing massive amounts of thrust to get the stack off the ground the RS-25s can provide more delta-v to the upper stages. An F-1 reboot would have required a lot of extra work to add in modern computer controls which already exist in the RS-25. They have also flown in the past few decades and are well understood.

[0] https://arstechnica.com/science/2013/04/new-f-1b-rocket-engi...


I heard this story as well. But in the end they picked something else they had and could make. I think the original idea was to start with the same as what went to the moon and build on that. At one point we made several thousand of something like them. They found they could not build that anymore, and it was not as good as stuff they could do, so they redesigned to something else that they could. That is as far as most people take that story.

In fact most 'stories' and 'news' is like that. The SLS is known to be in a congress sub committee hot potato game. So that there are different versions floating around of what happened and is happening is not surprising. Teasing out the 'truth' of what happened is not always easy. As many have interests in distorting it, or just remember wrong.


There were not thousands of F-1 engines manufactured. There were a bit over a hundred manufactured between flight and test models.

The whole "we can't make them anymore" meme has a tiny kernel of truth but it's been blown way out of proportion the more it's been repeated.

When the F-1 was manufactured a lot of the actual assembly of parts was done by hand. That meant Rocketdyne's engineers and machinists had a lot of tribal knowledge that was lightly or partially documented at best or not documented at worst. NASA and Rocketdyne maintained all of the designs and documentation and knew everyone involved in the engine manufacturing.

In the early 90s a lot of these Rocketdyne people were interviewed [0] and asked to review F-1 manufacturing techniques as part of the Space Exploration Initiative [1]. Restarting production of the F-1 would have been practical but more expensive than the alternate plan to go with an SSME derivative.

While the F-1 is an impressive engine it has downsides. It's enormous and heavy. The Saturn V's first stage (S-1C) was 10m in diameter while the Space Shuttle's external tank was 8.4m in diameter. The Shuttle's ET was going to be the core of the proposed SEI rocket (and SLS). With a 8.4m tank diameter it would be more difficult to mount more than two F-1s.

With only two engines it would be more dangerous of a vehicle for manned missions as a flame out in one engine could be disastrous. It would also make it more difficult to throttle the engines for different launch profiles. Moving to a larger core would have required all new manufacturing infrastructure, flight qualification, and unknown follow-on issues.

Tl;dr The F-1 was an awesome engine but not perfect or even desirable for all purposes and the "we can't build them" meme is stupid.

[0] https://www.thespacereview.com/article/3724/1

[1] https://en.m.wikipedia.org/wiki/Space_Exploration_Initiative


The comment you are replying to is "lying by omission" (if you want to call it that) but the video talks about your point so the snarky tone puts you in a worse spotlight than the original comment.

Basically, someone created a F1 engine inspired design with modern manufacturing technology but ultimately it was never used.


Wikipedia calls it an "aerogel", but rumors have it originally being closer to ordinary "closed-cell extruded polystyrene foam", which is often sold under the brand name Styrofoam. It might be something else entirely now. It's role is said to be absorbing the x-rays generated by the fission stage and turning into a plasma to help ignite the fusion stage. Edit: https://www.armscontrolwonk.com/archive/201814/fogbank/


I was going to comment about fogbank as well, this article explains the efforts the Govt. tried to reverse engineer the substance - https://www.washingtonexaminer.com/weekly-standard/the-fog-o...

Also it is no longer known how to make the Saturn V rocket which flew to the moon - https://www.nytimes.com/1987/05/26/science/hunt-is-on-for-sc...


Thank you. Very interesting article. I love nuclear history stuff.


I don't think this kind of thing is all that uncommon for military airplanes that have been long out of production. I recall reading a decade or two ago that our local air guard wing had to fabricate some of its own replacement parts for its F-16s (IIRC, then some of oldest that were still in service).

I also recall that Lockheed had to take special effort to preserve production know-how and tooling for the F-22 when they shutdown production early, so it would be possible to restart if needed. You can't capture everything you need to know in blueprints and documentation.


During the escalation of the Vietnam War, the Air Force ran short of jet transports and decided to pull a bunch of WW2 era propeller transports out of mothballs and put them back in service. The mechanics, however, complained that they followed the instructions to the letter on tuning the engines but could not get them to develop the power they were supposed to.

The AF finally hired a bunch of old WW2 airplane mechanics. Those mechanics never looked at the manual, they just listened to the engine, made few adjustments, and the engines were soon singing at full power.


A friend of mine looked into building a P-51 from scratch, as buying a real one was too expensive. He ordered the blueprints, which are readily available. But then he talked to some P-51 owners, who told him the blueprints wouldn't be much use. The airplane was defined by the jigs used to build it, and the blueprints were just drawn up later as documentation and never checked. The jigs were scrapped when the airplane went out of production.


>Long out of production.

The difference in technology likely plays a large role - these parts were designed without CAD and modern fabrication protocols have changed.

It reminded me of NASA's endeavor to reverse engineer the Saturn rockets

https://arstechnica.com/science/2013/04/how-nasa-brought-the...


> The difference in technology likely plays a large role - these parts were designed without CAD and modern fabrication protocols have changed.

Actually, the B-2 was one of the first military aircraft designed with CAD.

https://www.northropgrumman.com/wp-content/uploads/B-2-Spiri...

(search for "NCAD")


Preventing the loss of manufacturing capability is one of the reasons for offering military aid to friendly countries. We give them money to buy weapons, but they are obligated to buy certain weapons systems with that money.

For example, the Saudi's really don't need all those M-1s they buy from us (with money we give them), but we offer the it keeps our production lines open and active.

It's a devious way of laundering military readiness expenses.


Yeah no. Back in the 2000s when Iraq war was happening and marines were using MSN Messenger and I asked whether the unencrypted nature of the chat program would adversely impacted on their missions etc, and he basically said "hell they can spy all they want WE don't even know what we're doing next"

Which I thought he was joking at the time.


I wish my tax dollars were going to some galaxy-brain forward thinking group of men and women capable of making such reverse engineering challenges for itself; but it almost certainly is the case that what looks like sheer incompetence is just that.


Most of the companies are Initech ;)


Office Space (the movie) If you don't get the reference.

Similar: See Dilbert, the cartoon version had a funny episode about company names through acquisitions.


Nah, I think the one guy who knows the thing is gone, the data was on a hard drive that crashed and the backup system didn't work, and the papers copies are probably somewhere except no one knows where and it is faster to reverse engineer the damn thing.

I have no reason to believe that defense projects are not plagued by bureaucracy, office politics and incompetence like in a lot of government entities and large companies. If anything, it is worse because secrecy is a convenient way to hide incompetence. And when information is shared on a "need to know" basis and we realize that no one "needs to know", then it gets lost.

I have touched a bit on defense projects (not in the US) and generally, compared to civilian, open projects, if feels slow and primitive. I didn't get to the super-secret stuff but I have a feeling that the only parts where civilians don't do much better are the ones that only the army is interested in.


US DOD and their industry partners definitely has this capability already. If they're putting this call out there then there's a real need, not a covert effort to promote the discipline of reverse engineering.


Also, to quote Independence Day:

> You don't actually think they spend $20,000 on a hammer, $30,000 on a toilet seat, do you?

The budgets they use for secret reverse engineering aren't booked as any kind of reverse engineering.


One fears we'd be shocked by how much is actually going to General Smith's brother-in-law's toilet-seat dropship company. We'd like to think that Pentagon auditors could find that sort of thing, except that Pentagon people tell us continuously that they are unauditable.

https://www.thenation.com/article/archive/pentagon-audit-bud...


> If the US acquired a next-generation Chinese fighter, wouldn't it be handy if someone had the expertise to tell you exactly how its parts were made and machine replicas if needed?

I suppose they could always call the engineers at Boeing or Lockheed who designed it in the first place. Like getting tech support for your Huawei networking equipment, just use the Cisco guide! [0]

[0] https://blogs.cisco.com/news/huawei-and-ciscos-source-code-c...


For anyone unaware Lockheed was hacked in 2007 and the hackers got away with technical documentation related to the F-35 Joint Strike Fighter. The hack was attributed to hackers in China. So it is thought that the development of the Chinese J-20 was at least helped along by information from the hack.


Read about the history of NASIC and the other service intelligence centers [0]. Understanding foreign technology is one of their core missions. I'm pretty sure they have already been reverse engineering stuff for a long time.

[0]https://www.nasic.af.mil/About-Us/Fact-Sheets/Article/611728...


I've heard from various acquaintances that it's not unusual for the contractor to be required to destroy documentation and tooling for certain classified projects upon completion.


It is absolutely standard practice for this to happen. At contract completion, the contractor must destroy or return all classified materials: these sorts of designs would certainly be included.

It's pretty interesting, but a lot of this stuff is structured specifically so that the contractor organization doesn't retain knowledge that it's "not supposed to have", as wasteful as that ends up being. Even stuff like a list of personnel who worked on a project or production line might be included... so that a few years after the project is completed, the management of the contractor might not even be able to figure out who to ask, because they've destroyed even their knowledge of the list of people who worked on the component's design and manufacturing.


Even if you're not innovating everything you want to reverse engineer enemy systems because you want to discover vulnerabilities. This is common practice.


I think the US (and power nations in general) have a history of reverse engineering the foreign war tech they can get their hands on.


We generally only reverse-engineer technologies from elsewhere (no, not other countries...)


> Reverse engineering seems like a good skillset for the DoD to nurture, anyway.

I though they already had enough practice reverse engineering the alien tech at Area 51?


The US nuclear weapons program had to do something similar in the 2000's with a material known as Fogbank. This is used in the W76 warhead as an interstage material, and they needed more of it to keep the weapons in operational state.

Unfortunately, they'd forgotten the manufacturing process, decommissioned the plant that made it, and the people that knew how to do it weren't around any more. They figured it out, but it doesn't necessarily look easy to do (nor does the material seem easy at all to make in the first place).

https://www.armscontrolwonk.com/archive/201814/fogbank/

p20 here says more: https://www.lanl.gov/science/weapons_journal/wj_pubs/17nwj2_...

Edit: I somehow missed this other thread in this story, despite a search: https://news.ycombinator.com/item?id=26334367


Somethign I haven't seen mentioned here is that the technology used to design this thing is waaaaaaaaaaaay old. I know for a fact it was designed on an IBM mainframe (dad ran the one that the YF-22 was designed on). I can't remember what CAD package they used there, but its definitely conceivable that the software doesn't even exist anymore. How would you go about building a new one if you can't read the CAD files, if you have them.

Hell, the building in hawthorne where dad worked isn't even owned by northrop anymore. Its spacex.


Good callout. However, it might not be all that bad. CAD file formats are known to last for decades to maintain backwards compatibility, since the packages are expensive and upgrade paths of large companies slow.

CAD files (at least planar modeling as opposed to parametric or solid modelling) tend to have pretty straight forward data structures for geometry. Find the point or vertex struct and you're halfway there. The complexity is association and metadata which were nowhere near as bloated then as they are now.

As someone who works regularly for large asset owners, asset life-cycle information (digital and analogue) and the proper management of that information is often ignored because organizations pass that risk to the people working with the asset maintaining their own working knowledge to get their job done. We've had international Building Information Modeling standards since the turn of the century but there's often no business case or standard enterprise process to make leverage technology for asset management.

AI and ML are starting to appear more and more in the operations space, however many pilot projects have to start with digitizing the asset in the first place, which becomes a cost impediment for many getting off the ground.

It really frustrates me as there are great things that can already be done today but the initial step financially usually scares off anyone who has to know 5 years in advance what their operating budget is.


Your story reminds me of what I read about the McLaren F1, that it had a particular 90s Compaq that had to be kept running in order to service them. As it turns out, this is recently no longer true.

He also has "the laptop"—a mid-1990s Compaq running custom DOS software created by McLaren when the F1 was new. This gray brick was, for many years, required to access the car's engine control and body control modules. Today, McLaren uses a modern Windows computer running a software emulator for day-to-day computer maintenance. Hines keeps the vintage Compaq around just in case.

https://www.roadandtrack.com/car-culture/classic-cars/a14453...


Reverse engineering the files night be easier than reverse engineering the airplane.


Says the guy not constantly surrounded by aerospace engineers :)

What are the odds someone said the phrase "those parts? I could mock up copies in a weekend!"


I read once that if you tried to build WWII-era aircraft according to specs and mechanical drawings, they wouldn't fly. The real design was found in the dies and jigs that were adapted after many iterations and test flights.


If you find this stuff interesting, pick up a book aimed at hobbyist metal lathes. Just reading a little about how you set up a tiny home version can give you a whole new level of understanding and respect for this kind of stuff. Structural components are different but in that era and even to some extent today, most precision parts were machined with two primary tools: the lathe and mill. Today both would be CNC and you'd add other modern tools but the CNC lathe is still a lathe and they work the metal in the same fundamental way.

Due to the dispersed mass production in WW2 it's probably not totally true that they wouldn't fly without a custom tooling setup (it'd be a maintenance nightmare if two examples of the same model from different plants diverged dramatically, altho of course it happened) but you can be certain that it's of critical importance.


It's because the designs weren't made for mass production. The given tolerances were nominal and you were supposed to hand-fit parts together.

For mass production, they had to be re-tooled with larger tolerances which also allowed for interchangeable parts.


That's an interesting though, like software documentation that hasn't been updated over years of patches.

Do you remember where you read that? Sounds like an interesting bit of history.


A long write-up of an equivalent for the F-1 engine is here: https://arstechnica.com/science/2013/04/how-nasa-brought-the...

The analogy to out-of-date software documentation is very apt. As problems or inefficiencies popped up in the factory, they'd change things and just not write them down (because of deadline pressure, often). If you wanted to build new ones you'd have to either get the factory workers to show you how the thing was actually built, or tear down a working example to reverse engineer it. (Which has been done!)


Also sounds like a git repository where people were working on local branches to do production work and then when they left the company, never pushed their branch upstream.


That's assuming there is a git repo or any kind of version control at all.


They're not assuming. They're analogizing.


You might have fitting issues with what I like to call "slide ruler error".

How things came to be and are, are often at times not documented. Dimension and tolerancing is lost over time too.


Statistical quality control was not widespread at the time outside of a few enlightened industries. The Army was much more likely to buy a lot of parts where they could expect most of them would fit, rather than having all of the parts in statistical control within the tolerance limits.


A huge amount of industry all over the place ends up like this. The engineers produce the theoretical form of the design, and the tooling up process refines it into a practicable-at-scale reality.


I only worked tangentially in the field, but I get the feeling that modern semiconductor manufacturing is sort of the same, except with more excel sheets and the like?


Semiconductor manufacturing is essentially the interchangeable parts engineering taken to the extreme. The tolerances are so large (relative to the feature size) that instead of trying to eliminate them you exploit the relations between various manufacturing tolerances such that the errors cancel out.


This happens even outside Defense in all specialized industry. I was once asked if I could reverse engineer a part for a 30 year newspaper stacking machine in a printing press. Very interesting stuff and a consequence of being niche.


Exactly. We're dealing with:

- Limited production run. These aren't Ford Mustangs or Toyota Priuses with hundreds of thousands and millions being produced. There were 21 built over 13 years. This also leads to the consequence that each one should be considered a bespoke creation and not a "copy" of the others (even ignoring the 20+ years of individual maintenance work they've each had).

- Time. It's been 21 years since the last one was produced in 2000. Whatever facility produced this has long since lost that capability.

- Aging workforce. Whoever designed it is likely retired, and could even be dead at this point. Certainly the senior engineers who may have been 40+ when the project started in the 70s/80s. Even if they weren't retiring and dying, the people on the project have been doing other things for 20+ years.


The last part is extremely true. I have known many engineers that worked on the B-2 and have since retired. There's very little chance many of them would come out of retirement and honestly I don't think the government would pay for that.


And even if they did, no one remembers that well the things they did decades ago.


And even if they did, would that knowledge help reproduce it on modern equipment? Going to have to rebuild all of the tooling and processes anyways.


Large scale printing machine setups are insane. I've seen very old installations go for prices that are higher or comparable to a new setup just because you know the throughput and you know the civil engineering is sound.


Also in auto repair. A shop told me they wanted to take a gasket from the engine, draw the part, then send it to a machine shop to have it fabricated.


Try most web companies :) I'm a week into reverse engineering ancient URL canonicalization at my new place... sigh


This brings up an important principle: the higher the tech, the shorter its lifespan. A cellphone from the 1980s won't function anywhere but in a Faraday-caged lab equipped with a suitable cell site, while 2G and 3G systems are fast disappearing. Only recently have landline telephone exchanges begun to drop support for pulse dialing, but otherwise, a rotary-dial desk phone from 100 years ago would still work today, and DTMF dates to 1963.


You can get hundred year old manual lathes/mills and they still do their job. It's because the basic principle upon which they were built is still valid today. The only difference with modern machinist tools is that they are fully automated and you can just retrofit CNC parts to the old equipment.


> a rotary-dial desk phone from 100 years ago would still work today

This still amazes me, even though every time a call comes in on the landline, the 50-year-old rotary-dial phone in my living room rings alongside the cordless phone. It doesn't get used much, except for the loud ringer and during extended power outages.


I was active duty Air Force in the mid 00s, and we had a tropospheric scatter terminal called the TRC-170 where the entire manual was reverse engineered. Air Force Technical Manuals aren't the most riveting of documents to read, but man that one was painful. The technology is badass though, and still useful even half a century later!

The story (which I have no idea if is actually true, but it was widely repeated), was that the engineer who built it in the 60s didn't get paid the full amount that he thought he was owed, so he destroyed the technical manual as a middle-finger to the Air Force.


I'm not surprised. I was recently tasked with a very simple change in a bootloader in an industrial product used in the nuclear industry. Part of the task was also to find the latest sourcecode that was actually compiled into the product. The last known changes was some 20+ years ago!

I, as a contractor, and an employee (who was the one who actually coded the last change!) spent more than a week trying to hunt down any leads to anyone who could know where the source could be found. In the end we gave up, we had no leads at all it turned out. The code was not developed in the same company, and not even those working peripherally with these things were still working in the company.

In the end, the 15 lines of assembly that was supposed to be reverted/removed was done, but the uncertainty of the C-code part of the bootloader and some evidence that the known code was in fact no the latest and greatest, the change was abandoned altogether and a workaround in another part used instead.

I reverted to binary comparisons of machine code, and I also suggested editing the actual machine code to avoid recompiling other parts of the code, but it wasn't a very popular option and I advised against it if there were a possible workaround.


"This engineering effort is to reverse engineer the core of the B‐2 Load Heat Exchangers, develop disassembly process to remove defective cores, develop a stacking, vacuum brazing, and welding process to manufacture new heat exchanger cores and to develop a welding process to install the new cores on existing B‐2 Load Heat Exchangers. The requirement includes reverse-engineering the re‐core process for the B‐2 Load Heat Exchangers. The B‐2 Load Heat Exchanger (NSN 1660‐01‐350‐8209FW) uses air and Ethylene Glycol Water (EGW) liquid to produce cold air for the cooling system."

They need to change some plugged up radiators. Good to see that old bombers and old trucks have the same problems.


This is exactly the sort of problem that Jon Blow talked about in his talk about preventing the collapse of civilization [0] -- the human race forgets how it does amazing things.

[0] https://news.ycombinator.com/item?id=25788317


A related story:

One of the proposals for future upgrades to SLS (ugh) was liquid-fueled boosters using new-build F-1 engines (i.e. Saturn V main engines). Unfortunately, the specs were not complete; not only was lots of stuff different in the factory than in the drawings, but parts of the engine were literally hand-built. Think API documentation built on a tight deadline, and published while software is still under development.

Luckily, some NASA engineers were already taking one apart and modeling the parts for previous SLS work. The new Dynetics version ended up much simpler and cheaper because of new manufacturing methods, but the reverse engineering process required to get there was a serious project in its own right.

https://arstechnica.com/science/2013/04/how-nasa-brought-the...


Everyone once in a while you'll see a story in the news like "Congress budgets $x million to keep open a factory that builds 20 tanks a year even though we don't need more tanks"

Even though there's definitely a pork aspect, there is also actual value in being able to keep that sort of business continuity and institutional knowledge alive.


This is not new.

Older planes and other hardware have LONG had to be reverse-engineered (including microchips).


One of my aerospace professors in college used to be the secretary of the Air Force and B2 development started during his time in office there (back when Carter was president). I remember him saying that one of the missed opportunities in his opinion for the U.S. was not buying more B2 bombers. They can reach anywhere on earth and we only have 20 of them. But Congress didn’t want to keep making them after the Cold War.


For many reasons - limited production run, secrecy, and age — the need in this case for reverse engineering isn’t all that surprising.

Nevertheless, these things may become more common if we are at the apex of our civilization.

I could imagine a RFP in Rome from 300 AD or so soliciting bids to figuring out how the hell to make an aquaduct, amphitheater, or even cement!


My grandfather spent the last part of his career doing this sort of work—often on classified projects, I'm told. Usually there was an assumption made on the project about the useful working life of any part, and the expected duration of the program, and they made the "right" number of spares.

Things change, and the factory/tooling/people are long gone. Also, often the underlying tech isn't available. But the part needs to weigh the same amount, meet the same guidelines, and fit in the same hole.

It's cheaper to pay someone to spend the time to figure out how to make the replacement then to mothball the entire airplane—almost regardless of how expensive it might be.


The only possible explanation is that the Stealth Bomber was indeed crafted using alien technology. /s


The B2 was designed to stealthily penetrate long distances into the USSR. It was a 1980s black program that eventually (at absurd cost) only shipped about 20 aircraft for a mission role that no longer existed. It's not surprising that the infrastructure and supply chain to maintain them into the latter parts of their operational life didn't materialize.

I am curious about what the current role of the B-2 is, as the US doesn't really have much need to bomb anywhere that it doesn't already have complete air superiority (and retaliatory strikes that wouldn't be preceded by pounding the local air force flat are best served by missiles, drones and other things that don't take a crew somewhere they might crash and end up confessing to war crimes on live TV). It was used to deliver a few bombs in Afghanistan and Iraq, but not in a role that couldn't have easily been filled by a B-52. Or a B-17. (I checked. There are currently more than 40 B-17s still flying in the world. That's twice as many as there are B-2s. Hmmmm...)


It’s still a part of the so-called nuclear triad. Whether it’s actually critical for second-strike credibility (absent a 24/7 in-air alert, which I think no longer exists?) is up to debate, of course, but that would be the rationale.


Dupes / other identical submissions:

https://news.ycombinator.com/item?id=26321523

https://news.ycombinator.com/item?id=26322477

---

As an aside, does it still qualify as a dupe if there wasn't any significant discourse?


Re aside:

Generally no. Some submissions just don't take due to poor timing (not lining up with when an interested audience will catch, upvote, and discuss it).


Here is a relevant lecture on the decay of specialised knowledge across civilisations.

https://youtu.be/ZSRHeXYDLko

Included is an anecdote about Texas Instruments and loss of knowledge between generations of silicon hardware.


Sort of related: in UK an entire skyscraper went up in flames, and it was obvious there were multiple breaches of fire safety. Over 100 people died.

The inquery was told that the plans for the building were lost, and it was the intern's fault that he deleted them off his pc.

https://www.itv.com/news/london/2020-09-14/grenfell-files-lo...


I remember that. Shitty builders, developers, and inspectors.

Are these both then "their dogs ate their homework" defenses?


Tangentially reminds me of this video by Jonathan Blow [0], "Preventing the Collapse of Civilization". The eye-opening moment, for me, was the importance (and neglect) of the inter-generational transfer of knowledge in maintaining technological complexity.

[0] https://www.youtube.com/watch?v=ZSRHeXYDLko


Not an exact reply to your comment/that video, but I think this dovetails well with it. Burke's "Connections" (1978) from the BBC is an interesting look at how technology has been developed and built on itself over time.


That was the first thing I thought. Jon Blow was spot-on.


Not the first time this happens. Reminds me of a case where they had to reverse-engineer parts for one of the stockpiled atomic warhead types.


Proof the B-2 was built using UFO technology :-P


When you read the Skunk Works book you stumble upon passages where the government has to pay for tools maintenance and storage after the project is complete.

Some bureaucrat entity always decides at some moment that paying for storage of an arbitrarily old project tooling is not worth it.


If I were a reporter, I'd have 3 questions for DoD:

Aren't they spending tens of MIC billions on new bombers soon anyhow?

Why aren't they cannibalizing other airframes and just making do?

Why didn't they have a spares support contract for the entire lifecycle of the airframe?


This sounds like an awesome idea for the beginning of a sci-fi story: a test for a team the government wants to reverse engineer an alien spaceship or something.

I also just remembered this is the beginning of the original Stargate movie...


The B-2 pictured says "Spirit of Pennsylvania" on its gear flap. Are all of fleet named in similar spirit (pun)?

Would seem kind of reminiscent of WWII battleships named after states of each is named "Spirit of a State".


Yes. Spirit of (Something) is how the B-2s are named.


I've done installs of video streaming tech on USAF-managed NIPR net and we've had to reverse engineer some of the mechanisms on their own network... This does not surprise me at all.


> Twenty-one years after the last Spirit was delivered...

Isn't this just the Air Force not wanting to pay full-price for replacement parts from Northrop Grumman now that the patents have expired?


The B-2 is ancient, doesn't seem very surprising.


It is by no mean ancient.

The introduction of the B-2 in service was ~24 years ago, this is one of the most recent jet in the US air force. By military standard it's definitely young.

There are only 2 planes in service in the US more recent than that, the F-22 and F-35 (not counting transport aircrafts).

Those planes are designed for lifespans of 50-100 years.


My dad was at the Skunk Works for 20 years. He then went on to work at Northrop for a good chunk of time as well.

The B-2 was started in the 70s and the maiden flight occurred while I was still baking in the womb.

I realize aircraft can live a long time and can be retrofitted and upgraded with modern technology - but all things considered it is old.

Not saying ancient is bad, either, but as far as organizations go, org knowledge, handoffs of info and books and writing etc... I am not surprised to see the need for reverse engineering.


I had to check, it's not as ancient as some other bombers with B-2 first flight occurring in 1989. I met a pilot who flew on the same B-52 as his father, and his grandfather had flown some of the first B-52s. This was in the 00s, about 15-18 years ago.


It's impossible to overstate how long these things have been in service. The oldest B-52 airframes still in active service were built longer ago than the first flight of the Wright brothers was when they were built.


And likely to be in service for another 30 years. A 100 year airframe


I worked at Boeing Wichita (which is no more), and an engineer told me that the B-52 was getting heavier over time because the old wires weren't removed for avionics upgrades. Of course, they were always trying to put one over on me as a newbie (I worked on avionics S/W in Ada for the tankers).

He also said a big problem was the expense of doing a modern engine upgrade. Anyone know if they ever re-engined it? Was he joking again?

Having one of these fly over my apartment on Rock Rd. was a dish rattler--I used to see B-2s every once in a while too (fairly loud too, IIRC).


20 years later, looks like they are taking RFPs for re-engining:

[PDF] https://crsreports.congress.gov/product/pdf/IN/IN11413

I guess he was right about the engines.


It first flew 31 years ago. If that is ancient what is the F-16 that first flew 49 years ago?


Wasn't there a joke about this in the x-files when the lone gunmen were at a poker game?


There has to be a way to transfer the massive yearly income of public money into the right private pockets.


People who put effort into spending money effectively eventually get beat out by people who put effort into convincing the government to give them money.

This isn't unique to public money; SEO, advertising, marketing &c. are all ways to get private people to part with their money.


Kind of interesting, as this is essentially the plot for the universe of Battletech.


Sheesh, who reads that job description and feels like they're the perfect fit?


Advances in additive fabrication, plus other advances in substractive machining and the use of sintered metal molded preforms, composites etc., explain most of this. The time saved with the newer methods yield a net efficiency.


Ask China, I'm sure they have a copy of the plans.


> ... Air Force’s Rapid Sustainment Office (RSO) was looking to industry for a “cutting-edge, automated 3D scanning system,” specifically intended to replicate aircraft parts that are no longer in production, including at maintenance depots.

You see the real issue? Even if they have the blueprints, it will be difficult to even manufacture the parts here.

There's so many more problems like this coming down the pipeline. We've effectively lost (or rather acquiesced) almost all manufacturing skill to other countries. This was a result of the largest corporations taking advantage of labor arbitrage and then flooding their home market out of business.

If we really want to reverse this, we need to do what it takes to bring manufacturing back to the United States. It's a painful reversal of the open-trade policies of the past. Honestly, Trump was right to fight so hard for this, even going so far as to remove us from NAFTA.

I don't have a good strategy for this, but probably some set of minimum manufacturing of everything should happen within our borders, say 10%, and accomplishing that target with a combination of sticks and carrots.


IIRC military hardware (ships, tanks, aircraft, etc) is required by law to be manufactured within the country, for obvious reasons.

Its some of the little manufacturing the country has left


Yet just landed on Mars, bringing a helicopter. Please don't spread FUD.


How much of the spacecraft was manufactured in the USA? Are we able to create another one without our trading partners?


From the manufacturer’s website:

> Manufacturing, assembly and integration operations are located at Decatur, Alabama, and Harlingen, Texas.

https://www.ulalaunch.com/about


Isn't that a good thing? Trading partners are unlikely to attack each other.


Of the spacecraft? All of it.


Unfortunately, your assessment is very correct. There will be no solution and only a slow decline as previous systems become obsolete and there no on to fix or upgrade to better systems.

This country is producing more Tik Tok stars than engineers that can sustain civilization.


What a confident proclamation that is backed up by nothing, aside from your limited perspective.


Name a successful plane developed recently by US military or commercial enterprises?


Boeing 787 series. Young in plane years. Best airliner in the sky.

SpaceX makes working spacecraft.


787 should have been a straight forward simple twin engine jet. Yet, it had Billions of dollars in cost overruns and still has major manufacturing safety issues. https://www.fool.com/investing/2020/09/09/boeings-woes-mount...

This country built Saturn 5 rocket, 50 years ago. Landed astronauts on the moon, with moon buggy and golf clubs.

Now we can barely launch basic satellites. How is this progress?


There’s a reasonable chance you typed this on a pocket computer/communicator with an armful of MRNA vaccine in an electric car.


Which country built the pocket computer/communicator? Which country built the electronics, circuit boards, cables, batteries, housing and fastening screws?

Which country built the components and systems and subsystems that go into manufacturing an electric car?

Which country has lithium, rare earth metal processing capabilities critical in manufacturing all modern electronic systems?

Which country built the components and subsystems in the vaccine manufacturing plants?

Decades ago, US had the education system to move millions of people through manufacturing and engineering pipeline to build all above things.

Now, that US education system does not exist. The manufacturing base does not exist. The engineering pipeline does not exist.

We produce more dropout Tik Tok stars than Professional Engineers that develop critical cooling cores, that can withstand heat and stress thresholds in modern military aircraft, and master welders with 20 years of experience that can weld specialty metal alloys in compact designs.

It’s not just aircraft. Do you know who works on water treatment plants that provide clean drinking water to millions of people in US? These guys will be retiring in few years. Who are going to replace them? Who are going to do the hard and difficult job of maintaining these critical systems?


The UK has clean drinking water despite losing its manufacturing primacy to the US 100 years ago.


Tell that to people in Flint Michigan.

https://www.propublica.org/events/the-flint-water-crisis-an-...

No amount of rosy thinking is going change the course of decline in this country.


Is lack of national engineering expertise in the top five reasons for the situation in Flint?


I don't understand why you are getting downvoted.

Being unable to manufactures things we did in the past is extremely worrying


Unfortunately, this will result in failure.

This country simply does not have enough skilled engineers to design and produce original parts to requirements. Let alone reverse engineer a worn out part.

B2 is product of its time and the amount of engineering resources that went into design and development simply can not be replicated with current society.

Tik Tok stars make multi millions of dollars, so why would any smart person spend years in school and solve complex problems to build state of art Stealth Plane.

Boeing can’t even produce a simple jet without having it fall out of the sky few times, 737 Max.


I would disagree on this point. I work in aerospace with a a lot of smart folks. Boeing's 737 issues are a product of many years of poor management choices, but I don't think represent the talent of the industry as a whole.


737 Max is just the poster child for issues in aerospace. There are many other examples, F35, 787, F22, are just recent examples.

The flow of smart folks in the pipeline are dwindling. You may be working with smart folks, but how many are going to retire in few years? Who are going to replace them? Does the current education system produce the number and quality of candidates needed?


Project failures cannot be simply attributed to a workforce problem without a detailed analysis especially when the top example is a widely reported management failure.


It’s an industry failure, society failure, not just a single project failure.

This country can not build a new reliable airplane, like it did decades ago.

It’s failed project after project. Ask yourself why this is the case? What has changed in this country? What has changed in the industry. What has changed in society?

It’s not just aircraft. Where are US high speed rail systems, that other countries built 30, 40 years ago?

Where are modern infrastructure?

Who works in water treatment plants, providing clean drinking water for millions of people? What happens when all these guys retire in few years?

Tik Tok is all fun and games, and no one wants to do the hard messy work of treating water for consumption.


My working hypothesis is that the business class and politicians are looting the country until the empire crumbles.


The F35 seems like a project management issue more than engineering. No one stopped the scope from creeping out of control as new tech and capabilities were added.


> No one stopped the scope from creeping out of control as new tech and capabilities were added.

The problem with F-35 wasn't scope creep, it was the initial premise which operated under a mistaken belief that getting a single “do everything” airframe to maximize commonality would drive down unit cost, rather than driving it up through the conflicting needs of distinct missions.


depressing to read. We point to military technology as a sign of how ingenious we are. Then all that effort gets lost due to laziness and being overambitious. it's beyond belief, our much vaunted stealth technology will be completely worthless in a generation due to simple ineptitude. i'm not sure how the military can be trusted with a toy radio, let alone h bombs.


Yes, as the US cedes the economy to China, they will also surely become stronger militarily. And with the US (and other Western countries) in a perpetual lockdown in the coming years, which will accelerate the transition - we had better make sure we are able to copy the tech!


Why is the government allowed to reverse engineer AND create laws making it difficult for citizens to do so?

Obviously the answer is that it is legal to reverse engineer something if you own it.

That said... I think there's an opportunity here:

EFF and other pro-reverse engineering organizations should sue and demand that the DOD follow the same laws that citizens follow.

If they lose the lawsuit, it be great ammunition (so to speak) for stopping anti-reverse engineering laws.

If they win... bonus!


> EFF and other pro-reverse engineering organizations should sue and demand that the DOD follow the same laws that citizens follow.

They'd have to have standing for such a suit and they have none. NG and others involved in the design and manufacturing might have standing, but they're also unlikely to sue. This is not a novel activity for DOD.


>Obviously the answer is that it is legal to reverse engineer something if you own it.

Or the contract allows it.


And the patents would have expired after 20 years anyways.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: