Hacker News new | past | comments | ask | show | jobs | submit login
NASA solved a $100M problem for five bucks (2012) (gizmodo.com)
237 points by rbanffy on Jan 23, 2023 | hide | past | favorite | 124 comments



I did something like this once! I was BBSing and my CGA monitor's vertical coil died. So I blew through my lips to make a 'raspberry' sound which vibrated my eyes and allowed me to read what was being displayed on the single scan line.


So when people say "10x developer go BRRRR", they are specifically referring to you?


At long last we've found the mythical 10x develobrrrrrr


Much less clever, but I had a microwave with far too opaque a grid to look through. By shaking my head left and right, it would sort of disappear though. The funny thing is how automatic these human things are. I didn't notice I was doing it until someone asked me why I was shaking my head at the microwave.


I've noticed the same thing happen when you walk quickly by a fence with slits too small to see through when you're standing still.


> I didn't notice I was doing it until someone asked me why I was shaking my head at the microwave.

Congrats, you became the human owl.


I think that's more of a ocular SLAM kind of thing for depth mapping


This also works when trying to focus on a distant object through a screened window.


I'll try doing that at my boss, see if it makes him any clearer.


Thanks for sharing this story, I laughed out loud trying to visualize the notion of someone staring at a semingly blank screen doing that in order to read the "secret text". Brilliant.


I don't know what any of this means but making it up in my head comes up with a pretty amusing scene.


OP was reading text on an old Cathode Ray Tube monitor. It broke such that, instead of being spread throughout the screen, all of it was squished vertically into a single horizontal line. However, different parts of the line are written at different times, so if you vibrate your head, different parts will land at different parts of your eye.


Sadly blowing raspberries no longer works for debugging new fangled flat panel displays. On account of the ball bearings.


You forgot to mention the pixel return springs but you seem like a nice guy so I'll let it slide.


OP is probably using something like a retroencabulator where rather than the power being generated by the relative motion of conductors and fluxes, it’s produced by the modial interaction of magneto-reluctance and capacitive diractance. The original machine had a base plate of prefabulated amulite, surmounted by a malleable logarithmic casing in such a way that the two spurving bearings were in a direct line with the panametric fan. Thus no pixel return springs are necessary in this configuration.


I'm puzzled as to how this works with respect to phase and "vsync". How is it that what you saw wasn't scrambled, even if spread out vertically again?


Displayed would have been a static unmoving screen of text (a typical BBS display), so the unsynced perturbations of haphazard raspberry motion should create enough randomly coincident content time-slices to form a complete image.

EDIT to add: someone please make an emulation of this, with a slider to control the raspberry's pitch.


Reminds me of an apocryphal story about a packaging line in a cereal factory. They created a complicated weighing system to make sure no box was packed empty because of a cereal bag accidentally falling out....only to realize the line workers had already solved the problem by placing a pedestal fan that blew away the light empty boxes as the went on the line.


Right after high school I worked at one of the big office furniture companies (you've sat in one of their chairs/cubes if you've worked in an office job). At some point, a couple hundred batches of $1200 chairs went out with one of the legs cast wrong. The easy thing to do was to have a piece of the packaging stuffed in the leg during assembly.

The problem was, the service techs never knew about this, so when they'd go out and warranty a caster, they'd find a bunch of cardboard or whatever jammed in the leg, put the caster back on without replacing the cardboard, and the chair wouldn't be level. So, they would come back out next week with a new leg.


Wait.. so the cardboard was structural?


It sounds like the equivalent of placing a bunch of napkins under the foot of your restaurant table to keep it from wobbling except the napkins were inside the table rather than on the floor. Pretty amazing hack. :)


Though this was back in the 90s and things might have changed since then, when I stocked the cereal aisle, Kellogg's had somewhere in the range of 0.5% to 1% empty boxes. Really weird the first time you pull one out of the shipping box to find the individual box fully sealed but empty.


If I recall the story continues where the complex system included a sensor to detect an empty box. Realizing the fan worked they disabled the complex system until one day the fan quietly failed without anyone noticing and empty boxes where sent out. This created a great deal of embarrassment for everyone involved.

Even more time and effort was spent after the problem was already ‘fixed’ twice at which point they used both the fan and the more complex system.


This seems a bit unlikely given that air has long been very widely used in production lines to remove objects that are too light. But it makes a neat story.


1. NASA also made design changes like adding mass dampeners to the rocket to reduce the pogo oscillations (although I don't think the hardware was actually built before the program was canceled).

2. The fix described in the article definitely cost more than $5. The ICs might've cost <$100, but I wouldn't be surprised if the cost of the fix was six or seven figures after taking into account the design, installation, and QA steps


Adding water to something (dampening mass) is less effective than adding something solid to it (damping mass). One of my favorite little pet peeve typos...


Thanks for the correction!

Have you seen the designs for interplanetary spaceships that use the crew's water supply as a protective shield to block radiation? I wonder if they need mass dampers for their dampening mass.


I haven't, cool idea! I bet they do need to do something to minimize slosh dynamics.


freeze it :)


Huh. TIL. Do you know the reason for distinguishing between the two in this way?


It's a play on words, to "dampen" something is to make it moist.


Welp. That's what I get for commenting before coffee!


It's like saying that software built using an open source languages and libraries was built for free.


Back when I first started learning digital circuits, the first one would turn on an LED on and off, the rate was controlled by a potenciometer. I had a bit of fun changing the rate to see when the on/off became invisible. With a 'scope hooked up you could see the rate of the square wave.

Fast forward decades, and some friends were working on a portable device with an LED display. It had a power consumption problem. I suggested instead of turning the LEDs on, to strobe them. They'd use much less power, and the difference would be imperceptible to the user.

Then they had the idea of strobing the line of LEDs to form text messages you could see if you waved it back and forth, but was just a line of on LEDs if it was stationary. That worked surprisingly well, but as a toy it never caught on. But it was fun.


There absolutely were popular-ish toys in which a spinning line of LEDs wrote messages, at least a few years ago in Brazil. I can't find a reference.


If you pack the LEDs more densely, you can display images as well, and in front of the right background it somewhat looks as if it was floating in the air. This makes them somewhat popular for advertising.

The first one one google is https://holocircle.com/en/ but you will readily find devices on Amazon or at the retailer of your choice if you search for "hologram fan" or "hologram projector"


I worked for a company back in 2009-10 that was trying to make giant (8ftx12ft ish) LED billboards by using 6 spinners with LED arms that overlapped (IYSWIM.)

Only saw it spin up once and it terrified me - all the wheels had to be chain-linked and -driven because they'd tried keeping 6 motors synced up at the same exact speed whilst spinning round hefty 4ft propellors and it just led to the arms hitting each other and "rapid unscheduled disassembly" - not what you want for your roadside adverts.

They also made huge cylindrical ones that were actually deployed at a few places in London and around the UK. Never really took off because the mini-PCs inside[1] tended to shake themselves to bits after a few months of spinning around at 4000rpm.

[1] I obtained my first Mac Mini from them free because they'd tried that in a cylinder, the hard drive Did Not Like It, and they were just throwing it away.


There are a number of things like this. I can't find them now, but they used to make spinning displays. They had a single column of LEDs, but when you spun them, it would leave an afterimage behind, so the message "floated" in the air.

Now, you can get fancier items, like https://www.etsy.com/listing/491059634/strange-light-up-led-...


You're likely thinking of these displays from companies such as: https://hypervsn.com/ https://holofanco.com/


No, that's way more complicated.

They used to make spinners which had something like a dozen red LEDs in a vertical configuration. Very simple.

Resolution was like this https://www.amazon.com/128pixel-indoor-electronic-display-me..., but it only displayed one vertical slice at a time.


Isn't the item in the link just an acrylic glass with patterns lasered on? With persistence of vision you can get animations, so I would say that's the fancier one.


They are generally labeled with "persistence of vision".


Those still exist! Search for USB LED fan.


I use these: https://www.monkeylectric.com/

I'm surprised they're not more common



That’s kinda how Nintendo Virtual Boy worked. Though, they moved mirrors instead of waving the whole LED assembly.


1. Does the solution includes pay for the people to think?

2. Does it include opportunity cost of waiting for the 5$ solution?

3. Does the cost include extra prototyping and engineering time?

The title of the article is a lie based on misunderstanding of engineering.

After all engineering is done, the __remaining cost__ is the same as bill of materials...

Until you need to update the system with new components. And rockets made by NASA are made in few copies... so the Non-Recurring Engineering cost dominates the whole business.


5 USD of material, millions of USD spent to train, hire, motivate and retain the engineers who are finding such solutions.


Ford, whose electrical engineers couldn’t solve some problems they were having with a gigantic generator, called Steinmetz in to the plant. Upon arriving, Steinmetz rejected all assistance and asked only for a notebook, pencil and cot. According to Scott, Steinmetz listened to the generator and scribbled computations on the notepad for two straight days and nights. On the second night, he asked for a ladder, climbed up the generator and made a chalk mark on its side. Then he told Ford’s skeptical engineers to remove a plate at the mark and replace sixteen windings from the field coil. They did, and the generator performed to perfection.

Henry Ford was thrilled until he got an invoice from General Electric in the amount of $10,000. Ford acknowledged Steinmetz’s success but balked at the figure. He asked for an itemized bill.

Steinmetz, Scott wrote, responded personally to Ford’s request with the following:

Making chalk mark on generator $1.

Knowing where to make mark $9,999.

Ford paid the bill.


Picasso was at a Paris market when an admirer approached and asked if he could do a quick sketch on a paper napkin for her. Picasso politely agreed, promptly created a drawing, and handed back the napkin but not before asking for a million Francs.

The lady was shocked: “How can you ask for so much? It took you five minutes to draw this!”

“No”, Picasso replied, “It took me 40 years to draw this in five minutes.


Great story but I had no idea who he was. Found on Wikipedia https://en.wikipedia.org/wiki/Charles_Proteus_Steinmetz


There is a great series on YouTube:

https://m.youtube.com/@KathyLovesPhysics/videos

Really took Tesla down a peg. (No, Tesla did not invent the 3-phase power system we use today. That was a Russian in Switzerland, Dolivo-Dobrovolsky.)


I guess great minds think alike but fools rarely differ. I posted a link to the story you copied just below you, lol.


This also goes a long way to explaining why service economies are richer than goods-based ones; and why "manufacturing jobs" as a political project are mostly populist nonesense.


A lot of intelligence goes into manufacturing and a lot of innovation comes out of it. The overemphasis on financial engineering causes substantial market distortions and in my view is the principal driver of our increasingly dysfunctional economies. I.e. you don’t need to be a populist to care about manufacturing jobs


As a solution to class mobility, you do. And as a route to increases in high-paying low-skill labour.

My comment about "service jobs" is kinda evidenced with your own remark: the existence of high-value manufacturing jobs. The high-value here is always in the cognitive dimension, rather than the physical.


My assertion that more intelligence is required than commonly thought scales all the way down to the low value manufacturing. I’m often quite surprised at the ingenuity and skill that goes into low value manufacturing. In addition a substantial low value manufacturing base reduces the incremental cost of resources and tooling required by the high value manufacturing increasing the viability. Effectively setting up a cross subsidy.


Sure, but the political project is supposed to provide such jobs to people who are otherwise not very skilled.

My point being that the relevant political project for increasing class mobility, prosperity etc. is skills/education/training/etc. rather than this false-belief that there is something called "Manufacturing" which pays well without such things.

Or, to use the example given, it's "knowing where to put the chalk", rather than removing the metal plate. There is no modern rich economy which can sell "removing metal" as the route to prosperity.

That used to be the case, but all that labour has been automated away or is otherwise not well-paying,.

Now it is all about value-add via skills.


My concern is ‘modern rich economies’ are a temporary aberration due to financialization and that the intelligence vs work schism will fall apart due to the under-appreciation of the importance of maintaining a manufacturing base. I would agree with you if the windfall of having been intelligent was invested in becoming more intelligent and thus an advantage is maintained but by my observation that has not occurred. It seems people will instead use their relative luxury to hold increasingly illogical ideas. Not only will we not have a manufacturing base but we won’t be intelligent either. For now financialization is propping up our way of life but that won’t last forever.


Interesting view, imo, probably wrong.

We imagine the "value-add" due to advanced cognitive skill is somehow "less real" than that which could be replaced with a machine arm.

What we have discovered, and will do so, is that knowing where to place the chalk was always the basis of profit.


At no point did I suggest cognitive work was less real.

And what point is knowing where to place the chalk if there is no machine upon which to place it. Proximity is important. Then there is the matter of scale and a question of how many pure knowledge experts are really needed.


I think one point worth appreciating (apart from the political reasons to drive manufacturing jobs) is that manufacturing provides jobs where people who are uneducated/low-skilled but intelligent can make significant contributions.

Putting a fan next to a conveyor to filter out empty boxes doesn't require a college degree (if anything a college degree makes it more difficult to find that solution), but it is nonetheless a significant process improvement.


Like Charles Proteus who sent Henry Ford a bill for $10,000 for drawing a circle on a generator with chalk, it was $1 to draw the circle and $9,999 to know where to draw it.

https://www.smithsonianmag.com/history/charles-proteus-stein...


Exactly like old joke:

There was a problem with machine on factory floor, company tried to do everything already so they called specialist. To cut story short, specialist went around the machine took hammer out and smashed in one place - machine fixed. Invoice for the company $500 - so manager was angry

- "one swing of hammer $500?" - "swing of hammer was free, knowing where to hit $500"


It's no joke, dude. That was Charles Proteus Steinmetz.

https://www.smithsonianmag.com/history/charles-proteus-stein...


I was a guinea pig for this experiment- they spun us up in the centrifuge, vibrated the chair in a simulation of launch forces, and had us doing reading and manual tasks. There are parts of your brain that nope out at certain frequencies, it's a remarkable feeling.


Can you tell a bit more about how it actually felt? :)


The centrifuge itself- you get a little nausea as it spins up, and then it feels like forward Gs; an elephant sitting on you or the bottom of a dip in a roller coaster that never stops. Even at higher Gs it's just a continuous crushing that makes it arduous to breathe, but they never spun us civilians up to the point that anything seemed life threatening.

But then on some of the vibration runs (they did a bunch of different frequencies), there was this overwhelming all-focusing urgent need to make it stop. It just felt wrong, and dangerous, and there wasn't room in the head for anything but this worry that something dreadfully wrong was happening. It pushed out any other thoughts.

The target runs (and this was a decade ago so I'm dragging through memory) were mostly just visual effects, though. Everything was bouncing around and you couldn't focus on anything well enough to read; and there was a bit of the worry in my thoughts but not the panic from other frequencies.


Quite interesting! The part about vibration and the feeling of need to make it stop, I wonder if it is the same as human responses to things like panic-inducing infrasound. Thank you!


2012. Incidentally, the single test flight of Ares I rocket ended in what would likely have been a disaster, or at least an abort, with a real crew. Those vibrational issues ended up causing the the upper stage simulator to enter a flat spin upon separation from the booster.


Yeah the whole premise of the article is off. I guess “NASA tried a $5 fix but it wasn’t sufficient and the program got cancelled” doesn’t generate as many clicks.


My favorite thing is "Our new design is so screwed that it violently vibrates to the point that the crew cant read screens...How can we modify the screen to work?" rather than "Hey..uh, how do we make it not violently do this?"


Quickly-approaching-deadline mentality.


More like "You're only getting paid to do this because the solid rocket motor company is politically connected. Either use them, or the program goes away and you get fired."

Nevermind that solid rocket motors is and always have been a terrible idea for crewed launch systems, and were probably the worst mistake made on Shuttle.


"steam-actuated dials" - this article is trash. they are trying to say "steam gauges" but for some reason are up-complicating it and since they don't know the subject they made nonsense.


Immediately raised a red flag, since I've never heard of "steam gauges" until today, it took me a minute to realize they had confused the term, and obviously have no idea what they are talking about.

> Analog gauges, commonly called “steam gauges” because their faces resemble a steam pressure gauge

https://www.aopa.org/news-and-media/all-news/2017/march/flig...

Contrast with "glass panels" or "glass cockpit" which just means "using a LCD" or similar (nevermind that the "steam gauges" also have a glass front)


> Special thanks to Mark Rober, Jessica Culler, Dan Goods, Val Bunnell, and everybody at NASA JPL and NASA Ames for making this happen.

Well, that's Mark Rober before bis YouTube fame.


As far as I'm aware, this problem (and associated 'solution') still exists on SLS, which would make sense considering that it too gets a large portion of its thrust from those giant SRBs.

Of course the proper solution would have been to not mandate the use of such large SRBs since the vibrations still make SLS near useless for launching scientific payloads. The reason for choosing Falcon Heavy for Europa Clipper instead of SLS was that on top of the rocket price difference, it'd cost $1B extra to make Clipper able to handle SLS's vibrations, Congress was insistent on Clipper launching on SLS but backed down when they informed that they'd have to spend an additional $1B on science instead of corruption.

But alas, that sort of dumb stuff is what design by Congress gets you.


Related:

NASA solved a $100M vibration problem cheaply by strobing the display - https://news.ycombinator.com/item?id=28109420 - Aug 2021 (15 comments)


The Ares I had an even bigger flaw in that any abort during the first minute would result in the capsules parachute intersecting with fragments from the SRB plume, killing the crew.


That's an 11 year old article. Might want to indicate that in the subject/title, as is customary.


And find a source with a less clickbaity title. It wasn't a $100M problem, it was a small hurdle on a $100M project


The best engineers come up with the simplest solutions.


I think it's because they know the correct questions to ask.


So the astronauts could read the numbers, but could they do anything about them? I'm guessing that it is difficult to make manual actions with any precision while being subjected to that sort of vibration. Alternatively, if there was nothing they could do at that stage, did they need to read the numbers?


Yes, in the experiment we had to touch highlighted groups of numbers on the screen during some phases, do simple math on groups of numbers, or read out numbers. One of the harder parts is repeatedly reaching out to touch the screen under G forces, your arms get really tired really fast.


Reminds me of 2005 when I suggested BART cannibalize OTS noise-canceling headphones and put some speakers under the cars to attenuate the horrific screeching sounds

http://lumma.org/microwave/#2005.07.22


Remind me scene of movie Contact. Earth engineer fixed a seat to the alien craft as minimar support for Jodie Foster but the vibration was too much. Ended up that have no seat and fluctuating free on air was easiest most secure solution.


So instead of relieving the astronauts from an additional vibration force of 0.7G, they decided to take the cheap route?

Also how are the astronauts supposed to reach for the control buttons if they can't see them?


If the vibration still meets the requirements, then yes. They're not going to chase arbitrary ideas of "better". In the physical world, you add some margin to the design so you can tolerate stuff like this.

They can see the physical buttons because they don't refresh and momentarily disappear like a digital display.


Reminds me of that old joke (not sure if it's true or not) that NASA spent millions of dollars developing a ballpoint-pen that could work in zero gravity, and the soviets just used a pencil.


The Fisher space pen was developed outside of NASA. The Soviets used a pencil, but it came with the very real downside risk of graphite particles migrating everywhere.


The complete story is that NASA was spending millions on space pencils, because of floating, very sharp, bits of graphite creating various risks.

This was also after the tragedy of Apollo 1.

Meanwhile Soviets used grease pencils, which while safer, were horrible in usability.

Fischer's private development of space pen was quickly picked up by both US and Soviet space programs to solve the problem of writing in space.


NASA spent no money developing pencils. They bought a batch of already-existing mechanical pencils, from a regular pencil manufacturer, and they paid too much. The total bill came to something around $140 per pencil, for a total expenditure of just over $4,000. This caused an outcry and eventually evolved into the "millions of dollars" urban legend.


I didn't say they developed them. However, they did a lot of tests on top of that $140 per pencil purchase which helped propel the extrapolation of "millions" (probably in TCO, spread over time)


Zero available sources corroborate that. It was purely the cost of the products that caused the blowback.


I bought a Fisher Space Pen, as a teener. It leaked. Not very badly.


To expand a bit on what you wrote. The Fisher space pen was developed entirely with private funds, and sold to NASA to fill a real need to not release graphite to be inhaled or cause short circuits.


Cool stuff! How it's made / how it works: https://www.youtube.com/watch?v=3mjmAzf4oKs


Flammable materials migrating into circuitry.


Free graphene!


A story that is 100% B.S.


not sure if you are aware of this

https://www.spacepen.com/


Interesting bit about the patent application by NASA. I didn’t realize NASA filed for patents. Does a NASA patent essentially make this public?


I wonder how often they do the opposite, and spend $100M because nobody had the flash of insight needed to turn it into a five-buck problem


> I wonder how often they do the opposite

[From my contract-drafting course materials:] The parties in a 2020 Vermont supreme court case might have been better served if they had done periodic check-in calls — in that case:

- In a project to rebuild a railroad bridge, a subcontractor was supposed to handle one aspect of the project.

- The subcontractor encountered an unexpected problem.

- The subcontractor spent weeks trying to fix the problem — and billed the prime contractor an extra $120,000 for its efforts.

- At trial, however, it came out that — for only $9,600 — there had been a simpler way to solve the problem.

- If the prime contractor and subcontractor had just talked to one another — perhaps using the SPUR agenda, discussed at 7.2.6 — they just might have solved the problem sooner, at much-lower cost (and without having to go to litigation).

See Construction Drilling, Inc. v. Engineers Construction, Inc., 2020 VT 38 ¶¶ 6-7, 236 A.3d 193, 196-97 (2020) (affirming denial of subcontractor’s breach-of-contract claim). https://scholar.google.com/scholar_case?case=144056062390412...

[The SPUR agenda is a general-purpose meeting agenda: Status; Problems; Uncertainties; Risks.]


> NASA has a patent pending on the technology

That kind of bugs me. Why should our government be able to patent things?


To answer your question, NASA has the authority to grant licenses on its domestic and foreign patents and patent applications pursuant to 35 USC 207-209.


Sounds like they don't have to grant license to use the patent.


To keep others from patenting publicly-funded inventions?


Great engineering. Elegant solution while solving the general case - usually these avoid the general case


There is an underrated approach to solving problems - assume at the outset that the solution is simple and obvious. If it is, you've just saved massive amounts of time and effort. If it isn't, then at least the rest of your deduction process is calibrated.


> We did our best to show the before/after by putting our camera on the sled, but the image-stabilization was just too damn good (well played, Sony. Well played). You'll have to take my word for it.

Alternate solution: send crew up with image stabilized cameras.


This would turn out to be the absolutely incorrect solution. The core design of the system was driven by politics creating all kinds of engineering problems that would eventually doom the program. The actual solution was to cancel Ares I.


Is it really impossible to solve those oscillations? If the spacecraft is being shaken so severely that people cannot see, then surely other designs are being impacted as well.


This is a common issue with solid rockets - they vibrate a lot. There are payloads that are suitable for, say, Falcon 9 that can't fly on the SLS because of the SRBs. Other fixes were employed to avoid the entire rocket to enter resonant vibration (IIRC, resonant frequency of the solid rocket was something around 12 Hz and with the fueled upper stage at 14 Hz, it was too close for NASA's comfort). A series of vibration dampers were added to the booster skirt and, something, I don't remember what, but probably another set of dampers, was added to the interstage.

Ares I was, to put it simply, a bad design. By the end of all this, NASA decided, correctly, to ditch the idea.


Would this also be fixed by using a display with a high refresh rate? For instance, would a 240hz display be subject to the same vibration smearing?


Yes, the blurriness comes from the movement of your eyes. A constantly-lit piece of paper would also look blurry.


I meant "yes, it would be subject to it" so "no, it wouldn't fix it". In case it's not clear which question I was answering.


Did any other industry actually use this solution as suggested at the end of the article?


It was not really a solution.

But those vids on YouTube of helicopters flying around with still rotors, and hummingbirds with their wing held out, are great.


TL;DR summary:

Problem: cabin would inevitably vibrate at some point and astronaut's chairs would too, making displays impossible to read.

Soution: put a sensor on the chair and synchronize the displays to only pulse their light when the chair (and thus astronaut's eyes) returned to the same location.


The strobe and accelerometers cost more than five dollars.


are they using a pencil? /s




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: