Hacker News new | past | comments | ask | show | jobs | submit login
Black Start (wikipedia.org)
289 points by anthonyb on April 29, 2013 | hide | past | favorite | 105 comments

This was one of the more interesting topics in my EE classes on "high power" systems. We toured a gas plant in Long Beach California which had 'internal black start' capability. Sloshing all that power around is a complex business and fraught with danger. Since I didn't expect to deal with much over 5V DC I pretty much ignored the boring bits (only to have them come back to haunt me when provisioning power in data centers.)

I'm doing some work for a hedge fund at the moment that is placing diesel-based power generators at locations across the southwest. These generators are equipped with cellular MODBUS controllers that can be written to via a REST API and get the power generators started within 4 to 5 seconds. The goal is to take advantage emergency peak load demand on the grid and profit from short-term spike in electricity spot prices.

I'd love to see the spreadsheet on that one, that must be incredibly well thought out to be profitable. How often those 'emergency peak loads' exist, their duration and the write-off on equipment and fuel costs must have kept the designers of that system up for many long nights.

For areas such as Texas, New Mexico, Arizona, Florida, they happen very often during the summer. In Houston, last year we almost experienced load shedding (rolling blackouts) several times. ERCOT (the Texas ISO) forced large industrial customers to scale back demand to keep from imposing residential blackouts.

ERCOT sends out Emergency Response Service (ERS) notifications to QSEs (Qualified Service Entities) during these periods via a WS-Notification. The hedge fund predicts that during these events they can make several tens of thousands of dollars a minute with the diesel generators that they have deployed across the state. I think they believe that they can because they're having me write the code to start the units based on ERS notifications.

Wow, that is a whole lot cooler than building a CSP plant in the Mojave to feed the California grid on peak days. Back when I was looking at power usage inside of Google the diesel backups at the data centers didn't seem to provide very economical power. Especially when you took into account generator runtime vs maintenance required (there was something like 1 hour's worth of maintenance for each single digit hours of run time).

Given the efficiency of the Bloom boxes [1] it seems like you could print money with them if you could get a fast start version.

[1] http://www.bloomenergy.com/

Incredible comment. Thank you very much.

This is awesome -- a hedge fund is exploiting financial/market inefficiency and creating positive externalities (more efficient/reliable power grid). And hopefully making a lot of money doing it.

Sloshing all that power around is a complex business and fraught with danger.

Why's that?

While not directly relevant, the scale and scope of such power transmission is illustrated in an awesome anecdote by "Thomas L. Mc Mahon".


Like JWZ, this has become a favorite of mine to share.

Wow... that's impressive engineering. Thanks for sharing.

I remember that from last year. It's a fascinating read!

Ok, this is tangential, but how is it that one of the founders of Netscape/Mozilla has a website that is so painful to look at? Flourescent green on black.. my eyes!

I'm not new here.

That, however, is taking nostalgia far beyond any sense of probity. It definitely ain't what it used to be.

tlb of yc fame, turns out, is the guy who wrote the analog-tv-artifact-simulation code in jwz's xscreensaver:


I have little doubt that all of jwz.org text rendering will be fully analogue-simulated once the monstrosity that is the supermajority of the installed base of web browsers supports such folly.

I'm still trying to pry my jaw off the floor. Really, it's a thing of beauty. A sick, sadistic, twisted, nostalgic, mental beauty, but beauty all the same.

And yes, I'm familiar with the xanalogtv screensaver. Among my faves.

$10 for a gimmick program that will get used once?

$10 to tell the person who made it that we approve of the choices he's made in his life.

I bought it because it simply existing brings me joy. It has the best you-haven't-paid-yet nagware implementation ever conceived by man. I never even triggered it (I bought it instantly) and it gives me joy that it exists.

I use Cathode as my regular terminal. What are you trying to say? :-)

Browse around his blog some more, I guarantee you'll find images far more painful than mere green text on black!

That said, his recent "SOMA Nature Walk" series would appeal to people into big engineering.

Because you're dealing with enough power to run a city. At a minimum thousands of kV and/or Amps. If something fails, all that energy has to go somewhere...

My Dad has a story on this one. He was a mechanical engineer in a Brisbane power station in the 50s/60s. In those days, generators had to be manually phase synchronised before being connected to the grid. You had to spin up the generator and vary its speed whilst monitoring the relative phase on a meter. Once the relative phase between the grid and generator was zero, it was safe to throw the switch to put the generator on-line. One day, someone threw the switch when the phase was 180 degrees...

The entire generator, including its concrete foundation, ripped itself out of the ground.

There was also the one about an electrician working on a generator who blasted himself with 66kV (low impedance). Massive burns, but he lived.

Since people liked that one, my Dad also had another good engineering disaster story, though this one is pressure related, not electricity related.

He used to work for a company called James Hardie, that made asbestos cement sheeting. Part of the process involved baking the sheets in a huge autoclave, like the one pictured here [1] (note the size of the man). An autoclave is an oven, filled with high pressure steam.

One day the door blew off the autoclave. The door disappeared, and the autoclave itself acted as a rocket, about the size of a shuttle solid fuel booster. It sheared off the bolts holding it to its foundation, rocketed though the back wall of the factory and ended up in a body of water about a mile away.

[1] http://museumvictoria.com.au/collections/items/756810/glass-...

I wonder if the steam rocket[1] was invented as a result of an incident like this?

[1] http://en.wikipedia.org/wiki/Steam_rocket

Yes, obviously. I was prompting someone to provide more detail than that. E.g. what types of things can fail, what procedures have been put into place to mitigate the risk, examples of failures, or perhaps an interesting story or two.

I'll bite.

I was in a gas-fired power plant when the call came in to spin up the plant turbine to meet demand. So as the turbine is spinning up, the guy misses the sync (which involves throwing a breaker when a certain light is almost extinguished; think incandescent) only to make it on the second try. At that point, the turbine goes into "follow" mode where the turbine speed is regulated to keep phase with the rest of the grid. Everything is going swimmingly until some surge causes a sudden increase in turbine speed followed by a sharp decrease. That's when the fun begins. The turbine can't spin down fast enough, which causes a phase differential, which causes a breaker to take a turbine big as a house to suddenly and violently be removed from the grid. This would normally cause an over-rev that would rip said turbine up from floor and into the air were it not for a bank of resistors that are outside that can produce a load for all that energy. When all that energy gets dumped into a bank of resistors in a switch yard on a hot summer day, you can smell it.

Electricity is no joke. Respect it.

At the Prototype reactor site I qualified on there was a pretty impressive tale of a student who managed to try to bring a turbogenerator onto the electrical distribution system approximately 180 degrees out-of-phase.

The resultant current surge generated enough force and torque to physically lift the turbogenerator off of the deck before the breaker tripped. The amazing thing is that somehow the steam piping didn't rupture in the process.

Next year the aptly-named "sync check" modification was installed to prevent shutting the circuit breaker unless the two buses had approximately equal phase.

In the '80s, I built synchroscope controllers for Westinghouse's Nuclear Systems Division that were controlled via a main-frame computer. These were used in simulators (each reactor facility has to have a training simulator that's identical to the real control room) and were specifically used to train operators to match the phase of generators they were bringing on-line.

From what I can remember, the tolerance was about +/-5% and within that range, the generators would lock to each other. If one generator was putting out AC power that was ahead of the other, the current would cause the generator to behave like a motor and speed up until it matched. The other 350 degrees of the phase dial were much more dangerous ... explosions, physical damage, etc.

Everyone else commented on the danger of non-black start issues. If you've got live grid to connect to and sync to, why would you ever black start anyway? Or more precisely that has nothing to do with the "fun" of black starting.

The actual EE-ish problem with black starts is that darn control systems class the EEs have to sit thru full of bode stability plots and stuff.

The problem is that power plants and the innards are kind of like highly tuned race car engines. The peak of technological performance at 1000 horsepower out, but at idle they get kinda squirrelly and don't really know what to do so as not to oscillate and/or stall and/or blow themselves up. For the signal/telecom EE types, black start is kind of like having a very high power finicky high gain amplifier with no decent termination and no real input and not much control, the odds of that dude deciding to oscillate are unfortunately high. Another way to put it is "the grid" is more or less predictable constant impedance you can smoothly and predictably match to, but connected to nothing during a black start or initial power up is very much like keying a high power radio transmitter when not connected to an antenna or dummy load, its going to be pretty impressive no matter if it blows up or not.

Theres some interesting impedance issues. An alternator is kinda like a constant current source, for a given field winding current it'll squirt out a much higher current from the main windings. And inductors don't like to change their current flow. And the only limit to the voltage is arc over and/or the internal resistance of the winding (well.. AC is much more complicated, but..) So overexcite the field winding on a connected gen and the voltage across town goes up a hundredth of a volt who cares. Overexcite a unconnected / black start gen and it could just arc over and explode or blow transformers and just F the place all up in general. Dump an extra megawatt-hour of energy in the general direction of Chicago and Chicago doesn't sniffle, dump it into one little piece of switchgear in the yard all by itself because its disconnected / blackstarting and the arc blows it to pieces.

Another issue not at his relatively boring gas turbine plant is stuff like differential thermal expansion. You have to heat and cool boilers and steam turbos slowly or else differential expansion can rip them apart. Supposedly if you leak even a little steam on a stationary turbine it'll expand one side enough while leaving the other side unexpanded and cold that when you slam the valve open its an open guess if you'll destroy the bearings from vibration or smash the blades into the outside of the turbine. So you keep them spinning, even if its really slow like 10 RPM, all the time, forever, unless you've got a crane on site to replace. And boilers blow up if you heat them too quickly, same problem with parts at different temps.

Gas turbos are tougher than steam turbos for a variety of reasons one being they're so dang hot and so much outside air (full of bugs and stuff) flows thru them that they can't build to the tolerances that a steam turbo can use, so they don't even try, so powering one up isn't as exciting as you'd think. Oh its loud, but not as exciting as spinning up a steamie.

Another fun feature is black starting is pretty hard on your whole backup system. Idling the diesel for 15 minutes a week is not quite as exciting as slamming it open full throttle for 12 hours while you heat up the main boiler or whatever. So if you're going to blow up your UPS / diesel / whatever its going to happen at black start time.

There's a general engineering rule that its a heck of a lot easier to design something that constantly does the same thing than something that varies, maybe randomly.

Australia's energy regulator (http://www.aemo.com.au/) runs a separate pricing market (ancillary) for producers who are able to provide black start capability to the network, amongst other things like frequency regulation (I think...).

I can't speak for Australia, but I know that both are true for US pose markets. Power is a big part of what's sold, but there are many other markets too. Black start services, generation capacity, and frequency regulation, just to name a few.

Part of the key to serving electrical demand is having a balanced mix of generation to serve your load. Some units are better running flat out, some are better for brief periods at peak demand, and some are good at quickly taking up the slack if another unit fails. Different power markets exist within the same geographical footprint to ensure that all these types of needs (and more) are met. (24 hours a day, 7 days a week, and into the foreseeable future.)

For fun, consider the question of how you'd "black start" from a loss of all AC power on a submerged nuclear submarine? :)

Finally, a HN thread that I can contribute to and I'm 9 hours late to the party.

I was serving in the engineering department on a USN aircraft carrier (non-nuclear) when a cascading casualty led to all boilers going off line. The ship had an emergency diesel generator aligned to automatically power up, but it failed to start due to a previously-unknown problem. So an 80,000 ton warship with 5,000 people on board slowly drifted to a stop, completely dark except for emergency 60's-era-battle lanterns that were already starting to dim.

No steam = no electricity (the normal ship's generators were steam-powered) and no electricity = no steam (because the start-up feedwater pumps and air blowers for the boilers were electric). One of the other two EDGs was already down with a casualty which left a single diesel generator that had to be started with compressed air stored in two huge bottles--so basically had two chances to get the ship moving again. (No electricity = no air compressors to refill the bottles.) After a few hours of double- and triple-checking the alignment of the diesels, electrical systems, and boilers (the original steam plant casualty had to be diagnosed) using flashlights in temperatures well into the triple digits (no electricity = no air condi...you get the idea) we successfully started the one remaining diesel which provided AC to the boilers to start up.

By far the scariest day I ever had in the navy. Only saving grace was that we weren't flying planes at the time.

Could you have hacked something up using a starter cart or even an aircraft or APU or something (or a launch or cable to an escort), if the two tries had failed, but for some otherwise-corrected problem so a third try would work?

Or I guess just transported some compressed air from an escort, too -- I'm not sure what volumes you're talking about, and if the other gas cylinders a warship probably has on board would be suitable)

Presumably they have more than one of the same class of ship, which (further presumably) has the same emergency start tanks (and air pumps + power to refill them for additional tries).

Could be a loooong wait, though. I guess that's why they have two.

Interesting story, thanks for sharing. Glad I'm reading about it and not living it.

I don't know much about this sort of thing, so I'll ask the obvious dumb question: why no batteries to start the diesel generator? Was it too big?

Edit: Reading on, it seems like compressed air is the norm. Is it more space efficient than a battery big enough to start a large generator?

Square/cube law strikes again. You can only run an automotive starter continuously like a minute or so (at absolute best) before something melts down and the system is scrap. Scaling the power cube up by a factor of a thousand or so and the cooling square surface area up by a hundred or less and suddenly you get like one revolution of the motor before the starter windings and relay/contactor melt down. Not pretty. Of course you could make everything unimaginably immense but that starts making air tanks and air compressors look cheap.

I think it was because the system was simpler and more reliable with no moving parts besides valves, just a giant air bottle with pipes leading to each of the cylinders. No motor to worry about or lead-acid batteries to maintain. The whole system probably took up way more room than a starter motor would but excessive space or weight is not a big concern on an aircraft carrier.

I think it also had something to do with age - newer vessels (submarines, at least) use a large lead acid battery array for emergency power when needed... and by emergency power, I mean enough power to start the emergency diesel generator.

Submarines do have a large lead-acid battery for emergency power. In fact, the WWII-era boats had bigger and better batteries.

However the battery isn't there to start the Emergency Diesel Generator. Submarine EDGs are air started, for many very good reasons.

We still need the batteries since it's possible to suffer a loss of all AC power while the boat is submerged far beyond feasible snorkel depth, and have to recover from it without broaching.

I can think of a few possible answers:

1) wouldn't you just insert/remove control rods as needed to restart the nuclear chain reaction? i'm pretty sure most reactors design control rods to be manually insertable/removable, even if there are other way to insert/remove them

2) a standby diesel generator with good ventilation, especially passive ventilation?

3) a large dc power source (ie big, charged batteries) to "jumpstart" either a diesel generator or the nuclear reactor the way my car battery jumpstarts my car when I turn the key

4) some other form of stored potential energy and a generator that can extract it (eg: compressed gas through a turbine) to start 1 or 2.

5) (really just a subsection of 4) human powered generator of some kind. perhaps a few stationary bikes?

#1 actually no, a lot of designs have a spring pulling the rod into the core, and an electromagnet sticking to the rod, so if the plant utterly loses power the rod snaps back into the core and can't be pulled out. In theory you could attach the rods to bypass the safety feature but its just not happening.

Another issue is you run the coolant pumps to dump a couple MW into the system to reach operating temp/pressure after a day or so, THEN light off the reaction so you don't have varying steam conditions affecting the nuke physics. You don't "heat up" a nuke under nuke power as most assume. Its already thermally hot operating temp when you pull the rods.

so if i'm understanding correctly, from a black start it would take at least a day for the nuclear reactor to come online?

then you'd need at least a couple diesel generators and fuel or power cells to get the reactor into a state where it could start generating power.

Since you said megawatts, there will probably need to be a smaller generator or two to start the generators that will start the reactor, since i think MW diesel generators take a fair amount of work to get started themselves.

And we haven't even talked about backup generators in separate parts of the boat/ship in case of engine fire/damage.

Also, there would probably need to be at least another generator/power cell whose only job is to ensure that the sub can blow its ballast at any time (and of course, just regulate it) that comes up as soon as the reactor goes down.

I was mostly talking about stationary plants. You'll never get a straight story from the navy guys.

There's the slowest safest possible way to carefully test and monitor a power up guaranteed to cause minimal maint issues and longest possible life and highest safety to the civilian city downwind, and then theres OMG the missiles are flying we gotta get outta port right now. And there's numerous engineering changes you can apply to make things faster. And a tiny little 10 MW plant or whatever a sub has is a whole different kettle of fish than a 1 GW stationary plant. And the stationary plant has to care about efficiency because they're trying to pay off construction loans so any PITA that increases output is a win, but the sub boys have no (direct) loans to pay off and as long as the boat moves when the skipper says, the efficiency doesn't matter.

So they're going to be different.

But yeah, even booting up a sub is going to be some time and some work, even if you're in a hurry.

And there's historical issues like xenon poisoning which boils down to if you shut down a reactor in a "bad" state of the chemical shim system (which is kind of like intentionally dissolving a control rod in the coolant) after it was run hard and put away wet such that Xenon builds up in the core, that dude is not going critical for a day or so even if you pull the rods completely out, until the shim system cleans up the coolant... at least on land. I suppose if I was in the .mil I'd have a way to dump distilled water into the coolant for just such a situation and/or I'd have regs to never run the shim system that high and/or I'd have so much excess reactivity in the control system that, well, a landlubber would be nervous about it, so you could just burn thru the xenon poisoning anyway. And/or a regulation that says no shutting down the reactor in a rode hard / put away wet scenario so xenon would never be an issue. I mean if you're not paying for the fuel, and you're worried about xenon poisoning, one way around it is never shut the thing off, ever, unless its a major radiological issue. That's certainly one way around it.

Generally subs compress air before they need it, so there's usually plenty available before something "happens". If you're just pumping up the tanks right before you need them you've probably got serious problems or doing something completely ridiculous...

"Black start" of a sub would strictly be mid-ocean adventures which are probably classified and/or completely BS sea stories. Normally they power up off shore power while in port. I would imagine this does nothing for secrecy, the lights dim at the harbor and the backup generator is idling, hmm, I wonder whats shipping out tomorrow?

There are often no backups. The front of the sub usually isn't as sensitive as the engineering spaces and the engineering spaces aren't large. There's no snorkel coming out of the forward torpedo room so there's no backup diesel in there, besides, where would the torpedos be? I'm told by actual sub people that the front half of plastic models of subs are usually pretty realistic and usually based off completely declassified stuff, but the back half of most models is pretty much modelers artistic license time, and that's all they'll say, which I guess is how it should be...

I know the air is compressed when they flood the tanks, but I'd assume that some power is needed to blow or regulate ballast, just from a mechanical standpoint of opening and closing doors.

I'd be surprised if there were no backup generators at all. Maybe not in the front, but somewhere where the primary backups aren't. Shit happens.

Compressed air can be directed wherever it needs to go by installed piping and valves, and the valves themselves can be hand-operated.

Submarines do indeed carry an emergency diesel generator, and they also have a fairly large battery, but the emergency main ballast blow system can be run off of nothing but previously-stored energy and a hand-actuated valve.

Your mention of manual removal of control rods reminded me of the SL-1 reactor accident - where some unfortunate chap went to raise a control rod a few inches but accidentally pulled it out 26 inches - the resulting huge power surge and water hammer caused a very nasty accident killing all 3 operators:



Big ole batteries - you'd need them anyway to run systems in the first place, should the reactor be shut down at depth. It looks like something like, say, a Los Angeles class sub can actually run its generator from batteries

"[...]power can be provided from the submarine's battery through the Ship Service Motor Generators (SSMGs)."

This is a problem that the engineers designing nuclear submarines have thought long and hard about - the USS Thresher and all crew were lost in part due to inability to rapidly restart a scrammed reactor.

Congrats, great answer.

Everything I learned on the boat just taught me that I have so much to learn. So many minor details that a ship designer thought up went into each successive class of U.S. submarine. There were many days when I'm surprised again that we haven't lost more submarines due to design flaws or misfeatures.

Hah, that is surely one of the many differences between reading about a sub and serving on one.

Me: US navy nuclear submarines are a much-cited example of the successful design, development and deployment of high-reliability systems.

You: I sometimes wonder how it is that more of us are not sleeping with the fishes.

Still, everyone from designers to crews must be doing something right since it's apparently possible to drive such a machine into a rock at flank speed[1] or through a boat[2] or have it commanded by someone exhibiting strikingly poor judgement[3]. Without killing everyone on board and better yet, the planet.

[1] http://en.wikipedia.org/wiki/USS_San_Francisco_(SSN-711)#Col... [2] http://en.wikipedia.org/wiki/USS_Greeneville_(SSN-772)#The_E... [3] http://www.huffingtonpost.com/2013/04/12/michael-ward-affair...

I'm certain nuclear power submarines have two nuclear generators always running to prevent this from happening. I also think they may have a diesel powered generator for bootstrapping.

Only 1 U.S. submarine has ever had 2 nuclear reactors (bonus points for those who know why it was designed with 2).

Soviet submarines often had 2 though.

For the U.S. submarines they frequently would test shutting down the 1 operational reactor at-sea and trying to recover from the casualty, just to prove that it could be done when you didn't have time to pre-plan.

Edit: U.S. subs do have diesels though. So next question: how do you bring up the diesel engine and its required auxiliaries without AC?

Here's the black start procedure for the 1950's-era ship I worked on. No idea if this is at all similar to how they start diesels on a modern sub.

1. Hand-crank a small diesel engine connected to a small air compressor.

2. Use the small air compressor to build up pressure to start the emergency diesel generator.

3. Start the emergency diesel.

4. Using the emergency power, run one of the large air compressors to build up pressure for starting the main generators.

5. Start the main generators.

The emergency diesel was near the stern of the ship, away from the generator room, to ensure an generator room fire couldn't take out the emergency diesel.

I knew it! Generators all the way down!

It's actually similar on a submarine, except that there's no need to hand-crank an air compressor as we have tons of compressed air already available on a submarine.

Also, once the diesel is running it is used to power the electrical buses that provide the minimal pumps and other gear needed to recover the reactor and eventually restart (or emergency restart) the turbogenerators that provide ship's service power.

Unfortunately there's no room to have separate emergency generators and emergency diesel so they are co-located.

You skipped a step. LOLz.

I skipped quite a few in fact. I love getting people interested in submarines but I have no great desire to irritate my friends at NR by creeping past the rules regarding NNPI (whether of the U variety or otherwise).

I can appreciate that. And I was just having a little fun with you.

But I was simply referring to the part where you surface the boat before starting the diesels. I think some folks might be under the impression that you can run those while submerged which is most decidedly NOT the case.

What I think many people don't realize is that a reactor shutdown is likely to be a mission-over/career-ender type of deal for almost the entire officer corps onboard. It's not supposed to happen EVER. It's like forgetting your gun as you exit the helicopter.

BTW, It was "supposed to work" but didn't isn't an excuse. You (as an officer) are responsible. That's what maintenance and drills are for. Reactor shuts down and diesels won't come up? Wow. Major UNSAT. A dead-in-the-water carrier is almost as bad as running the damn thing aground.

I'm wondering how the CNO felt about that poodle-screw.

You can run the diesels underwater though. Obviously there is a limit, but surfacing the boat is neither required (sea state permitting) nor desired ("OH LOOK, A SUB!!").

DIW on a CVN is a big deal, that is true (though hardly a career killer by itself for O-gang, even the ones in Reactor Department). NR takes nuclear safety very seriously and the carrier won't sink just from loss of propulsion so the reactors will very much be configured to fail conservative if necessary.

Even on a submarine, which can flounder without propulsion and only has a single reactor, we do scram drills all the time. A scram can occur for no better reason than that reactor protection was feeling finicky that day, so the drills are designed to assume a scram will happen, and recover safely and efficiently.

On the other hand, an unplanned scram really is one of those things which may get someone disqualified from watchstanding, and possibly de-nuked completely, since those usually represent a crew that doesn't properly respect the power of the reactor.

I don't know that I'd call running with the snorkel up anything but non-submerged. Although I will give you that running at p-depth is almost the same thing. I'm sure you'll agree that anything other than underway on main propulsion is a far less preferable situation than normal ops.

BTW, I spent some time as a bubble-head, so I do understand what you're talking about here. Just keep in mind that not everything you've heard about subs is true and not everything I knew about subs is still true. A lot has changed since I was in.

USS Triton, needed high speed to shadow carrier groups

I would have also accepted: "needed two reactors because it was the only USN submarine with two shafts."

Technically, you mean the only USN SSN with two shafts, but even that is incorrect. Nautilus was a conventionally-driven submarine with counter-rotating screws just like its diesel-electric WWII-era forebears.

The first single-shaft U.S. submarine was a diesel research vessel, USS Albacore, which proved the feasibility of many innovations which were later ported over to the nuclear submarine fleet. USS Skipjack was the first nuclear submarine to include most of those features, including single-screw design.

Never been in the navy, so just guessing. Just kicking around ideas...

Option A: You hand-crank the beast somehow? To get the glow-plug going, you again use a hand generator? Hopefully enough to bootstrap the generator (maybe it's a smaller generator that runs a larger one)? Generators all the way down?

Option B: You blow ballast somehow and use the inrushing water to drive a turbine that runs the generator or gets it going?

Option C: You manually pull the control rods and switch the coolant loop to run a small turbine to prime the generator?

Option D: You mayday and get a jump from the Russian/Chinese/Iranian sub following you around?

Option E: You try something dumb and dangerous with hydrogen peroxide from the torpedoes (I think/hope that that's phased out now)?

I think it's mostly batteries, a sub operating at constant depth and in motion is negatively buoyant. If something goes wrong and it loses propulsion, power has to be available immediately to operate the systems that can handle the emergency - whether it's blowing ballast or restarting the reactor. This is a somewhat different constraint than the procedure described by the other reply - ships float without power, subs, not always.

I mentioned this upthread - USS Thresher[1] seems to have suffered a loss of both propulsion and ballast control while near test-depth with, unfortunately, tragic results for the vessel and all on board.

[1] http://en.wikipedia.org/wiki/USS_Thresher_(SSN-593)

Option F: Compressed air. Turns out mechanical systems are stupid simple to hand-power and yet can store a ton of energy usable without fancy equipment. Plus it turns out we might need compressed air for this no matter what depending on where the boat is at.

Ah, very cool. Oftentimes the crudest solutions are the best.

All merchant ships are required to have enough compressed air for several starts (and we're talking about much larger diesel engines). I'd assume it's the same for a nuclear sub.

With a storeable dc power source, just like a car or truck.

I suppose if that failed or had somehow discharged incorrectly, there should probably be a way to handcrank the diesel generator, like how early cars required hand cranking to start.

PS: I had to look it up, but that's an interesting reason why that us sub had 2 nuclear reactors.

Nice try, Ivan.

Remember, it's called the silent service for a reason. Those who know, don't tell; those who tell, don't know.

Water pressure?

You're certain??? Better check your sources.

That was the most interesting part of this article. Even the largest power plant comes down to a simple 12V battery needed to bootstrap the first generator.

Similarly, it's fun to realize that the entire 27-km Large Hadron Collider "runs" on a small bottle of hydrogen that you could hold in your hand.


Without that insignificant-looking bottle the whole machine would have nothing to do.

(Edit: I do have to wonder how wise it is to paint a bottle of hydrogen to look like a fire extinguisher.)

Gas bottle colour codes and fire extinguisher colour codes are two distinct namespaces... Here's a handy chart: http://www.ior.org.uk/ior_/images/pdf/se/New%20Colour%20ID%2...

Wow, human factors=massive fail with that chart. "Hmm, is this acetylene or nitrogen?"

That's not even getting into the whole red-green color blindness issue that could lead to confusing 'Flammable' with 'Inert' contents.

You can't confuse acetylene or nitrogen, old or new style; acetylene uses maroon for the whole bottle, nitrogen uses black for the shoulder only. Also the cylinders are quite different shapes.

Also the colours are for "the room is on fire, quick what's in that gas bottle over there?" situations. Are you allowed to be a safety responder if you're colour blind? For everyone else the rule is pretty simple; GTFO, and if acetylene is involved, GTFO out of the postcode.

Still, you don't go out of your way to pick confusing colors. This is Usability 101 stuff.

If the room's on fire and I'm in it, I'm a 'safety responder' whether I want to be or not.

Video games have taught us that fire extinguishers blow up so it cancels out.

Here's a video of what used to be the world's largest diesel engine, used for black starts if I remember correctly:

Built by Burmeister & Wain[1] , located at H.C. Ørstedværket [2]: https://www.youtube.com/watch?v=jPouLq3yZI8

Maybe a diesel engine of this size was required since we don't have much hydro power in Denmark (the article suggests starting a hydroelectric generating station first).

[1]: http://en.wikipedia.org/wiki/Burmeister_%26_Wain [2]: http://en.wikipedia.org/wiki/H._C._%C3%98rsted_Power_Station

I was wondering why nuclear plants require so much power to start up, when really you can't "shut off" uranium's power-generation ability. I guess it would have to do with powering up all the safety and management systems around the generator?

Uranium (or other radioactive materials) don't directly generate much electricity - they mostly generate heat. Most nuclear power plants generate electricity by having that heat turn water into steam, which is then used to power steam turbines that actually drive the generator coils. Of course, in order to do this, you have to be able to pump water into the reactor to be turned into steam, and that pumping requires power. So yes, while you definitely need power for the safety/management equipment, the major bootstrap issue is almost certainly all of the power necessary for the pumps.

The article touches on this:

Generating plants using steam turbines require station service power of up to 10% of their capacity for boiler feedwater pumps, boiler forced-draft combustion air blowers, and for fuel preparation.

Modern nuclear plants have two cascaded cooling loops and run at about 33% efficiency. This means that a 1000MW plant has 3000MW of thermal energy to manage at full load and 2000MW of energy to dissipate safely. The systems to do this take a great deal of power to run, so can't be bootstrapped off of batteries.

Also interesting is that power prices will occasionally go negative in cases where there's a 'surplus' of generation. What's happening in these cases is that generation owners are making the judgement that its more cost effective to pay people to take their power than it is to shut off their plant. Shutting down a big plant is an event, and it takes a while to get them back running. Sometimes, shutting them down doesn't make sense.

Also about ten percent of the heat energy is released later on, delayed, decay heat and all that. So there is no off switch. If you were running at 3000 MWt, 5 minutes after you scram you still have to do "something" with 300 MW of heat, and you don't want to have to rely on just one diesel. So the feds require multiple sources of power to operate, one of which is usually the diesel and one is the the grid.

Lets say your control and cooling systems require 5% of the heat they handle, that means your 3 GWt nuke would require an absolute minimum of 15 MWe to spin down. Or four if you want to run off grid so you've got two completely independent sources of power each with a 24x7 hot backup (I believe no one ever took the NRC up on that offer, but I could be wrong)

Is Wikipedia's statement that hydro stations require an external source for field excitation correct? I thought it was possible,to start a hydro generator by "self-excitation". When a generator stops, there is typically a small residual permanent magnetic field left, due to hysteresis. This small field may be used to "bootstrap" the generator, whereby on initial spin up a small current is generated, which may then be fed back to the field coil, further exciting the generator's field and increasing the output, until full capacity is reached.

All that is then required is a method of opening the penstock, which can presumably be hand cranked?

Some quick research on generator excitation:

* Each of Hoover Dam's generators appear to me, based on this photo [1], to have three stages of excitation, vertically stacked on the rotor shaft.

* A year 2010 upgrade to digital controls at Hoover Dam [2] shows:

___ o A total plant capacity of 2.074 GW from the 17 generators

___ o 20 seconds to go from reactive power control ("condense mode") to generator mode.

___ o Pictures of the original and replacement governor equipment; the red cabinetry shows a beautiful 1930s design style

* In "condense mode" [3], the turbine's penstock valve ("wicket gate") is closed off, the turbine volume filled with compressed air to displace the remaining water, then the generator operated as a massive synchronous motor. In this mode, the excitation controls can provide marketable power factor correction, aka "reactive power" for the grid.


[1] http://bayimages.net/view-photos/electric-power-generators-h...

[2] http://www.hydroworld.com/articles/hr/print/volume-29/issue-...

[3] http://www.bpa.gov/Finance/RateCases/BP-14RateAdjustmentProc...

Permanent magnet excitation is a possible way to generate the required field flux for a generator, but it's not a reliable way.

If the permanent magnetic field isn't strong enough you can't even generate enough voltage to power-up the excitation circuit at all, and that's the circuit which would feed the current back in.

So whether this is feasible or not depends on the generator design.

I've got memories of some of the Snowy Mountains scheme's hydroelectric generators being designed to start though self-excitation.

Wikipedia also has a bit about it [1]. It seems that there is usually a backup facility to run a DC current though the field winding before starting, thus magnetising the magnetic paths, for when the residual field has decayed too much. That backs up your comments about reliability.

[1] http://en.wikipedia.org/wiki/Self-excitation

Residual flux is usually enough to start, but if you lose all flux for some reason (when the core is completely demagnetize) zapping it with DC can help restore it.

Theres a lot of word choice issues in this discussion. A generator, like on a old fashioned bike light, does not require a field winding it uses a magnet. Magnets can't be controlled as you'd guess so they're not to popular above a couple watts. Even cars don't use them, at least not for decades now.

Generators have more parts and are pretty finicky and delicate compared to alternators which are mechanically pretty simple beasts in comparison.

Alternators are electrically controllable and have a field winding.

Something like a diesel locomotive has a couple stages with a small 32 volt generator providing enough field current to run a small alternator which provides the field for the big alternator which connects to the drive motors. In the old days they used DC drive motors but for decades now they basically have multiple giant rolling VFDs and ac motors. VFDs are more reliable than mechanical commutators, which sounded crazy in the old days but its true.

Also important for black start capability are back-up batteries for the the switching equipment. If those run out, then you won't be able to reconfigure your network topology while bringing it up. Doing that manually takes a lot longer and could add days to a black-out.

I think I remember reading that the phone system has nothing in place to do the equivalent - all the equipment relies on getting its code from the rest of the network, so if the entire phone network ever fell over there's no procedure to start it again. Anyone know one way or the other?

Noted for post-apocalypse.

This reminds me of how turbine aircraft usually start their engines. The battery on these aircraft are used to start the APU (Auxiliary Power Unit) which then starts the main engines. The APU is basically a generator, often in the tail of the aircraft.


Does this happen often? I suppose every time a plant gets shut down for maintenance, but I don't see mention of catastrophic failures necessitating many black starts.

Was there something in current events that prompted the rise of this post?

I didn't realize posting links to encyclopedia articles qualified as "hacker news".

While I think the discussion is an interesting one, it's still odd to see a random wikipedia article on the front page of HN

One of my projects at the moment is to make a hand cranked USB power source. I stumbled across this while Googling for "stepper motor power generator" or something like that. It looked interesting, so I submitted it.

This makes the discussion much more interesting to me.

Thanks for the context.

See my response to a similar statement in another thread: https://news.ycombinator.com/item?id=5630031

Hacker News isn't just about news, it's about learning and community. I, for one, would like to see more random Wikipedia articles on HN.

I like seeing this kind of stuff too, but it can be nice to see some context for how someone ran across it (which anthonyb provided in a sibling comment to yours).

"I didn't realize posting links to encyclopedia articles qualified as "hacker news"."

If you actually have to be told this, I don't know what to say, but the analogy is if your virtualization system can't boot without working DHCP and DNS, you've got a big problem if you virtualize your DHCP and DNS.

This is obvious. The more likely failure mode is some goofy homemade thing. Like you can't boot any server including the database server unless the LDAP backend can succesfully talk to the database of user info, but you can't talk to the database unless its already booted, so..

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact