Hacker News new | past | comments | ask | show | jobs | submit login
Nationwide FAA weather reporting outage (noaa.gov)
185 points by flerchin on Jan 19, 2023 | hide | past | favorite | 224 comments



METARs are perpetually down. one funny thing about them, they are deemed to be a critical service which requires something like 99.99% uptime to be in compliance with the FAA regulations governing service delivery. to get around this, a METAR is considered up as long as it has power, regardless as to whether it itsproducing data or providing accurate readings.

the technology was certified and frozen in the 90s. many of these devices are running windows 3.1 for networking!

they cost more than 2 million dollars a piece to install (at least around here) and require 4 trips per year to calibrate.

since there was little incentive and herculean constraints for the team in charge of METARs to improve the situation, another team at the FAA decided to create their own non certified version. to be installed in places that can't afford a METAR. They were able to build a better system using the same hardware but updated software for a hardware cost of $80k, 250k installed. this system only needs calibration once every 2 years. so it represents about a 1/10 cost of install and 1/8 cost of maintenance.

naturally this pissed off some people. I'm not going to get into internal politics beyond saying the project is currently frozen until a number of large efforts imposed on the program are completed. I believe it will still eventually come out because the cat is out of the bag and there are many groups desperate for a better alternative.


At this point I'm just waiting for the country to collapse under bureaucracy.


View it from the other side: even with this much dysfunction, things work most of the time and when they fail they are recoverable. There's no motivation to improve until something fails hard enough. If no one crashes as a result of this METAR outage, I predict nothing will change.


Define “work” if you don’t mind. There’s currently a power play going on about the national debt, wealth inequality is bad and getting worse, tax money gets wasted like it’s going out of style, social security isn’t and has never been solvent, the bigger government gets the more abstracted away from real people it becomes, and we all just stick our heads in the sand and ignore it because there is nothing the common person can do to effect change. Don’t tell me “go vote” because I do and it does fuck all nothing to change anything.

People adapt and say “meh it’s working well enough” because they don’t have a sane alternative.


It's not that USA is doing incredibly well with bureaucracy, it's that everybody else is doing worse for one reason or another


I don't see much evidence that the US is the best functioning country around. It has a great geographic position and managed to get a great economy (in no small part due to said geographic advantages in WWI and WWII), but in measures that involve the government providing good services the US isn't doing any better than other developed countries.


in fact we're doing far worse on many fronts. we spent more per student than almost any other first world nation but we place dead last in outcomes.


bureaucracy is a form of entropy. it will grow in any organization until it is outcompeted, a catastrophe occurs, or it collapses.


A METAR* is the weather message, not the observation system (AWOS, ASOS, etc).

* METeorological Aerodrome Report


yes you are correct.


I'm very interested in this after reading your account, but couldn't find additional information online. Do you have sources we could look at?



Thanks!

Cameras list/images: https://weathercams.faa.gov/


Thanks so much!


The flip side is that the military also relies on and distributes observations. Last I looked, the FMQ-19 used a windows server 2019 box. And it is used for some of the airports that are dual use, such as Hickam/Honolulu international.


how much does one of those cost?


This is really crazy to read about, and I am thankful you posted this.


> many of these devices are running windows 3.1 for networking!

Woah. Really?


Unlike the NOTAM outage, a METAR outage will and should actually affect flights. Without weather at your destination it becomes impossible to know if it’s safe to land there. The forecasts (TAF) are actually used more in flight planning, but actual weather is very valuable while enroute, when not close enough to hear weather over the radio from the destination airport.


To add some nuance to what other commenters have said, 95%+ of all NOTAMs are useless, telling you about grass that might be mowed, animals that might be present, temporary "obstacles" that you could only hit if you were flying both dangerously and illegally, correcting immaterial typos in the charts, or updates to dubiously helpful information published in the charts (number of hotels in the surrounding area).

Others are potentially useful, but not essential for safe flight. Things like closed taxiways, nonavailability of services at an airport, etc.

And a very few are really critically important. Runway closures, correction to vital chart information, airspace changes, malfunctioning navigation aids.

The FAA's reaction to the NOTAM outage was probably the correct course of action. But make no mistake, the volume of spam NOTAMs combined with the lack of an easy way for a pilot to quickly sort for important NOTAMs makes us all less safe.


> But make no mistake, the volume of spam NOTAMs combined with the lack of an easy way for a pilot to quickly sort for important NOTAMs makes us all less safe.

The system is in desperate need of some modernization, no argument there. The fact that there isn't a simple criticality filed with them that makes it easy to see what will actually impact flight planning (airspace closure / runway closure vs stupid chart updates) is insane.


The problem is that "criticality" is too binary. A tower light NOTAM might be low-criticality for a fixed-wing airplane pilot flying day VFR, but high-criticality for a helicopter pilot flying at night in IMC.

Similarly, chart updates are very important if you're flying IFR. Going below minimums while in the soup ends... very badly. If somebody's changed the MEA/DA/MDA, you can bet I want to know.

NOTAMs do have keywords to tell you the subject (i.e. airport closure vs tower light), and most briefings will highlight the ones likely to be urgent.

(That said, I'd argue printing out all the tower notams in textual format is somewhat useless. The NEXTGEN FSS briefings are plotting them graphically now, which is an improvement.)


The idea is probably that is you included a criticality flag, pilots would ignore low priority notices and potentially miss something (like grass being mowed at the time they plan on landing) that could affect them, but is otherwise immaterial for most others.

I can’t say one way or the other, as I’m not a pilot, but that’s the argument I’d make to keep the system w/o priority levels.


As a pilot of single-engine airplanes, I disagree that mowing is useful. I need to look at the runway I'm landing on for obstructions no matter what. The time of year when mowing might occur, I might also have to contend with deer, who are just as hazardous and don't file NOTAMs before grazing near the runway.


Unrelated to aviation, but I actually find METARs and TAFs are useful for hiking and skiing. I use the altimeter settings for my altimeter when hiking, and knowing the altitude of cloud layers is useful to know what the visibility will be like on the mountain (you have to do a bit of math with the airport elevation and the mountain’s elevation, but hey—it works!).


I used METAR for running, or rather to say which days I'm allowed to not run.

I noticed that when the day was long, and I was feeling tired and lazy it was a lot easier to find some excuse why not to run that day. This excuse was often the weather. But on the other hand I didn't want to say no matter the weather I must run, because that is obviously excessive. So I made up a simple "algorithm" to decide if the weather fits the minimums, and if it did I must go and run. And I choose to base it on the measurements from the METAR of the local airport.


Cool idea, what's your algorithm's decision criteria?


I think it was something super simple. Like temperature above X, no percipitation at the moment, no percipitation predicted during the next hour.

(I also remember I was willing to skip running in case of anything “exciting” being reported. Like tornados, or sandstorms or vulcanic ash. Not that any of that was a real possibility.)


In the era before cell-phone internet, being able to call the automated weather reporting station at the nearest airport and have it read out conditions was hugely useful.

For folks who haven't seen what's in a METAR/TAF: https://www.faa.gov/air_traffic/weather/asos/ (By default, clicking a station gives you the abbreviated format; you can select the "decode" radio button and click "update" to translate it into English.)


I like your idea of visibility a lot, and would like to know more. Are you using a barometric altimeter? I use one hiking, but mainly for dead reckoning using an USGS elevation map. I usually just set my altimeter using those maps.


NOTAMs cover things like runway closures or airspace closures due to hazard, as well as a huge host of other issues.

Stopping takeoffs was absolutely the right move, since they couldn't handle the phone throughput.


Err. NOTAM should also have effected flights. Please don't downplay the safety of airspace.


A pilot can legally land without a METAR, and this happens all the time. The stations can and do go offline. Sometimes they break, especially after a severe storm. Many smaller airfields don't have on-site weather reporting, some others don't have a network connection.

You look at the closest available weather station and you look at the forecasts — human-authored (TAFs) and computer-autored (GFA/MOS). And when you arrive, you look outside at the actual conditions — the windsock, actual flight visibility, etc. If the conditions are worse than expected, you divert to your alternate.


Yep. Not knowing the atmospheric pressure can lead to planes crashing, especially if it's inclement weather.


Wouldn't each airport measure and report the local weather anyways?


Each airport does measure and report the local weather-- that's where METAR data comes from. And busier airports broadcast that data locally, via ATIS or D-ATIS, for pilots in the immediate vicinity to use (either on the ground or on approach to the airport).

That doesn't help a pilot plan ahead for conditions on their route or at their destination, though-- that's where METARs come in.


Are local weather conditions not generally available for all airports at all times?


They are. I've even seen small aircraft pilots call the automated weather line from the cockpit while they're preparing to approach the destination airport.


That automated weather frequency relies on METAR data from the Aviation Weather Service here in the US, and that's where ATC gets their information from as well when they're advising pilots of surface winds when giving landing clearance.

Per the source, 167 airport weather stations are not reporting correctly (either due to a highly unlikely concurrent fault with the stations, or more likely an issue with the system they report to).

Local weather is always used, but that's precisely what is unavailable right now.


You have it backwards, ATIS doesn't pull from METARs... the METAR and ATIS recording are both produced at the same time locally by the local weather observer (usually a controller in the tower), reading raw data directly from the on-field weather station and making manual edits as necessary. The two should match up exactly, but that's because they're both produced by the same process.

If the tower is closed, or it's an untowered field, the system can run in automatic mode. In that case the radio broadcast gets updated every minute, much more frequently than METARs get updated (every hour).


And, if the tower is open, you have to acknowledge the latest ATIS report (by signifying the revision) before you can go.


Yes. And that service is called METAR (METeorological Aerodrome Reports).


>Unlike the NOTAM outage, a METAR outage will and should actually affect flights.

We have mines being blasted to our north in temporarily restricted military airspace - lets just allow pilots to roll the dice?


Has anything happened regarding IT staffing at the FAA recently? There was the NOTAM outage last week [1], and now this!

[1] https://news.ycombinator.com/item?id=34337158


The NOTAM system was apparently running on old SPARC hardware so I’d easily believe that they haven’t been given enough budget to hire engineers and are basically keeping the lights on. One peril of the government funding model is that it’s effective at shoveling money to contracting companies but not having skilled oversight, so you tend to see lots of big projects which founder under their own weight.


I would love to see more information about the underlying architecture of the NOTAM system, if you can recall where you saw that.


Not sure how reliable this guy is but it’s certainly easy to believe:

https://twitter.com/PatrickMoorhead/status/16139375120027729...


SPARC gear would last that long though, I recall playing with a T1 system and opening the chassis to watch the Christmas tree of self check lights on every part of the board turn on. The entire system was designed to self check every component down to like, individual VRMs.


Yeah, that's both good and bad — it's great that it's reliable but that's also the kind of thing I could see people getting complacent about because, hey, it hasn't failed before.


NOTAM could be run from a pre-historic Toshiba mainframe. it's static. Doesn't need billions in funding.


It's static, until a contractor ignores the post-it note on their monitor that says, "IMPORTANT: ALWAYS add an extra newline BEFORE saving the file".

Once that happens, you have to hope that your backups work - why bother having disaster recovery drills for a static system?

Once you realize that your backups are hosed, you may experience a creeping suspicion that software is never truly static when human operators are involved.


It doesn’t need billions but it needs enough for dedicated staffing. Many federal agencies have the problem that they’re not given general funding for staff but are given money for large contract projects. It’s not uncommon for that to be structured in a way that, say, a big project has rooms full of developers, PMs, testers, etc. but something small has like one dude keeping it going when they can find time and a stream of unanswered requests to management asking go either fund or decommission it, hire a replacement for people who retired, etc. or that the big project which was supposed to replace it is years overdue.

I would not be surprised to learn that this was the case here, too.


I don't know anything about these two incidents. But, FWIW, some of the past professional work I'm most proud of was on FAA contracts (on other systems). I'm sure they're handling the immediate situation, and -- consistent with their culture -- they'll analyze what went wrong and can be learned, and follow through on that.


Maybe there's an ongoing cyber attack ? :-)


Nope, just the regular old spending money on new projects that benefit the careers of powerful politicians but refusing to spend on maintenance attack.


That isn’t being admitted to?


Admitting attacks can lead to more attacks if you’re not prepared


Happens more frequently than you think.


Probably just more visibility on the matter. Once a major issue happens, things tend to get followed for a while. It can get bandwagoney


For what it's worth, the METAR network (NADIN) is maintained by the NWS, not the FAA.


(Following up on this, I was actually mistaken. I believe NADIN is actually an FAA network.)


You mean besides the last three years of reduced on-site staffing, and employee travel to support maintenance and modernization? No.

$5 says this is the XKCD for this. Corrupted DB file was the culprit, as announced by the FAA. It's a system that relies on user inputs, often manual, and it's all being put into a database. My bet is a user decided to put in a bunch of "fun characters" to make their input easier to read. You can't account for and sanitize all levels of stupidity on user inputs.

https://xkcd.com/327/


> You can't account for and sanitize all levels of stupidity on user inputs.

That is false and a defeatist attitude.

Sanitization is not the right solution anyway. If you are working with any form of a database and don't know from the top of your head how to avoid query injection attacks then you should look up in the manual of your database. The solution is most often called "parameterized query" or something similar.


Of course you can account for that. They're just byte strings. That xkcd is poking fun at incompetent programmers, not stupid users.


The outage seems to be in receiving reports from weather stations (primarily airports).

When arriving at an airport, a pilot is required to confirm they have the latest METAR information for the site. It's presumed negligence when pilots don't know the weather. This is because weather is the #1 environmental factor in accidents.

But the actual use of weather in flying (esp. for commercial flights) is not via METAR but via in-cockpit radar. This is an FAA-sponsored service, but it's unclear if that's affected.

So it seems to be more administrative than safety, but just as disabling.


You say radar, but I think what you might mean is ads-b[0].

This is the system where weather info (among other information) is transmitted from a network of ground stations.

[0]: https://www.faa.gov/air_traffic/technology/adsb/pilot


For what it's worth, there are also actually on-board radar systems in some aircraft


Yes there are but they don’t give you enough information to decide whether you can land at an airport.

ADS-B gives you METAR, for instance.


Does in cockpit radar actually get you on the ground pressure?


No, those come from ground reporting stations which feed into a whole bunch of sources - online stuff will give it to you including Foreflight, or just key up the ATIS and listen for a few seconds. Or ask a controller.


No, of course not. But pilots aren't going to aviationweather.gov while they're flying, either. They get that info from the controller or ATIS.


Maybe I misunderstand what was actually down then. It was just the website, not the actual weather reports that ATC has access to?


No doubt. Inflight Wi-Fi is absurdly expensive. No way they’re paying for it.


I think its $8 a flight on Alaska. Doesn't seem that outlandish.


SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE NWS NCEP CENTRAL OPERATIONS COLLEGE PARK MD 1120Z THU JAN 19 2023

...METAR OUTAGE...

AOMC reported that there is a metar outage going on...167 missing sites at last check. The FAA is investigating the issue.


> 167 missing sites at last check.

That seems like a lot but curious to know what percentage of their total sites this is.

I think I found the full list here: https://www.aviationweather.gov/docs/metar/stations.txt

This list is worldwide and I see no information about this outage being region specific so I'll assume it's worldwide.

Seems like they have about 9,517 sites so this is about 1.75% of their sites worldwide


You should not assume it's world wide. Given "the FAA is investigating" and there's no reports of other countries it's reasonable to assume that this is limited to the US, or at least northern continental America.

(The title of the submission is also "Nationwide FAA weather reporting outage")


> The title of the submission is also "Nationwide FAA weather reporting outage"

Yeah, but the title is made up. There is noting in the official report about this being "nationwide".

The report only states that there is a metar outage consisting of 167 stations.

It doesn't provide any context on which stations are missing or how they are distributed.

Adding "nationwide" to the title is pure FUD IMO.


if the outages are on the east coast and a flight from the west coast is preparing to fly to one of the east coast airports that are affected, how is this not nationwide?


Full list of reporting stations is here: https://www.aviationweather.gov/docs/metar/stations.txt

The report is that 167 are missing.

That would be roughly 5% of US stations or 0.2% of worldwide stations.

I was able to look up meters for airports that I'm familiar with. https://www.aviationweather.gov/metar

So, it is real but far from a total collapse of the system.


And then thousands of pilots checked Weather Underground's sister site, Weather Overground...


Never understood why WU went after miners first.


Much easier to predict the weather underground. Dark, damp.

Once you build up credibility for being right, you can expand into other more difficult areas. If you keep predicting for the underground, then it will also help keep you correct average up.


Untapped market to network the canaries.


You Don’t Need A Weatherman To Know Which Way The Wind Blows


Better stay away from those that carry 'round a fire hose


For those following along, these are lyrics from the Bob Dylan song that is the source of the name for both the Weather Underground website as well as the militant group, https://en.wikipedia.org/wiki/Weather_Underground


Don’t give Helium any more ideas.



a VC told them to use a bottom-up TAM, so they started at the bottom and now they're here


Condensation


One thing though, these types of things can't be helpful to the political future of Pete Buttigieg. Between airline issues and rail strikes and shipping/port issues, he has been very politically unlucky.


Not necessarily, if he gets a lot of airtime and is seen as identifying and fixing problems he could get a real big boost. Most of the problems the Transportation Secretary has to deal with are the fault of aging systems, a limited budget from congress, etc.


> if he gets a lot of airtime and is seen as identifying and fixing problems he could get a real big boost.

The thing is he is not. Even with the negotiations over the railroad strike, he was not talked about. It was Joe Biden in the news.


That's fairly normal. Secretaries who become The Story tend to find that's not a good spot, even if it's for positive reasons.

Buttigieg's biggest asset is likely his ability to get on Fox News for an interview and steamroller over their usual rhetorical tactics.


Just gotta ask: Southwest, NOTAM, and now this…

Is some group pen-testing?


Obviously not. Southwest issue isn't even close to having to do with FAA systems. Not everything is a cyber attack... most things are just accidental systems issues.


Southwest's meltdown was definitely not an attack. Their crew scheduling software was so outdated that it required people to call in and speak to a human to report when they were out of position due to a cancelled flight. The sheer volume of cancellations resulted in this very manual system simply falling over, and so when the weather cleared up they actually had no idea where most of their crews were.


Sure, that's the publicly-reported story. Is it sufficient? Hard to tell from the outside.

Just a tiny bit of blackmailer-sabotage or vandal mischief-making – the sort of thing often underreported for legitimate reasons! – could've been the margin between some mild embarassment & the total collapse we saw. (For example: tying up the call-in lines to turn a normal hold time of minutes into hours, or long busy signals.)

Simple monocausal stories are for unsophisticated spectators.


What sort of reason do you have to believe that sabotage is involved, when it is adequately explained by the positive feedback loop of out-of-position calls exceeding the rate at which the can be answered?


Why do you think I believe it was definitiely involved?

I just don't think it's ruled-out by the simple official story. Some but not all of the reasons why I believe there might be more to the story are based on decades of life experience:

* Whenever the media reports on something I know about intimately, they get it wrong, preferring pleasing simple narratives to the full details. Hence, I assume their stories on things I don't know intimately show a similar skew towards oversimplification, missing-details that are embarassing to the key actors, and audience-pleasing explanations.

* I've not ever been in IT security for a major or public company. But even in tiny organizations, I've observed strong incentives – not all misguided! – to downplay malicious mischief as a contributor to any problems. Organizations don't want to encourage the perpetrators with publicity, nor encourage copycats. Orgs also don't want to be embarassed by lax measures. From direct reports from individuals at larger organizations – and reliable public accounts of late-reported hack/extortion incidents – I believe these incentives can be even stronger at large, slow-moving, distributed-responsibility public companies (though of course the penalties for explicitly-misleading statements also larger).

* Plenty of mean, crazy, or self-interested people may have it out for Southwest, from previously-angered travellers to disgruntled employees to motivated short-sellers (individuals or formal funds). And even if the potential for sabotage was under formal investigation right now, the investigators – private or public – might want to hide that fact until definitive evidence collected & perpetrators are prosecutable. It can take months or years for the real story to emerge!

A fragile outdated system finally reaching a chaotic breaking point is one possible & sufficient answer, of course.

But it's also a potential weak-point to be pushed-over-the-edge by motivated saboteurs or extortionists. In fact, such a weak point is ideal for certain criminal schemes, because of its deniability by both perpetrator and victim as merely a problem of aged systems & incompetence.

So, only the naive would rule it out entirely based on only self-serving public narratives.


Nothing we know about anything rules out the possibility that we might later learn something new that shows differently. But we can only make informed decisions based on what we do know, because the unknown is infinite. In the end, most occurrences in the world are just as plain as they appear.

> So, only the naive would rule it out entirely based on only self-serving public narratives.

I don't technically disagree, but it is also just as naive to ignore our human proportionality bias and discount the relative probability of the available evidence.


But the comment that I was responding to claimed, "Southwest's meltdown was definitely not an attack." [emphasis added]

I'm not saying it definitely was an attack, nor even that it likely was. Just that it's premature to "definitely" rule it out so soon, a mere 3 weeks out, given the organizational incentives involved & base rates of both extortion/vandalism & (often well-meaning but at the very-least ass-covering) reporting-misdirection about the same.

You seem to agree with me that it remains a possibility, so not sure we actually disagree about any particulars of the event, just the discussion.

Imagine there were a well-refereed, bettable proposition like, say, "By the end of 2030, will either (a) someone be criminally charged for contributing to the Southwest service disruptions of late 2022; or (b) will a knowledgeable Southwest insider or law enforcement agent report they saw evidence that intentional acts worsened the Southwest service disruptions of late 2022?"

Just from base rates of such mischief, & without yet digging deeper, I'd consider an answer of "YES" to have around a 2-3% chance. And something with a 1-in-50 chance of having happened is absolutely a valid topic of speculation deep in forums like this!

Such tail events are where lots of the big wins, & big losses, for industry & society arrive. But also, such real-but-rarer outcomes get habitually ignored by simple mainstream summary coverage, which needs to put neat bows on stories for uninformed & distracted audiences, by short deadlines, reliant on spin from involved entities.

To protest deep in the threads, and insist that a few-weeks old official-sources story is "definitely" the whole explanation, case-closed, stop-speculating-its-hurting?

That's actually hostile to helping curious people understand a complex world based on limited & conflicted information sources.


All you're doing is letting your imagination run wild because you don't want to believe the simplest (and most likely) explanation. This isn't helpful to anyone.


It's a helpful exercise for naive young people who always believe the simple publicly-reported stories, and haven't yet had the life experience to know there's often more to the story, which only comes out years later (if ever), or via private conversations to deeply-knowledgeable personnel!


No, it's really not a helpful exercise. An actual helpful exercise is to determine when an official story doesn't make sense, and to understand that there is probably more to it. In this case, the official story make perfect sense. Doubting it is just creating fairy tales.


> perfect sense

So you just inherently believe all that is reported as fact?

Remember when the Gulf of Tonkin incident was “for real really real” and we invaded Vietnam?

Or the WMDs in Iraq?

I’m not tinfoil hatting - Just calling out how interesting it is 3 major events in a documented as fragile industry happen so close.

That’s literally all I mentioned in my orig comment.


> Remember when the Gulf of Tonkin incident was “for real really real” and we invaded Vietnam?

Well, no, the US military was (openly) in Vietnam in support of South Vietnam long before the Gulf of Tonkin incident. Heck, the US military was there (again overtly) in support of France for several years before South Vietnam existed as such. And, ironically, they were there in support of the Viet Minh – which later became both North Vietnam and the Viet Cong – even before that, with almost no break between, starting in July 1945.

Insofar as there was an international incident that led to the US invasion of Vietnam it was – though it took a few years for the response – Pearl Harbor, not the Gulf of Tonkin. All the subsequent fighting in Indochina was breakdown in relation between erstwhile allies who were all already present, after the Japanese were driven out and had surrendered.


My point was that the Tonkin incident was reported incorrectly and did not match fact; as we found out some decades later.

Tying that to this thread: why is what’s reported about the airlines taken to be fact? (And I’m not speaking as if I mean this is malicious (as the Tonkin incident was) - just reported incorrect from fact for other “capital friendly” (face saving) reasons.)

Didn’t someone say “believe half of what you read…” somewhere?


It's weird that americans always seem to have this sort of siege mentality. We already know the causes of the other outages, and they have nothing to do with the foreign interference boogeyman.


Why do you assume I am American? Or are you speaking generally?

“We” don’t know the causes, only what was reported.

I’ve worked on a lot of old systems needing to be updated, migrated, or sunset. For sure it’s 99% of the time due to management being lazy. I get that.

And I know correlation/causation, but hey my orig comment was a simple curiosity.

Sure, my nature distrusts what a corporation says because it’s always filtered through stockholder functions, but I still think “3 major airline issues in as many months” is kind of hilarious. Even if they seem to be in different facets.

Does this happen often but only now is it reported to “us”? Or is this truly an odd and rare timing?


>Sure, my nature distrusts what a corporation says because it’s always filtered through stockholder functions

OK, but in Southwest's case, you had former and current employees from various positions within the company publicly coming out and lambasting Southwest for the failure to actually invest in upgrading their scheduling system that led to the outage. It wasn't just a corporation coming out and going, "Here's what we hope you believe happened," it was pissed off employees publicly fucked off that their years of complaints went unheard and now they were all paying the price for being ignored.

>Or is this truly an odd and rare timing?

It's just odd/rare. I would wager that your brain's rush to try and tie them together like this is just it's way of trying to make sense of something it normally doesn't see.


> invest in upgrading

Meaning - a system is so old, slow, bug, and vulnerability prone a bad actor can pen-test around and mess stuff up?


"Can"? Sure, just like most systems. "Did"? No, that's ignoring the fundamentals of how the system crashed. I'd recommend doing a deeper dive into what actually happened.


Actually you are right, sorry. Your comment didn't indicate that. My reply was mostly tainted by the other replies your comment got, and I assumed things I shouldn't have.


Are you so sure you have 0 people of this type in [country you are from]?

Are you sure you are not seeing selection bias because the majority of users on this site are Americans?


I think that it's still pretty unique to the US. I completely agree with your point in general, and I think that internet discussions are often centered around America, so there's a huge selection biais in online discourse. But I truly think that the "the empire is under siege"/national security discourse is much more prevalent in the US. Which makes sense considering it is the empire ;). We don't really have that where I'm from. Instead of a siege mentality, we have a prevalent inferiority complex hahaha.


It's weird how Europeans have this fedual surf mentality.


I'm not european, and if anything, I'm looking into moving to the US soon. I'm not trying to do the usual "americans, amirite?" thing. It's just that there is much more of a "national security angle" to every discussion like this in the US. Which is pretty unique to america I thini


Go to Israel and say that. It is a feature but it is not unique. The US is a target for a variety of reasons. While a more mundane cause is likely, the implications of an attack are significant thus it would be unwise scoff this concern off as mere paranoia.


If you're going to attack them, you could at least spell some of those words correctly.


Yeah, total nonsequiter. Everybody knows Fritz and Jacque don't surf.


Part of it is this intellectual one-upsmanship, especially at places like HN. People think they look smarter if they reflexively reject "the narrative".


Once is happenstance, twice coincidence, three times is enemy action.

It could be Russia, Iran, China, climate direct action groups (probably not y'all qaeda though). Or maybe it's just ancient software running on older hardware that's starting to fall over after a couple years of lighter loads and a sudden increase in complexity and weirdness. Everything is breaking at the moment.


There's also the world's biggest viral pandemic that's still ongoing and currently in a post-holiday peak where many people are out sick...


"sudden increase in complexity and weirdness"


By "y'all qaeda" you mean the boojahideen, not actual Al-Qaeda?


Ah yes, the Yee-Hawdists. Vanilla ISIS. Banana Republicans. Meal Team 6. Walmartyrs.

Classic buffoonery. Until it's not.


Yes exactly, the gravy seals.


Meal Team Six?


Could just as easily be a bias in reporting given the interest in other flight related issues lately. Previously unremarkable outages like this can suddenly become newsworthy if other current events seem to give them context. Once an issue is proven to draw clicks headlines will go back to that well any chance they get.


I would think, of all forums on the internet, that this forum would realize how many heaping piles of fragile software the world is built on.


We’re being pen tested by every other country in the world constantly.


Uh oh, I've got a flight in just under 4 hours...

There doesn't seem to be any flight delays though or notices on the airport website...


hm is this going to affect flights?


Someone sell them a modern status page!


[flagged]


That’s possible. But also it’s possible that decades of underinvestment are catching up.


IDK about the Feds but every contact I have in state government and local government & public functions indicates that both have been shoved from "slowly deteriorating" to "brink of collapse of ability to provide basic service" by the shock of the Covid years and recent wage inflation (which neither's even come close to keeping up with). All were threadbare as hell already, so it was easy for Covid to tear some outright holes.


How do we go about getting entropy placed in the terrorist watch list? Maybe then we could fix some bridges and roads too.


[flagged]


I'm sorry, but "looking out the window" has been obsolete for thousands of years. What aviators really need to use is the Weather Rock [1] and its associated documentation.

   If the rock is wet, it's raining.
   If the rock is swinging, the wind is blowing.
   If the rock casts a shadow, the sun is shining.
   If the rock does not cast a shadow and is not wet, the sky is cloudy.
   If the rock is difficult to see, it is foggy.
   If the rock is white, it is snowing.  
   If the rock is coated with ice, there is a frost.  
   If the ice is thick, it's a heavy frost.
   If the rock is bouncing, there is an earthquake.
   If the rock is under water, there is a flood.
   If the rock is warm, it is sunny.
   If the rock is missing, there was a tornado.
   If the rock is wet and swinging violently, there is a  hurricane.
   If the rock can be felt but not seen, it is night time.
   If the rock has white splats on it, watch out for birds.
   If there are two rocks, stop drinking, you are drunk.
[1] https://en.wikipedia.org/wiki/Weather_rock


I've never heard of this, and its brilliantly hilarious! TIL about the weather rock! :-)


Someone should have prompted you to learn about it sooner by giving you a Round Tuit.


Lol nice! :-)


I suspect that's not good enough when planning a flight.


duh look through a skylight


only real pilots fly without computers


Doesn't work when you're flying through a cloud.


Well, it works ("I'm in a cloud!") but isn't sufficiently detailed for the needs of the situation.


Or at night.

Or to assess wind speed.


Or when you want to know the weather at your destination half a continent away.


Things I use as a pilot of a single engine piston:

- Temp dew point spread - Barometric pressure - TAF / forecast - Wind shear - Freezing level - Cloud tops - Cloud level AGL - visibility measured in actual distance

All of these things can’t be measured easily by a human. Visibility is probably the easiest, but can you tell me what the dew point is easily?

If I use it, imagine the critical importance for airliners.


Armchair sim-only pilot/aviation enthusiast, simply not having accurate wind speeds is enough to prevent a jet from landing because you don't have data to run performance calculations.

Can you land without running performance? Sure. But now a standard landing is an emergency since you can't verify you have enough runway to come to a complete stop. (See SWA 1248 for what happens when you don't run landing performance calculations correctly in iffy conditions).

This is, of course, ignoring all the other issues that simply not having accurate barometric pressure and cloud levels when flying IFR. If you're flying an approach that descends over mountains through a cloud layer and lack accurate weather data (specifically barometric pressure) then the chances of your EGPWS shouting "TERRAIN! TERRAIN! PULL UP!" because you're low due to an incorrect altimeter reading are non-zero. Do this in a mountain range and the EGPWS calls may end up being too little, too late even if you immediately perform terrain escape maneuvers given you still need accurate altimeter readings for EGPWS to function correctly.

Accurate METAR reports are safety critical for aviation. Full stop.


Jets plug these numbers into the flight computer to determine landing speed (Vref), needed runway length, and sometimes configuration (flaps etc.). So without weather I'd expect commercial flights to simply stay on the ground.


Isn't there a way to rough out the dew point from the cloud base altitude? I seem to remember that from ground school.


Yes, but generally this would be used the opposite direction. If cloud bases are unavailable, or if weather is changing rapidly, you could use say barometric pressure + dew point to determine "is this about to go south towards low clouds and low vis?"

(Also, cloud bases are part of the METAR info in the first place, so it's a "neither" situation)


I remember being the other way around, you could estimate height of the cloud based on dew point and base altitude.


Yup! 400' AGL for every 1 deg C of temperature spread.


Yah! So you can find the dew point by flying up to just below the cloud and look at the altimeter and do some math ;)


Wind shear is easy. feel the wind on your face, then jump as high as possible and feel the wind up there.


this is an advanced technique that they don't usually teach until the commercial certificate.


Oh, so that’s what turbulence is.


Wind shear is one of many various causes of turbulence in aviation. Certainly a common one, however.


I got curious and Google'd some of this:

> The dew point in relation to the temperature gives the pilots information about the humidity, and can affect visibility. If the dew point is close to the temperature, humidity is high, which can cause hazy conditions, or even fog.


It's also an easy way to get a rough estimate of the bottom of the clouds: 400' above the ground for every 1degC of difference between the temperature and the dew point.

This is useful for a couple of things, including whether or not you can legally take off or land when flying under visual flight rules.


I’m very much willing to believe you when you say you can do it practically, but there’s something logically or epistemically funny about measuring visibility using vision.


Haha - yes, this is really just a language problem.

Visibility for pilots is a precise measurement, not a classification. "Can I see" is a different question than "how many miles can I see", and even still "how many feet can I see down this runway."

These measurements dictate different flight rules, for example which approach I can take into a given airport.

Take this plate for example: https://flightaware.com/resources/airport/CHA/IAP/VOR+RWY+33... - There are two approaches you can take, categorized in the bottom, that use this plate. One is circling, the other is straight in. Your aircraft category (approach speed ranges) dictates the visibility requirements; in this case, a small Cessna needs a statute mile of visibility to execute either of these approaches.

However, a larger airplane - an airliner, for example, needs different visibility for the circling versus the straight in. How different? In this case, half a mile for category C (1 3/4 vs 2 1/4).

What if visibility is less than a mile?

Then you need to do something like this approach instead: https://flightaware.com/resources/airport/CHA/IAP/ILS+OR+LOC...



We get something like ten of these a year.


According to Tamitha Skov, right now is a particular intense solar storm: https://www.patreon.com/posts/77348743


Serious question: At what point does the failure of government services warrant the partial withholding of taxes?

This isn't sufficient of course but when the FAA repeatedly demonstrates incompetence, when do we eventually just tell them no, you can't have any more of my money, you're obviously wasting it and making us worse off?


This is the conservative playbook. Cut funding until the entity can't run properly, then yell JEEZ GUYS YOU REALLY SUCK wHY DON'T WE SHUT YOU DOWN

The previous president significantly cut the FAA's budget, maybe we're just seeing now how that affects us in the long term


I'm not a conversative.

Presidents don't pass budgets.

FAA funding has increased, not decreased.

I can criticize my government all I want. You're turning this into a political party thing when it's a government service failure.


https://www.washingtonpost.com/local/trafficandcommuting/whi...

Relevant quote:

"President Trump’s support for a plan to lop more than 30,000 Federal Aviation Administration workers from the federal payroll gives fresh momentum to an effort that stalled in Congress last year.

The proposal is included in Trump’s 2018 budget, which would cut funding for the Transportation Department by 13 percent."


To be fair, I think that proposal died when it got to Congress.


That didn't happen. It was proposed and rejected by Congress.

The FAA's budget was not "significantly decreased" as the comment above claimed.


Okay, that's a proposal. Now show where it passed and the results.


> Presidents don’t pass budgets.

No, they propose, sign, use the veto threat to shape the legislative process on, and use various administrative powers and means to rearrange the actual spending resulting from them, but “passing” budgets is the name for a step in the middle of the process done by someone else.

Not sure what your point is; the President is by a very wide margin the single most powerful actor in the shaping of both nominal budgets and actual federal spending.


It's the folks that hitch themselves to a political party narrative that are the most dangerous, because they can't decouple when the party fails them repeatedly. They start to make excuses, moving the goal posts, and white washing obvious corruption. They become so enamored in it, that they find people on the internet to harass, just so they can feel big and important for putting them in there place.


Presidents propose budgets (the CBO even uses the term "the President's budget"; https://www.cbo.gov/publication/58417), sign them into law, threaten Congress with vetos, and have significant influence over their party as a whole. Trump in particular was successful in drubbing opponents out of office; Liz Cheney went from #3 in House leadership to a pariah largely overnight.


>> Liz Cheney went from #3 in House leadership to a pariah largely overnight.

Seems to run in the family. The voters in Wyoming appear to agree.


This is all a lot of weasel words to avoid the fact that the President does not pass the budget, and their proposals bear little resemblance to what actually get passed.

Liz Cheney is completely irrelevant here.

Post FAA's ACTUAL BUDGET over the last 8 years and let us judge.


I'm not the one who made the budget assertion; you're mad at someone else.

I'm just contesting the idea that the President has no power over the US Federal budget.

(I'm of the general opinion that the FAA is fairly well funded, has been very effective at its mission, and could still do with some specific capital projects to modernize non-sexy stuff like NOTAM and METAR handling.)


There is no US federal budget - not a binding one at any rate. The president recommends and Congress passes budgets for internal purposes, but they do not carry the force of law. The numbers that matter are how much Congress appropriates, and if Congress appropriates $X dollars for something the executive branch is expected to spend it, not second guess Congress and say we really didn't want to spend that money.


> There is no US federal budget - not a binding one at any rate.

Yes there is.

> The president recommends and Congress passes budgets for internal purposes

This is simply false.

> but they do not carry the force of law.

Yes, budget bills, like other bills, have the force of law once passed and either signed by the President or vetoed and the President’s veto overridden by both Houses (the President’s budget proposals do not have the force of law, just like other unpassed legislative proposals.)

> The numbers that matter are how much Congress appropriates

“The budget” is just the aggregate of tax and appropriations bills (sometimes, there is an annual package entitled a “comprehensive” or “omnibus” budget bill that covers some tax policy and most or all of the annual appropriations, but that’s rare, and even in that case there are typically continuous and multiyear appropriations and tax policies that are left untouched and outside of it, but which are considered part of the “budget” even though they are aren’t part of the “budget bill”; more commonly, this doesn’t occur.)

> if Congress appropriates $X dollars for something the executive branch is expected to spend it, not second guess Congress and say we really didn’t want to spend that money.

While this is true, and more than just a soft expectation, since Train v. City of New York and the Impoundment Control Act of 1974, it is not invariably the practice notwithstanding the expectation and the law (cf., the withholding of funds appropriated for Ukraine aid that was central to Trump’s first impeachment.)


> budget bills, like other bills, have the force of law once passed

First of all congress generally does not pass budget bills, they pass budget resolutions, and congressional resolutions are not submitted to the president for his signature and do not have the force of law.

As far as budget bills (like the Budget Act of 1974) are concerned, it is impossible for a previous congress to pass laws that govern the actions of succeeding congresses in a binding fashion because such laws have the character of congressional rules not statutes, and under the Constitution each house has complete authority over its own rules. So Congress can simply neglect to pass a budget resolution and the only remedy is for individual representatives to raise points of order. The executive branch cannot indict or prosecute anyone in Congress for neglecting to follow congressional rules or laws that have the character of congressional rules.

On the other hand, if you wish to refer to an appropriations bill as a "budget bill" you are merely disagreeing with me (and with congressional practice) over nomenclature. Personally, I have never heard of an appropriations bill referred to as a "budget bill", but I am sure someone does it.


> First of all congress generally does not pass budget bills, they pass budget resolutions

The budget resolution is a planning framework for the appropriations bills that make up the budget. It is not a “budget”.

> and congressional resolutions are not submitted to the president for his signature, and congressional resolutions are not submitted to the president for his signature and do not have the force of law.

That's not true. All Congressional actions are resolutions (e.g. the USA PATRIOT Act was "House Resolution 3162" of the 107th Congress), and many end submitted to the President and with the force of law. Some are even primarily characterized as "resolutions" after they are signed and have the force of law (See, in terms of the budget space, the use of "continuing resolutions".)

There are types of resolutions that are nonbinding, and do not have the force of law, and the "budget resolution" (which, again, is not the same as the budget) is one of them, but this isn't generally true of "congressional resolutions".

> As far as budget bills (like the Budget Act of 1974) are concerned, it is impossible for a previous congress to pass laws that govern the actions of succeeding congresses in a binding fashion

The 1974 Budget Act is not (relevant to this discussion) a budget bill, it is a meta-budget bill, or maybe even a meta-meta-budget bill (that is, a bill on the process by which Congress will plan future budgets.) You are correct that it is not binding on future Congresses, but this has no bearing on the budget not being binding, merely a past Congress's plan on how to plan annually to arrive at a budget is not binding.

> So Congress can simply neglect to pass a budget resolution

…which isn’t a budget, but a plan for one…

> and the only remedy is for individual representatives to raise points of order.

Well, sure, but the budget resolution isn’t the budget. If Congress doesn’t pass a budget (or only passes part of one), then the remedy is “spending governed by the parts not passed stops” (that is, a government shutdown.) This is…rather noticeable.

> On the other hand, if you wish to refer to an appropriations bill as a “budget bill” you are merely disagreeing with me (and with congressional practice) over nomenclature. Personally, I have never heard of an appropriations bill referred to as a “budget bill”, but I am sure someone does it.

While it is common to refer to the aggregate of appropriations as a “budget”, its not usually used for individual appropriations bills as a “budget bill” unless they are a single consolidated bill, which is typically referred to as a “budget bill”. Budget bills are frequently introduced, but less frequently passed. But not having a single budget bill passed is not “not having a budget” or “not having a budget with the force of law”, its “not having a budget in which the annual appropriations were adopted in a single bill”. Which is…less significant.


> The budget resolution is a planning framework for the appropriations bills that make up the budget. It is not a “budget”.

Guess what the President's budget proposal says on the cover?

"Budget of the U.S. Government, Fiscal Year 2023"

And if we look inside do we find a list of congressional appropriations? No. Instead we find a list of appropriations requests and explanatory information for congressional appropriators. Typical chapter headings have language like this:

"The Budget requests $773 billion in discretionary funding for DOD, a $69 billion or 9.8-percent increase from the 2021 enacted level."

So I am sorry, I cannot take seriously the idea that appropriations bills are "budget bills" or specify a federal "budget" except in an informal sense, one contrary to that used by the federal government itself.

> All Congressional actions are resolutions.

I certainly agree in the technical sense, and since we are debating nomenclature here I should be more careful.


A budget from the executive branch is just a recommendation. Congress might use use it as a starting point, if they bother to pass a budget at all, which they are not in the habit of doing lately unfortunately.

As far as Cheney is concerned, Trump had been out of office for months by the time she lost her leadership position. Anything he did was from the sidelines by that point.


> Congress might use use it as a starting point, if they bother to pass a budget at all, which they are not in the habit of doing lately unfortunately.

A budget is a budget even if it is neither comprehensive (i.e., all functions in one bill) nor for a full fiscal year at a time nor uses the word “budget” in its title.


In congressional nomenclature however a budget is a guideline for appropriations bills, not something that carries the force of law. And Congress is quite specific about what it appropriates money for, even if it does it all in a several thousand page bill at the last minute.

So a federal agency like the FAA generally prepares a budget recommendation that would be submitted to Congress as part of the president's budget, but then Congress would decide how much (if any) to appropriate for each activity, down to the level of individual programs in many cases. By the time Congress gets done with it becomes a list of appropriations, and the agency has no authority to transfer things around, spend money on what Congress did not authorize, or even purposely refuse to spend money (or spend less without a good reason) on what Congress has appropriated funding for.


> Liz Cheney went from #3 in House leadership to a pariah largely overnight.

Oh it's Trump fault Liz did what she did? Nope. Liz thought she'd make a name for herself, she did, and is reaping the results of her actions.


Whatever you think of her actions, her journey (and similar for a variety of other anti-Trump Republicans) illustrates the power a President can have over Congress. This has pretty clear implications for the budget process.


Trump was not President at the time of the Liz Cheney leadership controversy, that was after he was out of office. Trump is not President anymore and no one should pretend that he is.


Correct. I guess the lines are blurry for those that let Trump live rent free in their brain. Everything is Trump, even when it isn't. The PsyOp really worked.


Aka the “I’m going to break all my toys to force mommy to buy me new ones I like better” mentality, projected into chronological adulthood.


Not quite. This is more like breaking all the toys in the playpen because you don't like that other kids get to play with them.


Large complex systems don't work.


That's demonstrably untrue.


Presidents don't pass budgets, Congress does.

There's a neat trick people do nowadays where people claim anything that's unsuccessful (usually government related) was either "sabotaged" or "set up to fail" as a way of absolving them of any failures. Literally impossible to criticize.

Finally, the FAA's budget was $23 billion this year and $28 billion last year due to pandemic / infra surpluses, nearly double what it was the last 4 years, where it remained flat (not cut as you say). So it's funny how 2 years of double funding can't get it back on track, but flat funding has caused it to fall apart in the same time period.


If we didn't get to withhold for the decades of bipartisanly meandering Iraq and Afghan war strategies, we definitely don't get to do it for a brief outage of the FAA's weather reporting system.

Especially when evaluating the FAA would need to include the absolutely stellar commercial aviation safety record they and the NTSB have created over the last few decades.


I think we should have been able to withhold for unnecessary foreign wars.


How do you imagine this withholding system would work?

Does your tax burden stay the same, and they just make sure your dollars only go to the programs you support? Do they give you a list of every active program? Every single one? What happens with 800 page appropriations bills where you don't support one line-item? Can I opt not to fund the salaries of specific members of Congress? If my neighbor is a veteran, and they annoy me, can I disagree with their VA benefits?


Would love to designate which things to prioritize from my tax money. 50% family services and education, 30% infrastructure, 10% military, 10% general.


Cool; we're now the nation of "every homeless person gets a puppy but no services". It sounds great on the individual level; it's entirely unworkable on a societal level.


Is it them wasting money, or do they have a constrained budget with an ever growing number of responsibilities and scope? Do you think there are fewer planes in the air now then 30 years ago? 10 years ago?


>Is it them wasting money, or do they have a constrained budget with an ever growing number of responsibilities and scope?

Wow, it could be. Can you share the FAA's budget this year compared to 2019 so we can see how it's been constrained?


I wish my budget was so constrained.

Have you looked at the numbers?


> Have you looked at the numbers?

Have you?

https://theblogbyjavier.com/2020/01/02/aviation-safety-evolu...

See the chart labeled "Evolution of accidents per million flights". Millions more flights, and a steady march towards zero deaths. We haven't had a commercial airliner crash in the US for over a decade.

The FAA's numbers look great.


Others have poked holes in the general line of thought here, so I won’t repeat their points. But also, the uptime for these services seems really quite good. I mean an outage is rare enough that it makes national news. Imagine if they’d hosted on any of the big corporate cloud services with billions of dollars to throw at infrastructure — they would have had multiple outages during our lifetimes just based on their provider doing down.


I don't think that's a serious question, but the serious answer is never, unless you're willing to go to jail or kill people to get away with it.


That’s not how taxes work. You don’t get to choose how the money is spent (other than indirectly, via elections).

If you did you’d be a consumer, not a taxpayer.


That's basically the entire point of Congress


How does shutting down the FAA improve the situation? Then we would have no weather, NOTAM, ATC etc 24/7.


From the governments point of view failure means they need more money, not less. Not paying taxes just means you get arrested, they don't have any incentive to fear you doing that.


Until the failure causes political embarrassment or change. It required the meltdown of healthcare.gov to kickstart the use of modern tooling and methodology in gov't IT.


Just swap civil services budget with military budget.

Imagine what NASA would do in ten years with Pentagon's budget swapped.


Done.

NASA's budget is now $0 though, since the military budget all went to Medicare with nothing left to spare for anything else. Along with the FAA and every other government department.


Citation needed?


You need me to Google Medicare budget and defense budget for you?

https://en.wikipedia.org/wiki/Medicare_(United_States)

>In 2020, US federal government spending on Medicare was $776.2 billion.[3]

https://en.wikipedia.org/wiki/United_States_Department_of_De...

>Annual budget US$721.5 billion (FY2020)[2]

Hopefully, I won't need a citation that $776 > $721


It's much worse than that. Defense budget is hidden by breaking it down to many many different divisions. "Homeland Security" is defense with massive budget but it's separate from Pentagon, DoD, etc.

Anyone think the NSA budget for "defense" is included in Pentagon, DoD, Homeland Security?

Anyone think the trillion dollars in weapons being sent to Ukraine for our proxy war with Russia is in any visible budget?

Medicare meanwhile cannot hide all the spending, it's right there, even the fraud in the system is visible in the spending. But defense spending hides twice maybe even three times the spending.


Your source doesn't support your assertion:

> NASA's budget is now $0 though, since the military budget all went to Medicare with nothing left to spare for anything else. Along with the FAA and every other government department.


This does not back up the original claim in the slightest which was obviously a grotesque exaggeration. Please refrain from hyperbole and sarcasm - it damages the quality of the conversation.


So, US spends approximately $12,000 per beneficiary per year. Given the cost of medical expenses, this does not seem too large..


Well the USA does spend the most per capita on health care.

Meanwhile other first world nations pay about half that and get mostly better results...




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: