Ok, I have something to add here for the following reasons:
* I build big call centers for a living
* I'm a really big Telecom Nerd
You might be wondering why something as important as 911 is not automated or accessible via any other mechanism; why is it that the most critical service in the country runs on a system that hasn't materially changed since 1984?
For one, the Public Switched Telephone Network (PSTN) is extremely reliable, authentic because it's logically addressed, and ubiquitous. The only real flaw is the addressing, which, because of Caller ID, can be faked, but that's another story (See Wikipedia on Swatting).
So, there's a huge movement to reshape 911 service to take advantage of modern things like the internet, SMS, GPS and all different kinds of connectivity. The way 911 works right now is sort of a hodgepodge all over the place depending on which vendor the municipality has selected for the Public Service Access Point. There is not one standardized 911 system across the country, which may or may not be a good thing depending on your perspective.
So everyone can agree fixing 911 is a great idea, BUT why hasn't it happened already? I can point to a couple potential reasons:
1) It's a really big contract; supposing PSAPs get standardized across the board, we're talking about many millions, potentially billions of dollars. That means lobbying and negotiation and time (oh by the way Sprint is currently leading the charge but I find it unlikely that the US will let a foreign-owned conglomerate manage their backbone emergency services).
2) The current vendors see nothing wrong with the existing solution, and have projects like what has happened in New York to stand against the tide of change. Their service contracts provide a big incentive to keep those boxes in those cities.
So I think there's a lot of reasons why 911 is as bad as it is, most of it isn't technical, but political, as with most old ridiculous infrastructures.
It's amazing how not planning for obsolescence almost automatically results in political conflict down the road.
What's amazing is that this story seems eerily similar to when they upgraded the call centres for the London Ambulance service 21 years ago, and a more recent failure in 2006. You would have thought anyone dealing with these systems would have read that case study at some point?
As with everything gov, too much money leads to under the table deals.
It's ridiculous that we have 200 people constantly updating code for my sound card in Linux at 2 different projects, but no one can give out time to improve a vital system like this which eats billions of my taxes.
...just because someone went to college with someone else or somebody got golf membership somewhere for free.
Just an FYI, 2600hz, where I work, is building an free/open-source operating system for Telecom that could easily support nationwide IP-based 911 service.
All we needed to fix the problems laid bare in 2000 was a foolproof way to encode ballots. Open source software running on a PC connected to a card punch machine would have solved it.
What we got was something that undermined the whole voting process. And at the cost of millions.
Again and again and again, our decision making system allows a minority to waste our money.
The only fix, in my opinion, is giving you the right to choose who represents you.
And I don't mean the right to pick from two alternatives chosen by someone else, and not even getting that unless you are in the majority.
If you picked your lawyer or doctor from two preselected choices and didn't even get that guy half the time, wouldn't you conclude that someone else was in control?
So Here's a brief overview of what we're doing and why we're doing it this way.
Asterisk doesn't scale without a lot of cruft and it's not accessible. At the high end of the market there's only gigantic switches from companies like Cisco/Alcatel etc; and those are $$$. So we see a market space inbetween cheapo asterisk and high-end cisco.
The crux of what we're doing is a lot of work in AMQP (RabbitMQ) to disintermediate the underlying infrastructure from the services. The point is that we don't ultimately want to care what underlying components you run on your network, only that you have components that can be consumed. Think of it as TelecomSDN from a management perspective and a distributed switch from an infrastructure perspective.
The stack as we distribute it today is:
* Kamailio (Border Controller)
* RabbitMQ (AMQP messaging bus)
* FreeSWITCH (Open-source switch)
* BigCouch (Distributed database with Dynamo Quorum to help deal with Network Partitions)
That's the gist of it; all on CentOS and all open-source. We may close source some modules as we go forward, but right now just about everything is open. Using a few clicks in AWS you can run an infrastructure capable of competing against AT&T.
There's a lot of telecom infrastructure that exists which can route calls. There aren't a lot of telecom infrastructures that have built-in fault tolerance and linear scaling.
EDIT: I should note that we've developed custom modules for both Kamailio and FreeSWITCH for our use case. We had to write a multi-threading module for Erlang for FreeSWITCH, as an example.
Will it scale down to a Raspberry Pi? I realise I'm completely not your target market but I need something that does VoIP better than my router does it.
We ran openSIPS before but we found the pace of development to be higher on Kamailio in general. Plus most of our customers are running it instead of openSIPS.
We're currently trying to build the world's smallest cluster; seven raspberry pi's will hopefully support 100 phone calls or so with no single point of failure.
If you're looking for something single server you want freeswitch by itself. That thing is a beast if you just want call processing and don't need carrier grade reliability. We made a frontend called BlueBox which is a very popular distribution; we're revamping it for Cluecon (hopefully we'll have it out by then) so if you're going to try this I'd recommend waiting until the new version comes out.
Lastly, there's a lot of research going on at FreeSWITCH about running it by itself on the Pi. I believe it is workable now, you'll probably find a bunch of info with a quick google search.
The supposed advantage of a hodgepodge is that somewhere the right approach has been tried and is working well, and pressure will mount on cities with inferior systems to either match or adopt the better systems. Where, if anywhere, is 911 being done well, and how slowly or quickly will the successful approaches spread? Do you think this dynamic will lead to places like New York having faster and more reliable 911 systems, or should they concentrate on their own efforts?
If you look at the way 911 works, there's no location information unless you use the cellular radio (and this assumes your carrier is using the cell tower to anchor your location). If you use IP to call, let's say via an application like RingCentral, how does the 911 service know whether you're in San Francisco or New York? It doesn't.
This isn't pie in the sky fantasy, this is actually how all over the top VoIP connections work. You pick a location and that's where your 911 calls originate. It doesn't matter whether you're using your SIP client in New York, your 911 call will still go to a San Francisco 911 center because your office address is in SF. That's not good.
So IMHO the right approach is a top-down rewrite and rolling that sucker out nationwide. Normally, we'd see one vendor start to dominate over other vendors, and while there has definitely been some consolidation in the 911 market, the fact that these services are federally mandated (Read: Allocated budget automatically) there's not a huge incentive for municipalities to shop around. These systems are essentially free as far as they're concerned.
Part of the reason we're not on IP is because the federal dollars can't be legally spent on IP in many places (gotta love lobbying). In short, the correct approach, IMHO, is IP-enabled PSAPs with GPS information for location combined with reform efforts in congress. The logically addressed world of the phone network is a fantasy and our emergency services should model reality instead of our dreams.
Taking for granted that VoIP location information is an unsolved problem, what about systems that are simply fast and reliable and don't leave people scrambling for pencil and paper?
I have no inside knowledge, but wouldn't be surprised at all if many of the problems of the new NYC 911 system is due to a decision to rewrite the code from scratch, losing much of the knowledge and bug fixes embedded in the old system's source code.
I know but you have to understand, Joel makes excellent points, but if most of the systems are written in COBOL it makes them hard to extend over time.
We're talking about software from 1984 in some cases although it's all littered with security holes, bad interfaces and slow response times.
In the case of NY, I could agree with you, but I would say that the problem was significantly overcomplicated. There are better systems that could be used to deliver something like this. I think that, yes, sometimes you do need to do a top-down rewrite, especially when everything is wrong.
I do agree with your point that the NY incident is likely attributable to poor software design, but I disagree with the conclusion that any rewrite would suffer from the same malady.
I call CHP 911 frequently (2-3 times/week, for reporting objects in roadways, etc.); it seems to work better than any non-military 911-type service I've ever used.
CHP was traditionally where all cellular 911 calls went, too, but that got improved over time.
You mention Sprint as leading the charge and they've got issues of their own. I can't find an article at the moment, but Sprint is a provider of PTT/cell phone services to a local Portland, OR area police force and that police department is being forced to look for another vendor because Sprint has proven to be really, really unreliable.
Sprint as two networks right now. One is the legacy iDen network they acquired in the Nextel deal, the other is their Sprint/Clearwire 4G WiMAX/LTE monstrosity.
The iDen network is basically analog, so transactions like presence and push to talk are binary and the state is always transmittable. The same cannot be said of IP. This is a problem we've spent a lot of time working on and I can guarantee none of the Big Telcos have a solution yet. So the answer is, the iDen network's PTT is basically analog and easy to manage whereas the LTE network is IP but difficult to manage. They want people off of iDen because running two networks is a massive waste of money.
You mean the Integrated Digital Enhanced Network is analog, wait what?
The actual problem why noone has come up with a solid way to support PTT on cell phones other than iDEN phones is because iDEN is a rare breed of radio system called a Cellular Trunked Radio System, it was designed to provide a trunked radio system that could scale to large footprints. Unfortunately it never caught on with anyone but Nextel and a few other providers, most likely because of the need to totally replace other systems. Project 25 did not have this issue because agencies could gradually transition from analog conventional and analog trunked systems to digital Project 25 trunked radio systems or hybrids thereof.
So, iDEN was intended to primarily support trunked radio like features, Direct Connect and Group Connect are two of those, several bolt on upgrades later provided us with Nationwide Direct Connect and Nationwide Group Connect. This is why it performs so well at those (infact, from what I have seen, outperforming existing trunked radio systems).
Well, that was an epic education. I feel rather foolish, but I really appreciate you taking the time to help correct my mistake :). I'm not as well-versed as I should be on wireless tech :/.
That's really interesting; could you possibly recommend some additional resources where I could read further. Something beyond the wikipedia entries? Thanks again!!
I am tired of HN (and the rest of the country) taking shots at government employees. They're talented. They work hard. They take pride in their work. They do a damn good job. If you look at any good technology product that's come out of government, it's almost always been a creation of federal employees: CFPB. Data.gov. Healthcare.gov. Some neat things coming out of the OPM. Accumulo.
Got two out-of-the-blue emails today asking
"who is behind http://CFPB.gov's great graphic design."
I'm proud to say: civil servants" [1]
On the other hand, feel free to tear apart the contracting system all you want. The system of "who can check all the boxes for the lowest price" is an unmitigated train wreck. It basically mandates a waterfall-designed usability disaster. The sooner we can tear apart that system, the better (and I say that as someone who has spent a majority of my career as a government contractor).
While I'm not attempting to come off contrarian, I should say that trading potshots at government workers for potshots at government contractors seems short-sighted and highly hypocritical.
Having worked in both, I can say that both have their strengths and weaknesses. Where you would fault contractors though, I would generally fault the contracts themselves, or the faults of the contract overseers. Government contracting is adversarial, and where contractors have made major gaffes, in most of those cases you'd find that it was the government itself to blame. There's a reason it takes several hundred thousand dollars to be the lowest bid for standing up a standard Wordpress website, and I promise you, it isn't because the contractors don't know Wordpress.
Based on my experience, agreed entirely. I've worked with stellar people at multiple large and small government contractors. The structure of the contract just doesn't lead to a mentality of "let's make something great", but rather "lets get these boxes checked".
Let's get these boxes checked... while attending two meetings a day with different groups of stakeholders with entirely different visions for the product, that want to use it completely differently.
Let's get these boxes checked... while every stakeholder wants to see our project fail because they'd rather waste a few million dollars and get a proof of concept so that they can then hand the contract off to the vendor they wished had won the bid in the first place. (Note, this happens to ALL vendors, because every group has a different favorite.)
Let's get these boxes checked... while having to justify every single change to the environment to the Enterprise Architecture Board ("Why do you need to use Python? Can't you build it in Java?") as well as to the Change Control Board ("Why do you need to open port 80 to this server? Can't we simply proxy that through our dated and oversaturated proxies? Why won't your load balancing work without sticky session support?")
Let's get these boxes checked... even though the government has given us three conflicting visions of the project and now refuses to pay us because we're behind schedule because we cannot meet all the new feature requirements that have arisen.
The structure of the contract just doesn't lead to a mentality of "let's make something great"...
This mentality is probably true for most systems that are built for the internal use of an organization rather than as a product that has to compete on the open market. For example, software built by IT departments for use inside large corporations is notoriously badly designed and hard to use.
I think it depends on the organization and how many hands in the mix... If you have one or two stake holders and one person taking charge of the direction of development, it can come out pretty well.
On the flip side it isn't always management.. I've seen developers create forms that should be simple with really weird layouts on screen that make absolutely no sense at all.
How is this a shot at government employees? If anything, this is a tale of how government employees are bearing up under pressure that would have caused any of us to quit in under a week. The government employees didn't design the craptacular computer system that's causing these problems. That was designed by Intergraph, a private company.
I wasn't replying to the article. I saw a handful of posts in the discussion blaming lazy, incompetent, or malicious government employees, and didn't want to post the same thing multiple times within the thread.
I don't. I think 911 should be run by the Government. I just see the difference between 911 calls I had in the US and their equivalent in Eastern Europe (Poland). To exaggarate: in the US you're like: "Help! Fast! Somebody's dying here! That's the address right here:..." Dispatcher:" What is your name, last name, shoe size, and how do you like the weather? Wait! Too fast! Typying it down!".
Poland:"what is the address? what's happening" Bam, dispatch!
Having been involved in long, arduous government procurement processes, I can assure you that it will leave you shaking your head and wondering how ANYTHING in government gets done.
The trip from RFP to signed contract is maddening and has probably driven more than one government worker to an alcoholic stupor, wishing for the sweet embrace of death.
On a side note, I'm no longer a government employee. My name is Greg and I'm an alcoholic....
When I was freelancing, I did some government work. I was flabbergasted by the slow pace at which things moved, and that I had to create accounts in 3 separate systems to submit and get paid for even very basic invoices.
Government is a place that is ripe for disruption, but the walls around it are so high, I don't know how to really breach them. There is a huge machine in place of contractors and lobbyists which essentially maintain a strict control over all IT contracts that come out of the public sector. Massive corporate entities with an entrenched web of influencers at every level of government, ensuring that only a select few even have an opportunity to realistically compete for the jobs. The amount of overspending that could be eliminated with the disruption of those entities probably represents a significant portion of local, state, and national budgets, not to mention the likely better solutions that could be presented by folks more accustomed to working efficiently.
I'm not necessarily saying the 20 somethings who were going to make "Snapchat for dogs" could do a better job as government contractors. But certainly someone who designed infrastructure for Twitter might have some interesting perspective on how to manage 911 traffic.
Large scale disruption is impossible because the government and its employees are protected by guns and the force of law, permitting special interests, cronyism, and incompetence to flourish. We in the marketplace (minus Industrial Military Complex) have no such protections, and actually have to produce value to make a living in a fair exchange of goods and services.
Most government employees are unemployable in the marketplace, so it's in their interest to follow orders, at whatever cost, to keep their jobs.
The best thing for young professionals is to fight the system intelligently for those that have the resources to do so, or avoid it entirely.
Edit: As is with NYC's 911 boondoggle, a SF startup could've built out an amazing system at a fraction of the price. But that wouldn't be the government thing to do. It has nothing to do with building a quality product.
> Most government employees are unemployable in the marketplace, so it's in their interest to follow orders, at whatever cost, to keep their jobs.
Citation needed.
If you actually have evidence, I'll bite, but my anecdotal evidence tends to indicate otherwise. Generally, issues are caused by good people being hamstrung by ridiculous rules, not by bad people.
> Most government employees are unemployable in the marketplace, so it's in their interest to follow orders, at whatever cost, to keep their jobs.
My anectodal observation is that this may be true for some individuals, this could be said of any large institution. I encountered many capable and intelligent individuals in my time contracting for the government.
The problem I see more has to do with the legal mandates in place, and the labyrinth of regulation and processes. Unless you are quite savvy to how the machine works, it's nearly impossible to get a contract. And the steps to learning how to navigate these processes is pretty lengthy and would make even the most resilient individual completely jaded with the whole thing. Finding a way to unwind the over-regulation in place seems to me to be the only way this market can be opened to true competition.
Responding to your edit about "a SF startup could've built out an amazing system at a fraction of the price"-- I hear that kind of attitude a lot, I'm not so sure it holds to scrutiny.
Tackling the problem of a system that must be 100% reliable, synchronous, with complete uptime from day 1 is very very difficult, and not many startup environments will prepare you for it. A lot of systems that are scaling decently now (twitter was used as an example early on) had extreme early growing pains.
In fact, I think it is impossible to build such a system without significant upfront cost, because a lot of the cost should go into the testing phase. It's almost as though you'd want to create a testing call center that receives realistic calls and needs to route them correctly, and have it running for months while the software is iterated on. A lot of effort should go into making the testing center as realistic as possible, and a lot of time should go into teasing out the inevitable bugs.
Sounds like it doesn't need to be 100% reliable, though -- they have a painful backup system in place.
Also, replaying traffic from the old system into the new seems like a viable way to test out this sort of system. Or, they could have run both systems in parallel, and just sent a percentage of the traffic to the new system and slowly ramped up. I wonder why they chose to just cut over rather than doing a full and seemingly irreversible cutover.
Not that many startups would have done things that way, since the nature of startups tends towards the green field. But plenty of more-established tech companies would have, I imagine.
> permitting special interests, cronyism, and incompetence to flourish. We in the marketplace (minus Industrial Military Complex) have no such protections, and actually have to produce value to make a living in a fair exchange of goods and services.
Cronyism and incompetence may or may not be inversely correlated with success, but the market does not select against those traits, specifically.
To abuse a metaphor: as a business, you don't have to outrun the bear. You don't even have to outrun the other guys. And businesses are, in the end, run by collections of irrational-in-aggregate human beings.
>> Most government employees are unemployable in the marketplace, so it's in their interest to follow orders, at whatever cost, to keep their jobs.
Some good points were raised about this, and what I should've said instead is that it would be extremely difficult to suddenly absorb a lot of displaced government workers into private industry. But aside from the protections they receive of near-absolute job security in any market condition, I subscribe to the thesis that as government expands, rights are lost.
> permitting special interests, cronyism, and incompetence to flourish.
Apt description of Enterprise class business's. or any large organization of humans.
Also, this system (as most things people blame gov for) was made not by a government but by private sector capitalist companies. Maybe who've should quit outsourcing so much to these fuckups.
I think I've seen a few of them posting on HN, but I can't remember who. They seem to be especially pushing to get things like data.gov working in a modern way.
This kind of disruption can happen. When I was in school I worked at a small 8(a) (women or minority owned federal contractor) firm. All the developers and designers were under 30 and we built all kinds of CRUD apps for the federal government. Now I work at a startup and the real main difference is that everything moves much faster.
Not everyone can start an 8(a) firm but we could get legislation passed that creates a more competitive way for any small business to work with the federal government.
I'm not entirely sure this type of disruption CAN happen because this whole process starts and ends with the Legislature, at least at the State level. I was mired in so much bureaucracy it felt like trying to run in molasses.
We certainly had the technical capabilities to disrupt the process, but the process itself isn't even self-aware enough to know it's the problem..
> Government is a place that is ripe for disruption, but the walls around it are so high, I don't know how to really breach them.
I have had this attitude before, but I am not sure it's a healthy way to look at the problem. What you were flabbergasted by was the inefficiency of bureaucracy. But this is not a government problem per se: large companies can also become bureaucracies; large militaries tend to have similar tiering and divisions to bureaucracies. I think we sometimes think of bureaucracy acausally -- "it's just always been that way, it always will" -- when the truth seems more subtle: something like "bureaucracy is a scaling choice."
I really think that bureaucracies are born as a sort of divide-and-conquer approach to scaling a social system. So it's something like an O(n * log n) total overhead, where n is some abstract quantity of resources; and what you see on-the-ground when interacting with the government is (overhead)/(resource), which grows like O(log n). And it seems to have unbelievable overhead just because n is very large. The pain point which drives this scaling choice is probably free-rider problems. If you can't address free-rider problems, then the potential exploiters grow at least linearly with the number of resources you command, and your scaling is O(n^2) at best. So maybe in some sense bureaucratic solutions are "efficient" -- as crazy as that sounds.
In other words, if you ask "Why do people get shackled by these strict rules?", there are two tones (meanings) you might intend: one is despair ("surely we can do better!") the other is curious ("really, I want to know why that rule is in place!"). Usually the curious tone reveals some sort of free-rider threat. That nice guy who drives the schoolbus is constrained by bureaucracy; he can't hug or praise a student who tells him that she's feeling sad today because kids are picking on her for her perceived smartness; he must act as a robot driver. But why do those rules exist? Probably due to a fear of predators who might become able to lurk in the blurred lines. And the fear of predators is created by the scale of the school-system, which makes it hard for the human discretion of administrators to protect the system from having pockets of abuse without spending even more money than bureaucracy would cost.
Don't get me wrong, if an alternative system of government can do O(n), I'd love to see how it deals with free-riders. I'm really interested in crypto-anarchism. I'm just cautioning you that maybe the status-quo -- which yes, at this scale involves big companies with political connections paralyzing the process -- protects against a lot of people just running off with the money in exchange for the government-contract equivalent of vaporware.
Even big corporates are adopting agile methodologies, and combined with an appropriate contract, they provide a game theoretic incentive for both parties to act how the other intended.
How the system is supposed to work: Contractor does a small increment of work, delivers it, and gets paid by client based on the time they put in.
Exceptions:
1. Contractor doesn't deliver anything, or the quality is so bad that bad faith is assumed. Response: Contractor gets paid for claimed work on increment (which is limited by how much the contractor is authorised to do). Contractor doesn't get any further work. The contractor doesn't want this since they lose the income from future iterations.
2. Contractor does work, but client refuses to pay. Response: Client doesn't do any more work for the contractor. Contractor's loss is limited because the iteration is short. The client doesn't want this because they lose the benefits of the contract on the next iteration.
3. Contractor does work, but it doesn't fully meet the quality / scope requirements of the client, but the client can see progress is being made. Client pays the contractor, and the contractor and client work together to ensure that the next iteration moves in the direction the client wants.
What you were flabbergasted by was the inefficiency of bureaucracy. But this is not a government problem per se: large companies can also become bureaucracies
Truer words have never been said! Try tendering or contracting at EMC...
Pre-9-11, I spent some time doing some high-level analysis, talking to users and administrators, of a nation-wide EMS system for an entire country (Sorry, can't say which) It was a U.S. firm interested in figuring out exactly how much it would cost to create a national system so they could bid on it.
This does not have to be a very complicated system at all. It's basically a real-time workflow system. Information comes in, is assigned a type and other data, and depending on the values is routed to various other people, which also tag and update it. In a way, it's almost like a bug-tracking system. Tickets get made. Tickets get worked on. Tickets get resolved.
Now combine that 10K problem with a large rollout, lots of administrative BS, and $2 Billion? It's wonder the damn thing works at all. A group of hackers could cobble together something over a week with better uptime and more reliability than I'm inferring from this article.
The technical complexity of the problem has nothing at all to do with the political and legal maneuvering required to get and execute a large government contract.
The technical complexity of the problem has nothing at all to do with the political and legal maneuvering required to get and execute a large government contract.
Or a large contract with any organization, as far as I can tell. In universities, the whole education-ware thing is a mess, with huge piles of contractor dollars going to junk like Blackboard, who manage to sell things to high-level administrators. By far the best education-ware software I've used is stuff that was developed for almost nothing by comparison: 1) a course-specific tool a student made as a senior project; and 2) a wiki. In industry, I have some third-hand knowledge of a large procurement project at a petrochemical company that spent somewhere in the 9 or 10 figures on document digitization and didn't deliver any document digitization.
Absolutely. And they're dysfunctional for many of the same reasons.
I've worked both, and government is the big leagues. I've seen private organizations squander a few hundred million bucks here or there on disasters, but government projects can easily break into the billions or tens of billions of bucks -- and drag on for years. There's just much more money available, many more players who all want some random benefit from the project, and random rules to stick to. But you're right: this is a function of large numbers of really smart people all trying as hard as they can to do the best job possible. That's the craziest part about it. It's a systemic problem, not one that you fix by just somehow getting smarter people involved or buying a new project management tool.
I'm not sure when it started, but I was hearing about it from around 1990 I think, and the project got axed around '95. I assume eventually they did get something done, either with a new contractor or in-house.
I actually had a part-time job scanning large technical diagrams through a sheet-fed scanner in the late '90s, for a different company (a NASA contractor). It was technically not what I was hired for; the main reason they needed to bring in someone part-time is that they needed additional staff to help "debug" a space toilet. So my main job was using it, and recording some basic data (pH, etc.) in a logbook whenever I did. Since I was sitting around for a few hours a day in between toilet debugging, scanning technical diagrams seemed to be the main other thing that needed doing. Seemed like a pretty ad-hoc process, but I don't know what that one was like on the backend.
"This does not have to be a very complicated system at all. It's basically a real-time workflow system. Information comes in, is assigned a type and other data, and depending on the values is routed to various other people, which also tag and update it. "
So this is basically something that can either be made with 100 usd for pizza and diet coke or 100 billion? Makes sence, we should probably make more of the first kinds of systems.
> Lowest bidder? No one loses their job for someone dying?
Government is not as bad as you think. It's worse.
Lowest bidder really causes a lot of problems. My company recently bid for a contract for an EU agency.
We were not chosen, because the winning bid offered senior staff at a rate of 0.1 euro per hour.
The bloody idiots at Government procurement calculated the average as (senior+junior)/2 and awarded the contract to these crooks with the flawed bid as their average was the lowest.
It's not unreasonable to assume they will debit enough extra junior hours to offset the senior time.
I am not even allowed to tell media about this, by contract.
I don't think it's a rare event.
Let's not forget that the developers of the system are potentially handing the system over to another party for O&M, didn't have flexibility in choosing what software to use, didn't get to prioritize the requirements and blue buttons were equal with system stability. They may have even had a separate contract do the design spec, and IV&V. The diffusion of responsibility combined with the rigidity of the procurement process is such that it is often impossible to achieve the flexibility needed to devise an optimal solution.
They don't always take lowest bidder.... sometimes it really just boils down to who the contractor knows, or what favors they've done for people (even if they are incompetant)
This is generally not true. What it generally boils down to with government contracts is a really sweet dog and pony show from the RFP process. Generally people lack the critical thinking skills necessary to determine if a company can really fill their needs. They don't know the difference between a COTS system and one built from the ground up. They don't know the difference between a hosted solution and one running on their own servers.
I would say very rarely does it come down to simply being the lowest bidder or nepotism.
Just to throw in a situation I'm in at the moment. A public hospital has put out a tender for a new MRI scanner. I know who is likely cheapest (GE) and I know who is likely the most expensive (Siemens). I suspect Philips falls nearer the higher end, but will be in between. I have nothing to do with the purchase but work with those who do. I'd bet everything I own that GE don't win as the corners cut and unreliability are not winning any favours. Their scanner ticks all the boxes, but in the worst possible way. Multi-transmit RF? Yes! (But it doesn't work). Wide bore? Yes! (But you can't scan any further laterally than with a narrow bore). Propeller type imaging? Yes! (But scanner crashes, weird artifacts arise or the utterly crap interface gets in the way). The list goes on for probably another 50 points. Philips and Siemens have faults, sure. But they are more the result of compromises and are stylistic where the system is that way because someone chose to make it like that. Not because someone screwed up, forgot, didn't check, didn't know etc. some places chose quality over price, and this publicly owned MRI scanner will be in this category.
Sometimes it does, due to the (near) total ignorance of what this is all about.
10 years ago I was briefly employed by a contractor for a state government that was working on a system a bit like this, although not hardly so life critical. The original system had been written and customized by the one of the big names like Lockheed or CSC and was pretty good as these things go ... but the state put continued maintenance out for bid and gave it to another company. With no transition, and really counterproductive budgeting.
I.e. around a year previously another guy was hired to do what I did, and let go after a month or two because of money.
What was I hired to do? The most difficult job of software archaeology I've ever attempted, not counting one code base that was impossible (https://news.ycombinator.com/item?id=6092960). The original programmers, probably on site, used SCCS (!) ... until they stopped some months before producing the binaries running on the system.
Other minor details: the system ran on DEC Alpha hardware and their version of UNIX(TM), and of course by then it was a doomed system with no migration path for binaries. The state budgeted the project and made its milestones based on estimates made by another consulting company, and those were not based on reality.
I bailed when it became clear there was no possible way to meet the state's demands, e.g. a point when I realized that if there was a way to recreate the production binaries I wasn't going to be able to find it, and they were assuming the sources were in a lot better shape.
They should. I think suing them over the deaths of hundreds of people and having the company that made this system go bankrupt is the very least they can do. Then the secret service can come in and crack some more necks. They got paid 88 million US dollars. For that you can make a system that Just Works or be honest up front if you can't get it done (on time).
Large organizations have a tendency to demand custom solutions, and then they want to throw the switch all at once. That is pretty much opposite the bottom up, self-organization, of HN themes.
If you could "bubble up" a "good enough" 911 system, it probably would be reliable and probably would scale. But the contract will be given to the company which will say they can provide a big custom solution on a deadline. It's almost worse if they believe it themselves. A dishonest vendor would at least understand the scale of the problem(s).
A lot of posts here talk about open source projects or the relative simplicity of implementing something like this at it's core: I agree with the idea though I understand that emergency systems require a high level of reliability similar to mil-spec hardware I imagine. If there was any justification for that price of 88+ mil, that would be it. Looks like that went right out the damn window. All systems have bugs, I think we can all agree on that, but I'd like to know that they followed the most sensical procedure to rolling this out. Did they test it prior to release? How did they test it. Did they bother with unit tests at the lowest level? Did they perform thorough integration tests? Did they consider rolling out incrementally (if possible) the new system so that they wouldn't be completely left out in the rain when it failed? What contingency plans if any did they build into the system upon failure? What was wrong with the old system, and what did the new system promise to deliver?
It's a lot of questions I wish more journalism would answer. On another note. It's interesting to think about open source. As some have mentioned, it's difficult to imagine the feasibility of actually doing an open source version and getting it adopted (though that may be because I don't know how EMS stuff is structured). Though I think all would benefit from a source code release of this system so we could actually crowdsource a better friggin' version because this one clearly isn't doing its job...since May.
I work in the call center branch of a fairly large corporation. Obviously we're not saving lives and dispatching fire trucks, but for some reason it seems like we're dead set against doing incremental rollouts and user testing in any meaningful way.
Don't get me wrong, I see things get to user testing phases and we have reasonable ways of logging bugs and problems, but almost every project I've seen had kept on trucking to meet an arbitrary deadline regardless of how poorly any of the testing goes, then the system hits the floor, chaos ensues and everyone runs around in a panic until someone managers to staple together a solution.
Kind of depressing that it looks to be the same in the government with calls that are a little more important than ecommerce sales.
I want to say that the guy that is in charge of the 911 phone network spoke a few times at a call center conference I recently went to.
"but almost every project I've seen had kept on trucking to meet an arbitrary deadline regardless of how poorly any of the testing goes, then the system hits the floor"
And the reason for this is simple..the one person who can pull the plug on a project of this magnitude, won't. Simply because s/he doesn't want to have to answer to having already dropped 88 million with nothing to show for it. There becomes a single person to blame at this point, whereas a failed system that gets implemented, suddenly allows for MANY people to be thrown under the bus.
Totally agree. Even in projects at work, the worst possible thing people think they could do is admit that something they're working on doesn't work the right the first time.
I will say, though, that in the case of most projects I've worked on that are failing at some point, we do have something to show for it. We just have things that need to be fixed. Even a totally failed try is something you know doesn't work.
It seems like people equate admitting any failure as admitting total failure.
It's one of the only reasons I like my job. My boss is comfortable saying "this didn't work, that's fine, let's figure out why it didn't work and make something that does."
I recently had a project to reduce a certain call-type to our call centers. We implemented a new system in one part of the company after doing some research into the call drivers, and did a follow up a few months later. My followup showed nothing had changed. I don't have any problem saying that. It just means we need to look into which part of the plan failed, why, and how we can fix it. Either way we'll have a better understanding of the problem. I'd rather do that than fudge the followup, keep the problem, and pretend everything was fixed.
"My boss is comfortable saying "this didn't work, that's fine, let's figure out why it didn't work and make something that does.""
The taxpayer is much less forgiving about "failures" and tend to trot out pitchforks and torches at the drop of a hat. They're much more guided by perception than actual facts. So for them, there's very little room for a distinction to be made between utter failure, failure, almost failure, not quite a failure, maybe a failure, no failure...
I don't actually know the answer to this question, as I'm not an overtly politically savvy person, but, are they really that much less forgiving?
Maybe I shouldn't say my boss is 'comfortable' with failure. He's just got the fortitude to deal with the realities of most situations and the track record to back it up.
Most of the managers and bosses I know of deal with projects in exactly the same way as this 911 project looks like it was handled. They push push push and don't accept any hint of failure because they think the perception others have of them would be unfavorable if they admitted some failure and tried to fix it. Of course, in this case it's the perception of the management and upper management members, and not a voting public. I have to say, though, their opinions don't seem much less fickle at times than the voting public.
In an anecdotal way, it doesn't seem like public works/government projects are immune to delays and push-backs. Seems like it might depend on the type of project as to whether there was public backlash?
I'm really just spitballing about something I'm not well-informed of at this point.
The main idea being that there's not a whole lot of forgiveness in the corporate environment I work in, but I stick with the boss I have because he's got the reputation and fortitude to own problems, when necessary, instead of trying to underbus everyone.
Even worse, this is a small, at least in $$$ part of a $2 billion total upgrade of emergency services. And it's the "customer's" entry into the rest of those systems.
I recently attended a lecture about developing shutdown software for a nuclear power plant.
The development process involved three key stages: Formal Requirements Documents, a Software Design Document, and finally, Coding. Each step in the process is accompanied by formal verification processes and an audit which produces a Hazard Analysis Report. Code also goes through code review and verification.
I wonder if Intergraph employed a testing plan quite as thorough.
I think that mil-spec or nuclear QC standards are probably overkill for civilian systems such as 911. Granted there should be some high standards in place, but the systems are a) not secret/classified and b) not as high liability. Yes, failure to handle a 911 call is serious problem, but not like a failed nuclear reactor shutdown in terms of cost, number of people affected, and as a last resort, ability to fall back to a paper-and-pen backup system.
I agree that this process might be overkill for a civilian system, but it seems to me that an ongoing failure to respond efficiently to 911 calls (over the course of weeks, months, years) is quite worthy of a development process that reflects the life or death nature of the software's purpose. Regardless, hopefully Intergraph fixes their software quickly so that the operators aren't put under more stress than they need to be.
Holy hyperbole. Definitely seems like this worker is frustrated beyond sense or has an agenda with saying things like:
They give rookie operators two months training, and after that just throw ’em in. They’re not ready.
Two months training is not enough training? This is implausible. At its core, they're answering the phone, determining the reason for a call and typing information from the call into a computer. They're not driving out to a site and performing cardiac bypass on people having a heart attack.
The whole thing seemed targeted toward eliciting an emotional response rather than a rational one.
I've worked in customer service and (while that's not exactly any way on par with what these operators go through) there's an entire volume of stuff to keep in mind before going in. Being an operator involves engaging in a fundamentally unnatural mindset that's often completely antithetical to what your normal responses would be. No matter who's screaming, shooting, dying, it's still your job to remain calm, clear and transfer as much information as possible to emergency workers.
Try listening to a few 911 calls. There are tons on the interwebs (LiveLeak has a whole lot). Be prepared to lose sleep for a day or two.
I did some contract work on network gear in a rural Ohio county's 911 center a couple of years ago and spent a few days around the dispatch center. I have a ton of respect for the work that the dispatchers do. I listened to them stay calm, cool, and collected while taking calls from people who were completely freaked-out (angry, injured, scared, etc).
The job is certainly a lot more than just answering the phone.
Obviously there's a whole heap of details, systems and other areas to be thoroughly familiar with. My criticism was of OP suggesting that somehow 2 months not being good enough was a stretch and the story was hyperbole. Although I haven't dealt with 1/100th of the stress a 911 operator has to go through, I can say that sentiment is quite silly.
You can say that based on experience? I don't know. I hear account A from someone who actually does the job, and retort B from some guy on HN who says little more than "answering phones can be stressful, but it's not hard." I'm inclined to believe the guy who has experience in the matter.
I was the CTO of a large publicly traded insurance company that had 3 call centers across the country and thousands of sales reps and customer service agents. I designed user interfaces and system architecture that enabled them to rapidly quote and service prospective customers and existing customers. I spent 1 day per week in one of our call centers every week with a headset on listening in on calls eight hours per day. I did that for 10 years.
I know call centers and I most likely know them better than you do. I think considering someone being given two months training "throwing them in" to be hyperbole.
None of that meets much of the training necessary to save lives. The insurance company has to keep current customers and make sure to gain new ones. That's it. Only monetary loss or gain.
The callers at 911 centers (and I'm guessing you still haven't listened to any of these calls) have to handle different systems (with no ability to modify the UIs) contact different services, handle vital data and filter it out from non-vital ones when anything critical lost would mean deaths (as it had been).
E.G. To become familiar with police, emergency medical and fire codes. Respond to caller situations appropriately, calmly and above all else, cogently. This is a far greater burden than any insurance operator will ever face in the line of work.
You've proven that no matter what I say, you will discount my experience and make guesses as to what I have or haven't done. Why would I continue writing to you? Have a nice day.
aren't you simply wasting a tone of precious time checking the details? Can't you find out the cause, send the folks the help, and only later gather all the details and pass them on to the crew on the way?
Just a couple of things callers are surprising ambiguous about (this is just from the calls I've heard):
- Where are you calling from? (Shockingly, this one is absurdly difficult to get from some people)
- What's the emergency? (Often times, they don't even know)
So you have to decipher, poke and prod until you get a response. You have to also understand that emergency resources are limited. A cop at one address is one less available at another, more critical, location. Same with ambulances and firefighters. What may turn out to be a mundane call could have diverted precious rescue resources form where it's really needed. So it's not that simple.
I've done Lean work for hospital new patient intake centers. When it comes to triaging patients in extremely homogenous centers, the real training period is a few months long (formally it's one month, but that's really just to learn your way around the computer systems). In disease centers where the patients are calling with heterogenous complaints, the formal training period is also a month, but everyone agrees the real training period is just about two years plus weekly meetings with docs to learn about how to triage that week's oddball/edge cases.
I suspect 911 calls are no less complex than scheduling people for a doctor's appointment.
Have you called 911? The operator essentially does triage over the phone, and advises what intervention the caller should engage in. They are not just there as dial-an-ambulance.
I thought it was very easy to believe. A central conceit of the piece is that the system's maddeningly insane and needlessly difficult to interact with. If the system actually worked as advertised, you would certainly be right, and no doubt that was a goal of the refactor.
But training people to deal with crashes and exceptions is very hard, especially if it is under pressure. Often, this sort of training only takes places as catastrophes happen. Imagine you're an operator, and you have to use a system that's made of failures and bugs, you're exhausted from working doubles, and you can't debug anything: when it fails you have to rely on someone else.
(I have never worked in 911 operator facilities, only industrial operations.)
I'm mostly glad to see a lot of reasonable responses to this.
This attitude is one of the reasons why so many people have such terrible experiences calling in to companies. I'm talking about companies that have nothing to do with real life emergencies and life threatening consequences.
Management tends to think "they just answer phones, so we can hire anyone, put them through a week of training and bam, they should be good to go, right?"
I work in a pretty standard e-commerce environment. I was in training for awhile, and now some data analysis and software development. It's like just about anything else. Predict how long you think it will take to competently train someone and triple it. It's not easy to get people ready to handle the majority of the calls you get in a call center, and it's really really hard to get them to handle the absurd number of one-offs you might have. Again, that's without even considering the number of call types and the weight of the calls I'd imagine 911 operators get, all while having to stay polite and calm.
Two months for a 911 operator in a major city like NY sounds kind of nutty to me, especially if the training is in the format I'm expecting it would be, and not something totally out of the box.
This attitude is one of the reasons why so many people have such terrible experiences calling in to companies...Management tends to think "they just answer phones, so we can hire anyone, put them through a week of training and bam, they should be good to go, right?"
You've missed the mark pretty terribly on this. The reason why so many people have such terrible experiences calling in to companies is because of substantial training.
People in call centers today are trained to give you poor customer service. I can't tell you how many scripts I've seen that are written like:
1. If customer asks for X, tell them we can't do that
2. If customer insists, explain we can't do X because of A, B and C.
3. If customer still insists for X, tell them we can't do X but we are willing to do Y
4. If customer still insists for X, tell them we will call them back and then don't call them back.
5. If customer calls back for X, give them X
And these types of scripts are written for things where a company is legally obligated to give their customer X.
Sure, there are training issues but by far you are receiving poor customer service because the company doesn't really want you calling in the first place and they're training you not to call.
Well, I did say "one of the reasons", but you're rights that it as implicit that was the main reason. I also should have prefaced that this has been my own personal experience in call centers.
What you're saying is definitely a problem in some (maybe most) call centers. Companies tend to see call centers as places where money goes to die, instead of a potential treasure trove of information about their customers and what's going wrong or right with their strategies. I think that does lead to what you're talking about. Everything becomes about low balling the customer. If they have a return, you try to give them ten bucks to keep it. If they think the item is broken, you tell them why it's normal for the item to be that way. I'll be the first to admit that I underplay that portion because my financial situation is fairly good and I'd leave a company that acted that way. It's misguided and doesn't do anything for building your brand.
I'd take that a step further, though, and say that your scenario suffers from another big problem. Some call centers place a huge priority on wrote scripting and the customer ends up feeling like they never stopped talking to the IVR. Some reps can put life into a script, but most can't, and you end up with a really sterile, impersonal experience that screams "you're a statistic to us."
My personal experience has been, the more you can let your reps drop the script, the higher your CSAT/NPS scores will climb (obviously to a point). Of course, people have guidelines, but I tend to think most of those go too far. If you're in a brick and mortar retail shop, you don't usually have someone hiding behind a shelf marking down whether or not you used the customers name 2.3 times during the interaction. And I should mention that this is going to change depending on the type of business. An e-commerce site processing a return isn't going to be quite as mired in legal scripting as a financial institution laying out the terms of a loan.
As a final note, I'll get back to the 911 operator context. Yes, the article is written in a way to elicit a response. I'm not going to hold it up the standards of a well-respected peer-reviewed journal article. I still absolutely think you're wrong with that one, for a lot of reasons others have spoken on. It's not in any way implausible or hyperbolic to think a 911 call center would take two months to adequately train a rep, especially if they're using fairly standard training practices.
If they don't triage properly, they'll be sending the police to deal with angry kids whose parents won't give them their pocket money. Or they'll not send the police to a violent confrontation.
Also, you have to consider the kind of people who make a lot of 911 calls. It might be a drug user getting threatened by a dealer. It could be a drunk guy, who's had an accident. It could be something really embarrassing ("I fell on it"). Doing triage can be quite difficult, when the caller doesn't really say what has happened, but they need to find out enough to justify sending help.
That would be true if you were a sociopath. But those with empathy need to learn to deal with stressful calls, and I imagine that would take some time and effort.
When signing up for a phone, you must pick a 911 provider. The cost is fixed, so you're not choosing based on saving money, but based on quality of service. Each provider would be audited and would publish response times, etc.
You could choose the government program described in this article, or go with someone more technologically advanced.
Because the beneficiary of the 911 provider isn't likely to be you, it's the person who needs the emergency who probably can't come to the phone.
If someone needs to call 911 for me, I want to know it's regulated and up to quality - not based on whether the person who happened to pick their provider actually bothered to research a good one, etc
Monopolies? 911 dispatch centers should really not be a battleground for competing commercial entities. They're a public service. You may disagree with the general idea of a public service, but it's really one of the very few areas where government can do (and frequently does) things for the good of everyone.
Call centers might be bad sometimes regionally, but that doesn't mean they can't work in principle. For example, in my home country (Germany), there is a public dispatch system for all emergency services and after having had to use it repeatedly for a range of things from routine to disaster response, I have to say it works really well. And I wouldn't say Germany is exactly the paragon of efficiency when it comes to public services either. So with a little bit of political will, it's achievable - as it should be.
Yep, I expected to hear that Intergraph were responsible.
12 plus years after disasters in NZ and Australia, (the latter resulting in a Royal Commission) they are still at it.
"This year [2000] we see the very two people who were involved in the police force who were making all the recommendations about Intergraph, go to work for Intergraph."
Now imagine for a moment 911 line was a private company. People subscribe to it, pay a monthly fee, call it, wait to get through and don't receive help in time. What do you think people would do then? Stop paying and likely file a lawsuit. Good luck doing this with a government.
This would of course tie in with the private police force and private fire service, right? And when the customers stop paying and sue the company into oblivion, who handles the calls? Do you prevent monopolies from forming so no one company becomes responsible for all 911 call traffic? That doesn't sound very libertarian to me.
Let's presume we're interfacing with public services here, just for shits and giggles. What happens when grandma Jean forgets to pay the bill one month? Does she lose access to those public services when she needs them most because her 911 is redirecting to a busy tone?
1. When people say "but what about monopolies?" they seem to forget that we do currently have a giant monopoly, which is government, that prohibits others from entering the market by force, not just by market instruments such as predatory pricing (although that's the case too, since it is claimed the price of government services is zero). Even if you disagree with me that monopolies historically never remained monopolies for long without government support, wouldn't you at least agree that a private monopoly is still better than government, because I can still stop paying immediately and not expect men in suits from IRS on my doorstep?
2. Proponents of government services always bring up cases where grandmas or disabled or poor people somehow suffer because they don't have enough money/physical abilities/brains to handle things. I agree those cases exists, but it very often is presented as if half of the society is in someway dysfunctional and requires government help (even if that's indeed the case, we should rightly see this situation as abnormal and fix it, not feed it). But the truth is, in a society there's indeed always a % of people who are dysfunctional. My guess is, it should be no more than 5-10%. Those people can be helped by voluntary donations and charities. Beyond that, we got people who made their choices in life (not buying insurance, eating unhealthy, drunk driving etc.). While I'm not necessarily claiming those people should not be helped when they need it, but the decision to help or not should lie with the body that actually does the job: that is, if a drunk uninsured driver is brought to a hospital, it is up to the charity that sponsors it and the hospital managers or doctors to decide whether to help him or not. By leaving this decision up to the government we simply incentivize all sorts of bad behaviors at the expense of people who have no intention of supporting it.
1. The government monopoly, as broken as it is, should not discriminate in matters of health and safety, whereas a corporate system that you're proposing would. I am Canadian and am not interested in defending the American system. We have free healthcare (so long as you're a resident paying taxes, and others of course are billed after receiving care).
2. So, in your world, the guy who seemed to have alcohol on his breath can be left to die at a hospital. Sure. I think that's poor example, you'd be hard pressed to find doctors who would refuse to save a life in such a case. The reason I trot out grandma is because it is the poorest and weakest in our society that need protection the most, and your system would leave all that in the hands of some charity... hand-waving away all the issues that creates.
1. I respect the fact you don't want to discriminate when it comes to matters of health and safety. I, on the other hand, would like to discriminate. If a person who led unhealthy lifestyle knowingly requires medical attention, I want to have a default option of not paying for this out of my own pocket, because I don't know that person at all. Now, say I'm a Canadian citizen too: would you give me the same amount of respect and not require me to pay tax money for the things I have no intention of supporting and allow me to only pay to my insurance company?
2. In my world what's most likely to happen is that this guy would be helped immediately, but the hospital stuff later will report the incident, the guy will lose his driving licence forever, will be unable to buy alcohol and will be required to pay to the hospital over time. I'm not gonna go into detail of how it is achieved without a government, but it's quite very possible and would even be enforced more effectively.
So, yes, my main point is: forced universal anything is bad and immoral. Everyone should know exactly how and where their money go and be free to decide what to do with their money. If I dislike how my charity works and if I dislike that it doesn't help drunk drivers, I switch to a different one which does. If everyone in a society agrees to pay for universal healthcare voluntarily without a threat of IRS or similar agency showing up if they don't pay, then I have no problem with it.
> "If a person who led unhealthy lifestyle knowingly requires medical attention"
It's interesting how, when this particular viewpoint comes up, the hypothetical individual is always deeply irresponsible, borderline malicious. But the person making the argument never is.
> "Now, say I'm a Canadian citizen too: would you give me the same amount of respect and not require me to pay tax money for the things I have no intention of supporting and allow me to only pay to my insurance company?"
No, because this is a fundamental philosophical difference that runs to the core of who we are. I value the collective, and am willing to make sacrifices to my individual in order to advance the whole.
As a Canadian citizen myself, stay on your side of the border, please.
> It's interesting how, when this particular viewpoint comes up, the hypothetical individual is always deeply irresponsible, borderline malicious. But the person making the argument never is.
No, I may be the most irresponsible person in the world. The difference is, I'm not asking you or anyone else to pay for it.
> No, because this is a fundamental philosophical difference that runs to the core of who we are. I value the collective, and am willing to make sacrifices to my individual in order to advance the whole.
So would you then personally come to my house with a gun and demand money and escort me to jail if I politely refuse to pay, explaining my reasons? Will you do this, knowing my family is going to suffer? Will you look my kids in the eyes and say "Your daddy is a criminal because he refuses to pay for what I think is right".
> As a Canadian citizen myself, stay on your side of the border, please.
Please do not assume everyone here is from the US. I'm not talking about a specific country, I'm talking about a principle. US is socialist in many ways too.
Sure you are. The moment you're out of your financial depth you'll scream like a little girl and recant.
You presume that a private system yields some kind of low-cost alternative to single-payer systems. That's not the case, and it's so far from true in the US that it's laughable.
You are the one who needs to look into his kids eyes and explain why they can't have critically needed health care. It should start with "Daddy can't do math", and end with "and so my beliefs were more important than your well-being".
You're going to have a very hard time owning anything in Canada that isn't subject to seizure without paying taxes. So this hypothetical property you'd be trying to protect would cease to be yours.
We don't throw people in jail or point guns at them, and frankly even if you weren't paying income tax you would be paying tons of sales tax that you can't avoid. Worst case scenario you'd be forced into bankruptcy, tax debt is dischargeable up here, unlike in the US.
So the question remains: would you personally come to my house to cease it and explain to my kids that they are now homeless because their daddy simply disagrees with what you think is right and refuses to pay to people you believe are good?
>because their daddy simply disagrees with what you think is right and refuses to pay to people you believe are good?
I'd leave out this part.
edit: or to be clear, I'd be as likely to say that as I would be to say to the children of a burglar, "I'm arresting your father because he wanted you to have nice things."
Here's how Canadians (and I'm one of them) think about health care: Some of us do crazy, unhealthy stuff, and some don't. We all have the same healthcare. Be free, do your thing. Live your life the way you want to. It costs way too much to have different healthcare for different people.
Some of you "land of the free" people really don't get it.
Being free to leave your job, and free to live your life that way you want to (without actuarial oversight) are pretty important.
Those are way more important to me than the last nth percent of optimization I might be able to do on my own personal wallet by taking on all the idiocy of privatized, customized health care.
That policy is about as backwards and fucked up as the idea of privatizing a 911 service. Those "firefighters" should be ashamed.
As a counterpoint, I refer you to all the other counties in the US where receiving this vital service isn't contingent on your bills being paid. I can see these people losing their home to a fire because they failed to acquire home insurance, but I can't see how moral people would stand by and let a house burn to the ground over an unpaid bill.
I'm not so sure. It makes some sense: Here is a service we are happy to provide, in a sense a form of insurance. The point of it is that the community pitches in, to cover the catastrophic case that we hope no one has. If you decide not to pay into such a cooperative scheme, you're abusing the trust of all of your neighbors if you expect them to save your home anyway. This is part of the reason we often use taxes to cover the cost of things like this.
On the other hand, in this case they had already deployed, and it would have been nicer if there had been SOME price that they had thought of as the "fee" for service to non-subscribers. 5k, 10k, etc: better than losing the house. Once you advertise something like that, though, people will stop subscribing, and decide to gamble that they won't have a catastrophe. The price has to either be high enough to discourage that, or high enough that the fire department can cover its operating costs by servicing X fires per year.
If you had bothered to read the article, you would have found that the fire fighting service was offered by a neighboring jurisdiction to a jurisdiction that had no fire service of its own.
Many rural U.S. counties have no standing fire fighters. Fires are fought by volunteers, often using donated equipment. There is nothing immoral about a town offering enhanced fire service to subscribers outside the town.
Similar arrangements are also common between ambulance companies and subscribers. Old people with a fixed income consider it a great deal to pay $50 a year or so for discounted service.
If there was a 0% tax in Monaco, there would be no way to fund public healthcare. In reality, employers are required to contribute to the CSM (Caisses Sociales de Monaco) for each employee. That's a tax.
Oh, so not just less tax than your current situation, but 0% tax.
Well, you could live on a boat in international waters, or renounce the use of all forms of currency and live on pure barter, however you seem to be asking for a state that cannot raise revenue.
There are some of them around the world, but you might want to hire a small military detail if you want to live there, as funnily enough they have a few security issues.
Freedom from having people try to kill you on a regular basis.
Your argument is that it is unethical for someone to use force to compel someone to do something against there will, right? Well, that works great as long as everyone shares your ethical views. Unfortunately, not everyone does, and there are people in this world who would happily kill you and take all your property if they could. It doesn't even have to be all that many people - if just 0.1% of the population believes "might makes right" and you come in contact with >1000 people, you're probably dead.
The reason we have the state is so that we can realize certain economies of scale in protection. We entrust a monopoly on the use of physical force to the state; in return (in lawful democratic regimes), it agrees to use that force only when just. That way we can trust that we can go about our business without being randomly killed by a dude on the street, because they know that they are in for a world of hurt if they kill someone. There have been times (eg. the Wild Wild West in America, or post-Gorbachev looting & unrest in the Soviet Union) where the state has not had that monopoly on force, and it usually ends up being very difficult to conduct business during those times.
That's not anyone's problem, but it also can't be an argument. When I say taxes are theft and people say I can always leave, they should realize there's nowhere to leave. So you're basically saying: "suck it up and pay and learn to call it taxation instead of theft to make yourself feel better about it. If you disagree, you can go let some other country rob you or go to jail." So yeah, great argument you have.
You don't have an argument -- you have an axiom that claims that taxes are theft. Pretty much everyone else disagrees. So are you going to leave or just keep whining like a 5 year old that you can't get your own way?
Well, then don't you have an axiom that taxes are not theft? Simply because most people agree with you doesn't mean you are right and moral and I am not. Would you call slavery moral? Yet throughout most of the human history, most people had no problem with it. Those who realized it was immoral either kept quiet or were persecuted.
Next you will probably tell me: but how can you compare taxation to slavery? Well, both are done by human beings to other humans against their will because the majority implicitly agrees. In both cases, if a person resists, he is punished. Tell me, how is it moral? The only difference is the degree to which the freedom is restricted and fruits of labour are confiscated. If you believe a person made his money in a dishonest way, it is justified that this person is tried and convicted. However, if you agree a person made his money honestly, by benefiting other people and creating value, what moral right on earth do you have to take any of his earned money for any purpose? He didn't rob anyone, he created value, out of nothing. Not you, not any other person has any moral right to confiscate any part of it.
You agree to the taxation by using the currency of a tax raising organisation whose stability, and therefore the stability of the currency you own, depends on that organisation being able to defend territory and keep a stable economy running in that territory, which takes resources which in that system then requires some of the currency it created.
If you want to create an alternative economic system that requires no physical territory or external inputs to enable it to run and therefore can exist without taxation, then feel free.
Empirically, what you're saying is not true. Many countries use USD as their primary currency, however their citizens do not pay taxes to the US.
But let's say I agree. I'm gonna switch to Bitcoin and start my own country. Problem is, no one's gonna let me occupy a territory I choose even though it may not be used at all (!) by a government or any of the citizens. The moment I start building my house in the woods, someone's gonna notice and send people in suits to evict me. What's the justification for that? No one uses the land anyway! Why is some government more entitled to it than I am?
In countries that use the USD as their primary currency, the USD is the currency of the tax raising authority in that country. However the citizens do also pay a small tax to the USA from the use of the USD in the form of seigniorage, i.e. they still have to buy the dollars. For coins this is apparently around 45 cents per dollar.
As for entitlement to territory, the only entitlement to territory that really counts is the ability to keep hold of it. If you want an anarchist utopia, you may have problems keeping it for any length of time unless you are fairly inaccessible or fairly martial.
So apparently the existing legacy technologies make reliable and scalable systems? A 'fad' technology that works is better than old technology that doesn't.
This seems like a great opportunity for someone to develop an open source high-availability, fault-tolerant 911 dispatch system. Being able to get first responders where they're needed as fast as possible would provide a tremendous public good.
This seems like a great opportunity for someone to develop an open source high-availability, fault-tolerant 911 dispatch system.
You're right! And then we can make a one-size-fits-all Retail Bank program that every bank will use. And a one-size-fits-all Policy Administration System for insurance companies. Once we've solved those problems we can move on to a one-size-fits-all Dispatch system for every company that has employees in the field.
Then we can wake up and realize that there's a reason why this hasn't happened in the last 50 years. Any system can accommodate maybe 80% of a particular niche's needs. But I'll be damned if you can get business users to agree on that final 20%. One says "black", the other says "white" and then you have some oddball that says, "potato."
And then there's the whole continent of minefields in regard to disparate devices, patents, trademarks, proprietary system interfaces, etc.
If you do take this on, best of luck to you - but don't quit your day job.
That's not how things work... You can't just create some system and see governments flock to you. There are years of consulting and specification requirements.
Linux is open source... Office Libre... Go convince a public office to switch to Ubuntu..
Saving money is often at the bottom of their focus.. and then you have competing interests releasing figures that show it is more costly to switch to free alternatives.
Yes, First they ignore you, then they laugh at you, then they fight you, then you win.
Sorry, I know it's trite, but there's some truth there. Being ignored for years is not necessarily a reason not to do something. In some cases it's the very best reason to do it.
[EDIT] - would those people down-voting please explain why?
Maybe you could partner with just one small government to start. Of all the municipalities in the world, you'd think at-least one would be open to the idea, even if in tandem.
Maybe it could be started by a person who works (or worked) at the dispatch center, and is also a programmer (there must be at-least one of those in the world too).
The point is that NYC laughing/ignoring/fighting you is not a good enough reason not to start. (and who's to say that NYC would do any of those things, given the current state of affairs)
I downvoted you because you asked people to explain why they were downvoting you. This adds nothing to the conversation (everyone who gets downvoted wants to know why, everyone who downvotes knows that the person who they are downvoting would want to know why) and is bad for HN (talking about the moderation system distracts from actual on-topic conversation, and leads to gaming of the moderation system - indeed, I would say even that line can be gaming the moderation system, as it can lead people to reconsider whether to downvote).
And for a tiny fraction of $88 million. There's something about throwing such a large amount of money at a project that causes it to bloat into failure.
The London Ambulance Service had similar problems when switching to a different system back in 1992.
It's actually an incredibly well studied case of an IT project failure. It's studied as a textbook case in just about every IT project management course in the UK.
Interesting that NYC is repeating what sounds like exactly the same mistakes.
This sort of thing doesn't surprise me. Anecdotally, I've heard more than a few people bemoan a new POS system. Because I write software for a living, I usually prod them for details.
Even though their old system was some DOS based thing from 1991, it worked. It was fast and simple. Now they have some new, enterprisey, WPF + EF behemoth that is buggy and slow.
Sure, new software will almost always have more bugs than the system it is replacing; the old system, whatever its flaws (perceived or actual), was battle-hardened. It was probably written in a time when speed was more important than the technology stack it used.
These people don't care so much about a nicer looking interface. They need something that works and stays out of their way so that they can do their job.
Are there any examples of complex, large-scale government IT projects that have succeeded, come in on cost and time and worked better than what they replaced?
I can only think of one example: the UK's Government Digital Service (who built www.gov.uk). Admittedly not a mission-critical project that saves lives, but they actually managed to bring the hundreds of different government websites under one site, re-wrote and re-rorganised the content so it was easier to find and read, and actually improved considerably on what was in place before.
It's an exception rather than the rule though to most government or local council projects in the UK. Any other examples of success? (There must be some)
Funny how that works! A company that can't even get a website right probably has a few things wrong. First it's likely they take their customers for granted and don't care. However, it's likely the guys at the top actually think they have a good website. Companies like that make every decision by committee and as long as the boxes are checked, the 'product' works.
I was surprised to see that it was the same Integraph. Their NT graphics workstations were big competition to SGI in the late 90's, when SGI was losing steam to the PC market.
It would be fun to have hackathon to design a better 911 system - I'd bet 24 hours worth of work would produce some amazing stuff (probably not super-ultra reliable production ready, but some cool stuff that could save lives)
Wow. It's a new case study on how process risk equals 1970's-style failures: Big Design Upfront and failure to manage risk. Gambling with people's lives instead of ironing the bugs out in a mock environment, going live on a limited basis (smaller center/group first) and not letting the consultants drive the product.
The operators should've had final say on production worthiness. The arrogance of top-down "solutions" that don't work.
Other observation: results are often inversely proportional to budget, up to a point. Spending instead of thinking is not a solution.
$9/hr during training (seriously!), $33K at your first assignment.
Pay follows a "public school teacher" trajectory, or at least like the trejectory in my district. Such that you'll start your career eating ramen and living in your car, but by retirement you'll be hauling in $100K or so.
The FAA workforce is gray. No other way to say it. So you tend to see ridiculous averages like your local TRACON might pay on average $80K. However as a noob you'd only get $30K or so... you need 30 years in to get rich.
Most pilots get to know ATC people for obvious professional reasons. You can probably get interesting anecdotes from a private pilot.
Also more than a few ATC personnel ARE private pilots. Helps to understand the other side of the radio, so to speak.
I am told there is a narrow window where you're "too old" to be an ATC and get mandatory retirement, yet still young enough to be a CFI, and this is where gray haired flight instructors come from. I do not know if this is true or not. In other words mandatory retirement age for CFI > ATC but not by much.
Nonsense. This has nothing to do with Windows and everything to do with poorly-implemented software running on it.
I have seen Java-based systems running on Linux that stall/hang and run agonizingly slowly under what should not be demanding levels of use. Bloated, heavyweight software stacks exist on all platforms, and they are favorites in government contractor architectures.
Twitter can handle north of 4k tweets per second on virtually entirely open source software and these morons can't record phone calls. If this is not a glaring example of government mismanagement and waste I don't know what is. Where are the criminal indictments?
Comparing twitter to a 911 call center is like comparing a monochrome display to a 4k display
911 call centers have to interface with many different computer systems (for example the National Criminal Information center, department of motor vehicles, FBI CJIS, in house databases, court records systems, mapping, computer aided dispatch, weather information, etc.
Many of these systems (the NCIC, and CJIS have very strict requirements on what can and can not connect to their systems, and on top of that who can use a terminal that connects to their systems).
Its not an easy task. However comparing it to twitter is just ... wow
While not entirely disagreeing, I would ask whether there's room to question any of those requirements. WHY does the 911 call center need to interface to all those systems. WHY do I need an interface to the FBI or court records in order to respond to a call from a woman whose husband is having a heart attack in the living room?
Tight coupling of many separate systems is a classic anti-pattern in software.
Errr, but this one doesn't sound like it's quite that bad, from the OP it's a "find the address and dispatch to fire, police or EMS" system. Only the police system, which is I assume a bigger part of this 2 billion total "upgrade" really needs most of what you've outlined. Doesn't sound like there's anything particularly sensitive in this system, although it's certainly harder than Twitter, which thezach in this subthread tells us is still presenting Fail Whales.
911 doesn't need to interface with the NCIS. That isn't helpful for a 911 dispatch. The police systems need that access but this whole discussion isn't about the police system, it's about triaging calls to the appropriate responding agency. FBI data is irrelevant. The FBI is never dispatched via 911, FBI dispatch would happen only after the local agencies deemed it necessary (at least in this context.) 911 operators do not have access to the NCIS.
Its NCIC... not NCIS... Gibbs would be slapping you on the back of the head right now.
And yes 911 operators do have access to NCIC and FBI CJIS. They often are running people involved in incidents when cars are en route. Knowing who they are dealing with often dictates the number of cars that respond, the officers tactics when responding, etc. Both of those databases have nationwide info that states share on warrants, convictions, drivers licenses, etc
while Twitter's infrastructure is impressive[1] (though, not because of the tweets/second number you've cited), that is an entirely different problem set with different challenges and no regulatory/integration concerns.
Hate to break it to you, but that sounds more than adequate. I would choose differently if given the choice, but it's not a valid excuse for pausing 3 minutes to generate a job number by quite a long shot.
I actually agree, I wasn't making a good/bad comment, just speculating. Very few government contractors would bid a linux or open-source-software-based system.
I do agree, a well designed and implemented system running on Oracle and Windows can certainly handle something like 911 dispatch and do it well.
For those who don't understand my amazement: they build aircraft carriers for this amount of money! Don't tell me it's the same amount of engineering, knowledge and effort in building an emergency response system?
You (and I) truly believe that the telcos have proof positive location on our landlines and cells, and this data is collected whole hog by the govt, but BUT somehow they cant locate a caller having a heart attack on long island.
You know, the govt has the systems it deserves, and I really dont care to hear whining about pencil and paper.
It takes no longer to write down on pencil and paper a 911 call and theres a mandatory handoff anyway.
This has nothing to do with the folks working the phones not being up to par. They are. It has everything to do with big telcos and govt contractors holding taxpayers hostage.
We dont need them. Use the pencil and paper, make a phone call to the next dept. That takes the same amount of time as data entry.
Grow up, be a human being and stop pretending that life and death situations are some variant of farmville.
Have you ever been in a 911 call center? Many, many, many things are computer driven, not the least of which is interfacing with the various agencies one may have to deal with on a per call basis. Then there's "whose jurisdiction is this"? Kind of hard to determine with pen and paper. Who's dispatched to what? Who's covering which stations now? How many calls are in the queue? The list goes on and on.
Having to record things on pen and paper is hardly sensationalism. It's more or less a worse case scenario in the 911 dispatching arena...
And handwriting can be (a) drastically slower and (b) less legible (therefore more prone to mistakes or lost information). In a 911 environment, either of these alone can cost lives. Throw in all of gregd's additional variables, and it would be a nightmare of the sort described in the article.
Oh boy. In 1993 handwriting wasn't the only option. And we're talking about 911 dispatching, not just police. Fire, medics, police, dog catchers, tow trucks etc., are all involved now.
Successful management of any endeavor requires prioritization. If dog catchers and tow trucks are in the same bullet list as ambulances, they're doing something wrong.
In most cases an ambulance is required much more urgently than a tow truck. The cops can call the towing company they prefer, once they're at the scene. There's no need to complicate the 911 system with towing, unless it's important that kickbacks from towing companies be paid at a high level rather than directly to individual cops.
Cops at the scene should not be looking up phone numbers for this week's roster of towing companies. That is handled by the central police dispatcher. Who also dispatches SWAT, bomb disposal, fire, hazmat, linemen for downed electric wires, animal control for welfare checks of sick people who have pets, etc. 911 is just one part of an integrated emergency response system.
Even out here in the rural hinterlands, 911 operators and police dispatchers are different groups. The cops can call the tow truck however the hell they want. The 911 operators still shouldn't care about the dogcatcher.
* I build big call centers for a living
* I'm a really big Telecom Nerd
You might be wondering why something as important as 911 is not automated or accessible via any other mechanism; why is it that the most critical service in the country runs on a system that hasn't materially changed since 1984?
For one, the Public Switched Telephone Network (PSTN) is extremely reliable, authentic because it's logically addressed, and ubiquitous. The only real flaw is the addressing, which, because of Caller ID, can be faked, but that's another story (See Wikipedia on Swatting).
So, there's a huge movement to reshape 911 service to take advantage of modern things like the internet, SMS, GPS and all different kinds of connectivity. The way 911 works right now is sort of a hodgepodge all over the place depending on which vendor the municipality has selected for the Public Service Access Point. There is not one standardized 911 system across the country, which may or may not be a good thing depending on your perspective.
So everyone can agree fixing 911 is a great idea, BUT why hasn't it happened already? I can point to a couple potential reasons:
1) It's a really big contract; supposing PSAPs get standardized across the board, we're talking about many millions, potentially billions of dollars. That means lobbying and negotiation and time (oh by the way Sprint is currently leading the charge but I find it unlikely that the US will let a foreign-owned conglomerate manage their backbone emergency services).
2) The current vendors see nothing wrong with the existing solution, and have projects like what has happened in New York to stand against the tide of change. Their service contracts provide a big incentive to keep those boxes in those cities.
So I think there's a lot of reasons why 911 is as bad as it is, most of it isn't technical, but political, as with most old ridiculous infrastructures.
It's amazing how not planning for obsolescence almost automatically results in political conflict down the road.