Is Armchair Quarterbacking of healthcare.gov really good for the site?

[berlinbrown]

I have seen a lot of analysis of healthcare.gov. Some are saying it needs to be written. Some are just looking at little snippets of JavaScript and saying that the tech leaders need to be brought in to fix it. Others want to fix it over github or through open source.

The site was released this month, publicly. Should we really analyze the site that quickly and only take the pessimistic view. I know the site is a political football already, let's not make snap decisions before even the month is up.

* I am curious how analysts got a hold of the code? Is it closed source or open source. If auditors were allowed in to look at the code, then why not analyze the code over the life of the project and critically analyze it early on as opposed to weeks after release.

* If you don't have access to the full repository of source, then how can you call the project a failure? Who looks at snippets of JavaScript and then declare the project horrible.

* And why just this site? I have heard of CGI and SAIC and QSSI. There are many enterprise software firms out there that work with government contracts. Why is suddenly this one abyssal failure?

* I feel that the customers of the site (whoever launched the contract with CGI/QSSI) should have the last input on how the site should be enhanced or fixed. Random people on the Internet throwing out there suggestions can't be healthy. And I even wonder if Obama suggesting that he would talk with industry leaders was nothing more than politics. The contract was not between "industry leaders" and the government. Apparently the contract was awarded to CGI. Why get the entire software community involved?

* I remember, people were critical of Paul Graham's code and site, news.ycombinator.com. It seems to have survived, I wonder how many of those suggestions were really taking into consideration.

[ScottWhigham]

If you don't have access to the full repository of source, then how can you call the project a failure?

Because the people who wanted to use it, can't. Because it gives out the wrong quotes. Because whenever someone is able to get through and create an account, often their account details are wrong (their daughter is now their wife, etc). The whole idea that you've asked this question is mind boggling to me.

Random people on the Internet throwing out there suggestions can't be healthy.

This is really just a rant, isn't it? Fine - rant on. But come on - to act as though people won't talk around the water cooler on Monday morning is just head-in-the-sand...

[senthilnayagam]

if 90-97% people are unable to complete a signup process completely what do we call it?

it has been over 40 years since US reached moon, without modern computers and simulations

what you see is sheer incompetence and arrogance,

I dare the administration to release the numbers, they won't, but the president can talk about software glitches what apple iPhone had

[hga]

I'd say it depends on the level of speculation involved in the speculation. Javascript analysis I can't particularly speak to, seeing as how I haven't done that sort of thing since 1997. On the other hand, which someone tells us he observed a GET URL with variable substitution missing, or we hear of all sorts of 404s that simply should be there, we've got some meat to judge it by.

Me, I mostly stick to process and management, noting that the HHS's CMS is the integrator and integration tester, and that the overall technical management has been done by in/unexperienced non-specialists from the White House on down to CMS. This helps to explain a lot, like how per the NYT 7 major change orders were made in the last 10 months, and changes were ordered through the week before launch. As well as integration testing beginning no sooner than 2 weeks before launch, and CMS doing a test a week before launch simulating 200 simultaneous logins, which failed hard.

And nonetheless the system was launched knowing it couldn't work. And after not listening to those raising red flags for months and months, and evidence from an insider they knew they were in trouble in March ("... I hope it's not a Third World experience.")

Since then, CMS panicked 3 days after launch, proposing to CGI Federal that QSSI's identity module (based on Oracle's known to work, if gnarly, package for this) be punting a redone from scratch. CMS eventually decided that would take more time, but we can see a lot of management bandwidth at CGI Federal and QSSI was taken by this.

Then we hear that since launch, the White House and CMS have been intensely working on a remediation plan, hopefully to be finished today. Oh, yeah, part of this is said to be a "tech 'surge'", ignoring what us specialists learned in reading The Mythical Man Month plus experience.

As a student of project management and project success and failure, this all fits together and sufficiently explains what happens, and e.g. why we shouldn't blame CGI Federal, there's no possible way they could have succeeded.

Looking forward, people like me can see that absent a wholesale replacement of those in charge, or their learning on the job enough of this stuff (we all learned a large fraction of this sort of thing in real projects, after all), we can predict the site's problems aren't going to fixed in the foreseeable future, let along "soon".

The latter is critical, because there's one absolutely hard deadline facing a lot of people who have major medical plans, which are now outlawed, although according to an account I just read 3 out of the 19 million outstanding will be grandfathered. That is, they are perforce having them canceled in the 100,000s of thousands per news reports, they will be without insurance Jan 1 absent getting an official Obamacare one.

Unfortunately there's now just one basic gold-plated plan, with 10 "essential" things added like coverage of your children through age 26. A lot of people will simply not be able to afford these new plans without subsidies, and you can't get those without people using Healthcare.gov. That includes state exchanges, they have to use the federal one for subsidy calculations, or "manual" (paper/phone mediated) ones, which also have to use components of the Federal system. (Don't ask how if you don't have a credit record, your lack of a FICO score appears to be a total nightmare, there's supposed to be a process to deal with this, but how to get it started is (generally?) not known/knowable).

These people need good guidance, which they're not getting from the powers that be, on how to deal with this critical problem before the effective deadline in less than 2 months (you've got to be signed up by Dec 15th? to be sure of getting coverage Jan 1). Predictions that by the end of the year more people will have lost their medical insurance than gain it because of Obamacare are credible.

Looking further, failure to fix the system so that the 7 million minimum needed for financial viability can enroll will tell is if a adverse selection deadly cycle is happening. It's one thing to imagine that 3 million young and healthy will out of the goodness of their hearts by expensive insurance they don't need to subsidize those 50-64 (the fine/penalty is too small to force them), it's another to hope they can when the site they need to do this doesn't really work unless they put in a totally unreasonable amount of effort.

To finish by addressing your bulleted points, the 1st was covered above, the 2nd is ludicrous given the outside the system observables that it's failing hard, underlined by the Administration's total refusal to tell us how many have completed the process, the 3rd is covered above, and the 4th is not to the point. Outside of occasional and easily recoverable problems, news.ycombinator.com works. The "glitches" are truly glitches, not total failures of function. We get "can't complete you simple transaction" ones, not the sort being reliably reported for Healthcare.gov. I don't need to look at the code to know however it can be criticized, it gets the job done, which Healthcare.gov manifestly doesn't.

Or let me ask you this: you're not a newbie, you've been on this site for a couple of months longer than I, but how long have you been a programmer studying and doing "software engineering"? I have since 1977, and have been getting paid for it since 1980.