The root problem is not government. The same thing happens in the private sector. Large corporations routinely spend gazillions on software projects that end up working poorly or not at all. Everyone who has worked in Corporate America knows this all too well.
The root problem is not bureaucracy either. Companies like Amazon, Google, and Microsoft have fairly large bureaucracies yet they routinely successfully complete massive software projects.
The root problem is not "vested interests with complex requirements" either. When coders with large-scale-project experience were put in charge of revamping Obamacare's healthcare.gov, they figured out how to cope with a wide range of demands from a wide range of constituencies in an efficient manner, and got the job done: http://www.thewire.com/politics/2014/02/how-silicon-valley-s...
The root problem, I think, is that the people who are put in charge of these massive software projects -- typically, MBAs or lawyers by training -- often don't know anything about how to develop software at scale. It's like asking people who don't understand rockets to lead development of a new kind of rocket that has never been built before.
Edit: added more detail and context to the third paragraph.
Having worked in large companies, small startups, and government, government is orders of magnitude the worst offender when it comes to simply wasting money and getting nothing done.
The main difference between small and large companies was it simply took longer for people who got nothing done to be let go. In the worst cases for companies, money stops coming in the door and cuts are forced.
Contrast the above with the government where it sometimes literally takes an act of congress to fire someone. If the populace is unhappy with how the government is spending its tax dollars they cannot simply stop paying taxes and go somewhere else.
I've done corporate, startup, government (higher ed).
Another difference with government is the intractable complexity.
This plays to the difference between digitization and automation. Automation is the removal of human judgement. It requires simplification of the rules and processes.
My local government revamped their payroll system. The first effort tried to digitize existing rules, and failed utterly. The second effort reworked the payroll rules (automation), allowing them to use software closer to off the shelf, and succeeded.
I consider the projects I worked on for higher ed to be complete failures. Publishing course catalogs, registering for classes, auditing degrees, etc. Conceptually simple stuff. Alas, management tried to implement every single psychotic rule and mutant edge case for each and every stake holder, preserving paper-based processes over 100 years old. Such fun.
My work for a telco wasn't any less nutty making. There's a reason your phone bill is too high: bureaucracy. Their internal IT is terrible. They can't even provide conf call services or roaming wifi to themselves. I sometimes fantasize that Apple or Google will introduce ubiquitous flat rate service, gut the dinosaurs, and making another gagillion dollars just thru operational simplicity.
> Automation is the removal of human judgement. It requires simplification of the rules and processes.
My impression, is that government projects often go like this: 1) We want to digitize and make some process X more efficient, 2) Oh, now that we're automating we can avoid "errors" by removing case-by-case handling (X is of course now set up for individual evaluation by a professional bureaucrat). The result is that not only do you get feature/scope creep (handle all the things for all the cases, or declare failure), if you succeed, the system ends up being much worse (it now follows outdated written documentation to the letter, doesn't allow for any leeway).
The obvious way around this, is make a system that automates the easy 80% of the cases. Allow the system to give good information to, the now much fewer, bureaucrats, and allow them to handle the 20% on a case-by-case basis.
After a year, evaluate, and reduce the 20% to another 80/20.
Give up the idea that you should automate policy. It's completely insane, even if it might look good on paper.
It's like with accounting systems: you should always have a way for a clerk to enter in and account for unforeseen things in a way that a) accommodates the system ("-500 towels missing/destroyed") along with a comment-field to elaborate, and to facilitate revision ("Bus crashed through the window, gave away 500 towels for use as bandages").
> preserving paper-based processes over 100 years old. Such fun
This is probably the most important part - every digitization effort I've seen fail, failed because they tried to digitize the existing process, rather than look at what information needed to be collected and creating a new digital process to collect that information.
On the upside when a gov project works they are often willing to keep it for a very long time and killing a project does not mean the original need is not still there. So, often really old government boondoggles are often 2 or 3 separate projects which keep the name after the first one fails.
> cannot simply stop paying taxes and go somewhere else
Plenty of people leave the US every year. It's generally really easy to do. At a local level choosing a city or state based on tax rate is even easer.
ED: Unless you meant the moving was the hard part, which seems reasonable.
Persons intending to renounce U.S. citizenship should be aware that, unless they already possess a foreign nationality, they may be rendered stateless and, thus, lack the protection of any government.
There are also a lot of exceptions to that IRS rule, depending on the type of income.
PS: As to renouncing Citizenship; Social Security is an example of long term obligation that transcends where your living in a single year. If your willing to give up Social Security by renouncing Citizenship then things get simple.
People really complain about paying taxes while living outside the country. IMO it's a really complex situation, but the tradeoffs are surprisingly reasonable if you consider it in the long term from both the individual and state level in light of various free rider problems.
I have experience of developing large software systems for both government and corporations, in the UK.
Large IT systems fail in many ways, small and large and I don't think it's possible to isolate a single root cause but I do have one observation which I think is often overlooked.
When a company like Amazon or Google release a new service or a new product, they generally have the luxury of choosing which customer requirements they want to meet. The more customers they satisfy, the more customers they can hope to win.
This freedom means that these companies can follow the common trajectory of releasing a simple realization of their product which is then honed by iteration. But companies contracted to produce government and corporate systems rarely have this freedom in my experience, but for different reasons.
In the case of government systems it is politically (and often morally) unacceptable to exclude specific classes of end users from a system; the requirement to make a system available to nearly 100% of possible users while at the same time catering for the labyrinthine complexities of local and national government is simply very difficult.
In corporate environments, there is a similar pressure to meet 100% of customer needs, but in this case the users are often staff and though the problem is similar at a technical level, the source of the pressure is different:
Google is a large corporation, and when it releases a product like Gmail, users can choose to use it, or not: they can't singly demand changes, so Google has the power to stand up to users and, in short, be a good custodian of the user experience.
When developing software for corporate customers, this relationship is inverted, and very often, every wrinkle in every corporate business process is expected to be modelled in the software. As a contractor, you know you could produce better software for the corporation, if they were willing to adapt some of their processes in the same way customer adapt their behaviour to using Gmail, but they have the power.
The result is the same in both cases: excess complexity.
"Management" is a secondary skill in the sense that it only makes any sense if one also has a primary skill. An engineer might learn to manage and thus become an engineering manager. A baker might learn to manage and might manage a baking company or a restaurant/cafe. But only knowing "management" alone is absurd and vapid. I truly cannot comprehend how anyone could purport to manage something they do not understand at the ground level.
To some extent I blame academia for pushing the idea that one can get an MBA and manage things without understanding what one is managing.
Programming is particularly bad. There are no physical analogies for it. It's so alien to ordinary human experience that asking someone with no experience in it to manage it is like asking a blind person to direct a silent film. Actually the latter is far more likely to be successful. I can't actually think of an analogy right now that suitably captures just how inept a non-developer is at managing developers.
From the link:
"Members of the "apparat" were frequently transferred between different areas of responsibility, usually with little or no actual training for their new areas of responsibility. Thus, the term apparatchik, or "agent of the apparatus" was usually the best possible description of the person's profession and occupation."
"It is often considered a derogatory term, with negative connotations in terms of the quality, competence, and attitude of a person thus described."
MBAs as an empty suit who manages what they don't understand is as much of an useless stereotype as programmer as a code monkey who just creates what you tell him to. You have the same ignorance about them that you accuse them of having about you.
I also don't believe programming is as alien as you think it is.
I will grant you that most of the diploma mill MBA programs are worthless. But the MBA's going to work in F500 companies aren't just idiots.
> I will grant you that most of the diploma mill MBA programs are worthless. But the MBA's going to work in F500 companies aren't just idiots.
Are you saying that there are some great MBA programs, that will produce an MBA that adds value without other experience, or are you saying that the MBAs going to work in F500 companies have experience beyond an MBA diploma?
If it's the latter, isn't that pretty much what parent claimed? If it's the former, is that because those MBA programs are cross-disciplinary with real project work loads?
I meant the former, but the later is also true. Though even great the MBA programs only take students with good work experience. However, it's not impossible to change industries--e.x. bankers going into consulting. It's hard to tell how much value the MBA actually adds. It's hard to tell if they are just selecting the best and brightest, or if they actually teach something worthwhile. I suspect it's a mixture of both.
I took a couple MBA courses at Kellogg when I was at Northwestern in another grad program. The ones I took weren't real project work loads, but the courses delved into case studies of what companies did wrong and right during critical points in their history. Students would use real world experiences from their past to explain things. I took an IP asset management class with one of guys who sued Zuckerberg for allegedly ripping off HarvardConnect to make Facebook. Unfortunately he never talked about it. I was very curious.
Another large value of an elite MBA is the networking. How many billions of VC bucks have been dolled out to MBA grads at Stanford by people they met there?
In fact, the whole idea of an MBA program is somewhat similar to what YCombinator does.
> To some extent I blame academia for pushing the idea that one can get an MBA and manage things without understanding what one is managing.
Where is this pushed? The advice I've seen is "go work in industry for a few years and get practical experience before getting an MBA."
The ads on the subway for MBA programs that I've seen are usually in the terms you describe of going from a baker to manager of a bakery.
> The MBA (who has no other expertise) is capitalism's answer to the Soviet "apparatchik."
I'm an MBA. Are you suggesting that I have no other expertise? Do you think that we undergo some sort of specialised lobotomy at business school to remove the knowledge and experience we accumulated before enroling in business school, and to prevent us from learning anything thereafter?
I think you slightly misread what I meant. I was referring to those who only get an MBA and then think they can manage anything even if they have no other expertise. Maybe your mileage has varied, but I've never seen anything but disasters in the real world when that happens. You can't manage something you don't understand.
By placing "who has no other expertise" in paranthese, it reads like a general assertion about "the MBA", rather than a qualifier intended to indicate that it is referring to a subset of MBAs.
I'll allow it this time. But I'm keeping my eye on you. ;-)
The root problem is incentives. When a failed project doesn't mean any loss (financial or otherwise), there is no market correction. That, and no one has been able to actually defined what a failure is. Is 10 years too long? Who knows...there are no experts who can plan for that. [0]
When healthcare.gov was turned around by a select set of motivated individuals, it only happened because the people who were involved had a high degree of motivation. IMHO I believe they were motivated by eliminating personal frustration (or having empathy for those who are frustrated) and the upside of a highly visible software project (with none of the downside...it was already failing). Money wasn't a focal point here.
But when individual private companies screw up, other companies do what they tried to do better, and in the end the problem gets solved.
> The root problem, I think, is that the people who are put in charge of these massive software projects -- typically, MBAs or lawyers by training -- often don't know anything about how to develop software at scale
Sounds like a... branch problem. What is the root cause of government projects being run by incompetents?
"What is the root cause of government projects being run by incompetents?"
The incentives tend not to reward competence at achieving a task very well. They tend to reward political climbing. They don't necessarily preclude competence at a task, but energy expended competently performing a task is energy not expended playing politics, which means it tends to get outcompeted.
This is not, strictly speaking, a characteristic of "government". Large corporations that get sufficiently rich and powerful that they end up detached from market discipline develop the same problems.
Security, wealth, and comfort are even more dangerous to large organizations than they are to individuals.
In the early 2000s I worked for a big government contract company that had a contract with the CDC to make a website with.. wait for it... forms. Yes it was basically big complex, single page forms that wrote data into a... wait for it. Oracle database. These were the days of J2EE. They had about 30 programmers on the case. There were the architects, a lead architect, and many programmers. The contract was for about 100 million USD.
Their interview process was: "can you write computer code?.. OK great, you are hired. Come on in." I should have known something may be amiss..
The dev server took about 5 minutes to start on a souped up computer. That and several static variables hiding up high in an uber base class. These variables were of course designed to hold per session data. I tried submitting a form twice simultaneously. This crashed the server.
I left that and years later the project was I believe cancelled.
I wonder why the government does't just use a forms as a service thing?
> The root problem is not government. The same thing happens in the private sector. Large corporations routinely spend gazillions on software projects that end up working poorly or not at all. Everyone who has worked in Corporate America knows this all too well.
> The root problem is not bureaucracy either. Companies like Amazon, Google, and Microsoft have fairly large bureaucracies yet they routinely successfully complete massive software projects.
>...The root problem, I think, is that the people who are put in charge of these massive software projects -- typically, MBAs or lawyers by training -- often don't know anything about how to develop software at scale.
I don't think these explanations are mutually exclusive. If you're right about the root problem being the technical expertise of leaders, then one can still argue that the Amazons of the world slowly rise to the top and displace the K-marts, but an incompetent government can persist indefinitely because it has a monopoly on governance.
There isn't really a monopoly on governance, though. On the short term there is political alternance, on the long term there is the rise and fall of states.
Of course, the first one would matter a lot more if: a) people voted based on previous performance, not vague future promises; b) there were more than two options in competition.
As for the second. The second works quite well, but not a time scale that yet cares about well-designed websites... fail to figure out say steel alloys and gunpowder, though, and you'll see this competition in action... ;)
The root of the problem is really that instead of trying to figure out how to digitize immigration forms, we should be figuring out how to fill capacity within the country and not simply kick poor and underserved communities to the ditch and just cream people from other countries and societies where we are cherry picking off the best.
I get it, that won't be a popular sentiment here because the hidden racism backed into immigration to fill holes created by incompetence and ignorance of long term problems by filling them with short term solutions is still quite popular, let alone not even realized. It's kind of like simply going to the title loan place to leverage the future to pay back the credit card of bad past choices.
Digitizing the immigration forms is really more of a sad reminder of how racist our country is that we refuse to provide for our own people, while we are more than willing to deprive source countries of success by making people build industries and technologies in their own country.
I know, I'm all wrong ... screw those mostly black poor kids in poor neighborhoods and screw those other countries, America needs to undermine their success in order to dominate.
Always with the predictable knee-jerk responses. I'm neither a racist nor a xenophobe. I don't hate people because of their origins or race, nor to I not like foreigners. You are surely not going to like hearing ths, but immigrants are essentially moochers on one hand, and traitors on the other. Moochers on both ends because if they were educated for essentially free in, e.g., Europe and come to the USA they both mooched off their source country and then they come to the USA to mooch off what the USA and Americans built. Traitors because they mooched off their country to get an education and are now putting their labor, knowledge, and skill to use for a different country or society instead of building up the country they came from and still hold on to even when they are in the USA.
The humongous forest that pro immigrant people can't see for all the trees is that immigration is a core driver and reason for many of not only America's problems, but also the lack of source countries from gaining any traction. All the way from sapping out pressure for social change by motivated people leaving for places where they can mooch over the fight that was already won by others, to extracting the skill and knowledge to do the thing that the do from their society. What pro immigration types don't quite get is that immigration to the USA is like kryptonite or more like a witch that sucks the life force out of the source countries and societies.
You may and probably confuse what I said with xenophobia and hatred and are angry because I am not saying your favorite thing, but reality is that I am right, whether you or others realize it. I'm fine with that, but it needs to be said. The point is not about "go back to where you came from" it's "why didn't you put your energy, drive, knowledge, and skill towards your own country and society and instead went to make the rich, richer and the powerful more powerful" Don't complain about America's actions and policies around the world if you are going to support our life force driven economic, political, and military engine.
I have spent 10 years in American Immigration system (became permanent resident 2 years ago) and it goes without saying that the immigration process has been the most frustrating part of my stay here.
Here's something funny. In last few years, all the reputed lawyers (Murthy, Ron Gotcher and others) have setup an online portal where you, as a client, fill in everything online. This includes scanning various documents and uploading them. The online portal provides a great workflow which minimizes data entry and avoids data errors. The same portal allows you to chat with your lawyer/paralegal if you have any questions. At the end of this process, the lawyer firm will generate correctly filled in immigration documents which they will print and send it to you. You sign them and send it back so that they can forward it to USCIS! I know it's not practical or possible, but when I was working through this portal, I wished USCIS would just buy this system so that I don't have to pay thousands of dollars to laywers.
The amount of repetitive paperwork and needless bureaucratic steps for immigration-related matters is utterly ridiculous. The closest approximation I can make to dealing with immigration-related paperwork is filing taxes without the aid of software and being forced to complete and mail in one section (and have it approved) before moving onto the next.
"The most familiar quotations are the most likely to be misquoted...Some have settled down to false versions that have obscured the true ones. They have passed over from literature into speech."
-- Carl Van Doren (maybe)
As someone who has gone through the IR1/CR1 process, yeah the entire thing is fucked.
- The forms themselves are poorly designed (needlessly complex, come with misleading/incomplete instructions, tons of "gotchas," etc). Between me and my wife we have three degrees and a masters, are native English speakers, yet we still had to Google to figure things out.
- Instead of having one form, they have three, and you cannot submit the latter two until the proceeding one is accepted (even if they're completely independant). They don't actually check the proceeding forms until the visa is complete (we know that because they flagged something months later).
- Paying fees is a nightmare.
- Even their embassies know how terrible it all is. They often hold workshops for people going through the process and are pretty frank about the bureaucracy. They describe the NVC as a "blackhole" (where things often get lost/delayed/mistakenly rejected).
Digitalisation is useful but the US's house is in such disarray that it seems like the tip of the iceberg. I legitimately believe that the US uses bureaucracy and delays as a way to discourage people from applying for visas and or push visas into the next calendar year. It is the only explanation that makes sense.
To be honest the entire US visa process could be done, end to end, in one day if they wished (ex. background check). And in fact US embassies in exceptional circumstances actually do accomplish the entire visa in 24-48 hours (yes, even for immigration visa).
I spoke with some USDS/18F folks a few months back, and I know they're working with USCIS to try and remedy the failures of contracting this all out to IBM. I'm hopeful things will get better in the next few years.
Creating USDS/18F may be one of the most under appreciated, impactful changes of the Obama administration in the years and decades to come. Keep up the good work!
I've always found the biggest joke in these forms, among other federal forms, to be the OMB "paperwork reduction" statements. [1] Even airport landing customs cards have these.
Why these make no sense:
- They take up a good chunk of a page and consume both paper and the reader's time.
- These forms aren't exactly optional, so I am going to spend time filling them out anyway. They would be easier to fill out if they were electronic, and if I wasn't repeating the same entries for the Nth time. The long disclosure adds nothing to usability and is only taunting.
"Paperwork Reduction Act Statement: A federal agency may not conduct or sponsor, and a person is not required to
respond to, nor shall a person be subject to a penalty for failure to comply with a collection of information subject to the
requirements of the Paperwork Reduction Act unless that collection of information displays a current valid OMB Control Number.
The OMB Control Number for this information collection is <number> Public reporting for this collection of information is
estimated to be approximately <n> minutes per response, including the time for reviewing instructions, completing and reviewing
the collection of information. All responses to this collection of information are mandated by <statute>. Comments
concerning the accuracy of this burden and suggestions for reducing the burden should be directed to the <agency> at: <addr>."
I printed a student loan form and the last page that printed contained only the Paperwork Reduction Act notice. I have to say, it was beautifully Orwellian.
I think they've started getting rid of custom cards. My folks visited from Europe a few weeks ago and told me they didn't have to fill in anything this time, everything was electronic.
But yeah, otherwise I agree with your statement. When I filled for my green card extension I had to put my application in a box, not an envelop, a box.
The worse problem is it also makes it difficult to change forms - or at least it creates the perception forms can't be reduced/simplified without 9+ months of process. *I work at USDS/SSA
It seems to me the the same effort that was applied to the HealthCare.gov should be applied here.[1]
It is beyond...well I don't know...beyond something that our federal government proves itself time and time again so incompetent at this sort of thing.
Yes, I understand in Washington that the political process drives everything and that contractors have turned milking-the-cow into an art form, but certain important issues, such as immigration reform, should (pardon me) trump this stasis.
I remember taking my wife for her medical exam as part of her green card process. What a joke! The doctor--who he has to select as someone on a small approved list--was basically retired and using his position on that list as a way to make extra money. I don't think he did a thing to examine her. He just chatted and told us stories about when he was fighting in Korea and accidentally stepped in a kimchi pot. Very nice fellow, but clearly just milking the process.
Had the same experience during my visit. Some older Russian doctor who chatted with me for 15 minutes, telling me that America was ok, that green card medical visit were pretty much all she was doing now, 3 half-day a week and that was plenty for her to make good money. She pocketed the $600 and gave me the sealed envelop.
I have no problem with not passing any cost to the American taxpayers (and even that could be debatable, yes I'm an immigrant but I've never been unemployed and I believe I create wealth so it's a full net win for the country), but the system is really hostile and dysfunctional.
Honestly, I don't care if you create wealth for this country. Having a diverse population to interact with and bring their point of view to things is a net benefit to the whole society instead of cultural homogeneity.
We had a similarly absurd experience with fingerprints.
Would you believe that the NY State Police aren't a certified fingerprinting service for INS? (or at least they weren't 20 years ago)
My wife had her prints done by the state police, but Immigration rejected them on that basis. Instead, we had to drive an hour to a camera shop that had the appropriate certification, where the clerk - an Indian immigrant himself, by appearance - took her fingerprints.
It boggles the mind that they'd prefer a clerk in a camera store rather than the state police. It seems clear that the goal isn't proper documentation or security or anything like that, but more along the lines of collecting certification fees and perpetuating the bureacracy.
My "doctor" from the list also was an old guy. He examined me in his living room with 4 cats wandering around and jumping on tables. Very bizarre situation.
The medical exams are a really odd requirement. You can live on an H1-B for 10 years, but they still have to check if you cough funny before granting a greencard.
* A permanent residency application based on my marriage.
* An application to remove conditions on said permanent residency
* An expedited naturalization application
We didn't bother getting a lawyer: the paperwork was easy. The problem isn't that the paperwork is hard. The problem is that there is so much paperwork to file.
My wife and I are going through the same thing for her Adjustment of status, she's here on an F-1 visa. It took us a little while to get the paperwork all in order(she was away for work for a few months and we needed to sort out a number of things to give us the highest chance of getting approved). For us, the hardest part was sorting through all the tiny little details and that a lot of the I-xxx forms kept expiring every couple of months during this past year. I got laid off and had to start a new job during the process, but then I had to wait a couple months to have a few paychecks to prove I can support her, but oh wait, the paychecks have my filing status as 'single' because HR screwed up, now I have to wait a few more months to get some checks with the correct tax filing status... Everything that I've read and seem makes it seem like any single incorrect detail can tank the entire application process, so we octuple-checked our forms.
I did the interviews 8 years ago (as part of a K-1 visa and then for permanent residency). It really wasn't a big deal. The guy barely looked through our supporting evidence and seemed more interesting in asking general questions about what my home country was like.
I think you're overstating it slightly. "Easy" is not the word I'd choose to use.
I too decided not to get a lawyer. It went fine. But it was not "easy" it was long, painful, and trivial to make an error (the paperwork is extremely poorly designed, we used online guides instead of the official instructions to avoid tons of gotchas, in particular with the financials).
People just need to be extremely careful without a lawyer. It isn't "easy" but it isn't so hard that you NEED a lawyer either (which could cost thousands).
True. It is definitely not easy. Another problem is the waiting periods these days is frustratingly long - so if you made a small mistake then it adds anywhere from a few weeks to a few months to remedy it, to an already lengthy process. This is why most people hire attorneys. The ironic thing is that attorneys make mistakes too - happened to me and a handful of people I know.
> The ironic thing is that attorneys make mistakes too - happened to me and a handful of people I know.
I commented on this above, but it nearly happened to me, too. We picked an immigration attorney to look over our paperwork, just to make sure we didn't make any glaring mistakes. While we did get some good suggestions, they missed something pretty big that we found on both the official government documentation and a number of forums, and had decided not to go with their services anymore. Luckily we caught it prior to submission.
I agree completely, having gone through this with my wife. It's easy to see how someone for whom English isn't their first language could have the best of intentions, yet still accidentally wind up on the wrong side of the law.
I have a few friends that have done it without lawyers and others that have.
I would not say that a lawyer is needed but if your case is not straight forward you better get you a lawyer or suffer the consequences(deportation with a multi year wait for your paper work to go through)
Care to elaborate? Family-based immigration (the overwhelming majority of cases) only requires 3 to 5 forms and a lawyer can't help much as you have to provide all the infos and documents yourself. And for employment-based immigration, most of the burden is on the employer.
Mine was employment based / EB1.
It was not the number of forms, but the structure that made it complicated. The final package we submitted was over 150 pages total for me and my wife. This is in part because the same information is required multiple times, since the different steps get reviewed by different departments.
I finished the EB-1 process this year. As thankful as I may be for how it worked out eventually (and knowing that most employment based petitions do not fall in this category), I still regularly show people the picture I snapped of the 6 pounds of documentation I had to prepare and send to support my case. Maybe they just did a good job convincing me of their worth, but I can't imagine doing this without lawyers.
Thanks! Just would like to precise this is the exceptional category for 'alien with exceptional abilities' [0] who need to prove their exceptional abilities; all other employment-based petitions are filed by employers.
I went through it 8 years ago (via a K-1 and then as part of the permanent resident process) and didn't find it that onerous. There was a lot of repetition and some hassle with having to travel to another city for an interview and waiting for an employment authorisation document, but at no point did we consider involving a lawyer. I suspect the type of visa and country of origin make a huge difference.
I think this is a bigger part of it. There are occasionally scandals that get media attention where someone got through the immigration system when they arguably shouldn't have (past involvement with organized crime, etc.), which produces public pressure to add another check to make sure it doesn't happen again. So the system is oriented around an adversarial sort of approach: we think there's a good chance you probably shouldn't be here, and you need to go through this complex process to prove otherwise. The system is designed to find dozens of reasons to exclude the applicant (since voters demand excluding people for those reasons), so there are a ton of pitfalls. Which is where the immigration lawyers then come in, since an individual immigrant with no idea how the system is designed or what it's looking for has a high likelihood of stumbling into one of those pitfalls.
I have heard that consulting companies tend to milk government contracts. I also believe that some people really don't like immigrants and very much want to limit the amount that can come in any way possible.
Also, what tends to happen with a business process is that it involves many people doing their specific part and handing it off to the next people. Properly automating a process can involve eliminating many jobs. And since people like to keep their jobs, the more streamlined redesigns for systems and processes are often not really considered as they would result in too many people being out of work.
That can be especially difficult when decisions are being made by the business organization that wants to protect its power.
Another issue is that business processes just naturally tend to become more and more complex over time. And its hard for people to accept that they need to be streamlined. Because business people generally don't understand user interface design principles and have lots of rules and laws sent down to them from above. So all of those constraints and great ideas from the higher-ups tend to eventually get passed on as over-complex forms and processes.
It seems likely that some of those issues may have factored into this.
"But only a fraction of applicants ever used that form before the agency took it offline, after officials decided to abandon the initial technology and development method and move toward a cloud-based system"
When government agencies like this talk about a "cloud-based system", what do they actually mean?
I assume it means that they rent hardware rather than buying a mainframe from IBM. Remember, the project has been going on for ten years, so the hardware is probably quite out of date.
> the initial $500 million contract had been awarded to IBM
So IBM accepted a pile of money for the project and didn't get it done. The blame lies solely on them. Sure, there were bureaucratic obstacles, and government officials who didn't do things necessary for the project, but that's no excuse; IBM was responsible for making it happen.
I keep hearing people talk about government inefficiency as though it were actual government employees who were failing. But it's usually like this: a failure of the bidding process and of accountability, where a large corporation (it's always one of the same few) collects money for a job and doesn't do it.
> I keep hearing people talk about government inefficiency as though it were actual government employees who were failing. But it's usually like this: a failure of the bidding process and of accountability, where a large corporation (it's always one of the same few) collects money for a job and doesn't do it.
"The bidding process and accountability" is a function performed by actual government employees, so its failure is a failure of actual government employees, even if that failure is visible through the failure of a government contractor.
A big problem is that government often does not have staff that understand enough about IT projects to effectively execute them, so it hires other people to do them: but the same skills that they lack that motivate outsourcing means they also don't understand IT projects well enough to do any of (1) effectively specify what it is that the contractor is supposed to do to allow contractors to bid properly, (2) effectively evaluate potential contractors, or (3) effectively oversee the contractor once one is selected.
> Agency officials did not complete the basic plans for the computer system until nearly three years after the initial $500 million contract had been awarded to IBM
Nothing wrong with a company bidding on a project of undefined scope. They likely properly laid out what that money bought -- hours, months, years of consulting / implementation. If the client did not have the interested in getting to implementation in short order, that is the client's issue. The company (IBM's) clock starts immediately and the dollars start disappearing.
The problem here, as in most large government contracts (aerospace is a prime example) is that all the bidders know that an honest bid will never win. So, to win, all the bidders knowingly underbid, and hope that they can get adequate funds as an add-on, when the project as bid begins to obviously fail.
The only way to stop this would be to punish the contractors of the failing projects in some extreme way.But that, of course, will never happen.
Contracting seems to be a big part of it. The justification for contracting these kinds of jobs out rather than just doing them in-house is that private-sector competition through a bidding process will lower costs, ultimately making the government more efficient. But in many cases I wonder if it wouldn't be cheaper to just hire some in-house developers, rather than bleeding all this money to the likes of IBM, Oracle, etc.
The point is totally to make it more expensive for the taxpayer, so long as the right outsourcing companies get the contracts.
An in-house development team would know the domain, the managers would be much closer to the business and have more authority to make day-to-day decisions.
Plus, if the developers are permanent staff, they're aware that they will be responsible for the system once it's live.
That's much better for everyone, except the politician, whose wife has shares in IBM.
It boggles my mind that any organization still pays for "professional services" provided by these massive consulting firms. Every single project I've seen like this involves countless layers of middleware implemented by consultants barely intelligent enough to follow a generic runbook, countless calls "home" to deal with your "unusual environment", delays, and a creaking disaster of a finished product.
This looks like an opportunity. There should be an app like facebook where you enter all your data and an app store for forms. Want insurance or DMV or immigration? download the relevant form and it tells you what's missing so you can fill it and the app also give an updated status on your application.
Is there a way technology could help here? Perhaps not by involving the US Government directly, but would it be possible to set up a service that allows you to enter your information online, makes sure you have everything correct and consistent, then prints and ships the paper forms to you to sign?
Its not really any easy problem. Its about 6 million applications per year.
What we should really do is make an open source app or website (perhaps p2p, perhaps run on donated servers) and completely eliminate any part of U.S. Citizenship and Immigration Services related to processing applications, as well as all of the relevant laws. The laws are too complicated, and the bureaucracy of USCIS uses a very large budget of more than 3 billion dollars. A short (probably 100-1000x shorter) set of streamlined replacement laws can be designed by experienced lawyers within a few months.
We can make a contest that provides 5 million dollars for the best 3 designs for protocols and schemas for a common data exchange system, then another 5 million for the best 3 apps that allow people to process data. This will spend 30 million. We can use 100 million to buy off the politicians or companies that try to stop us. That should leave hundreds of millions left to donate to poor immigrant families and/or unemployed federal workers.
The app or site can be staffed by about 200-1000 online volunteers. This will be plenty assuming most routine applications can be processed within a few minutes.
The immigration process is a nightmare, but I thought that it was done on purpose. It took me many years and probably hundreds of pages of documents and forms to get my US Citizenship but I didn't use an attorney.
I find a major problem in very large organizations is the centralization of processes when there is ample budget to establish competing organizations.
For example, clouds and datacenters in large orgs should have three different groups offering competing solutions. If one (inevitably) underperforms, then it gets axed and a new datacenter option is bootstrapped.
There are three main categories for the underperforming group's failure:
1. Structural organization issues: for example, the clients of the underperforming datacenter group play political games that undermine the datacenter group and prevent it from performing well
2. Poor management within the datacenter group itself: for example, employees forced to attend lengthy, unproductive meetings
3. Performance issues with the actual rank-and-file employees
So when you decide to 'axe' the underperforming org, what does that entail?
You could redistribute everyone, but if the error is category #2 or #3, then you saddle the higher performance groups with the causes of the problem. But if you fire everyone and the problem category #1, now you have to hire a bunch of new people and they are still doomed to failure.
How do you solve the problem where the organization needs to be self-aware enough to figure out if it has a category 1, 2 or 3 dysfunction?
When I see stuff like this, I can't help but wonder if this is the result of some in-fighting sabotage to delay policy change in much the same way stuxnet slowed Iran's nuclear program. I don't know how else to explain such colossal incompetence.
Is this the same agency that not so long ago had their computer systems simply "down"... for a week or so. It was some form of immigration or Visa granting agency.
The root problem is not bureaucracy either. Companies like Amazon, Google, and Microsoft have fairly large bureaucracies yet they routinely successfully complete massive software projects.
The root problem is not "vested interests with complex requirements" either. When coders with large-scale-project experience were put in charge of revamping Obamacare's healthcare.gov, they figured out how to cope with a wide range of demands from a wide range of constituencies in an efficient manner, and got the job done: http://www.thewire.com/politics/2014/02/how-silicon-valley-s...
The root problem, I think, is that the people who are put in charge of these massive software projects -- typically, MBAs or lawyers by training -- often don't know anything about how to develop software at scale. It's like asking people who don't understand rockets to lead development of a new kind of rocket that has never been built before.
Edit: added more detail and context to the third paragraph.