
People’s freedom jeopardised by new software adopted by California’s courts - dct
http://www.bbc.co.uk/news/technology-38153992
======
noir_lord
Software is currently in a strange place legislatively, in the 18th century
Civil Engineering exploded (sometimes literally) and the number of disasters
went up in radically (for an interesting case
[https://en.wikipedia.org/wiki/Tay_Bridge_disaster](https://en.wikipedia.org/wiki/Tay_Bridge_disaster)),
over time professional standards bodies grew alongside the maturing industry
to both ensure that people working in the industry where adequately trained
_and_ adequately protected from external factors (people pushing them to do
things cheaply, skimp on designs etc).

Given that we've handed a large part of the running of the modern world over
to computers, software and the people who write that software I think at some
point we need to start looking at a similar system, now the usual argument to
this is "We don't want regulation" or "It's not my code, it's other peoples"
but that doesn't alter the reality that this isn't going to get better until
the way the industry is run changes, it's not a software issue, better tooling
won't save us (it might help though) and there is no perfect outcome.

EDIT: It's worth noting that if as an industry we don't work with our
customers on this stuff and reach an accord that fits everyone eventually one
will be imposed on us from outside, Software Engineering doesn't exist in
isolation, other fields of Engineering are covered already.

~~~
jdietrich
We need formal codes of practice, we need good institutions, but we also need
a cultural change.

We no longer have the luxury of saying "oh, it's just a CRUD app, it doesn't
matter if things go wrong". Software has become too important, it has become
too deeply intertwined in our daily lives. Errors and leaks from trivial
little apps can have life-changing consequences for users.

If your software handles personally identifying data, there's a non-zero
chance that you could ruin someone's life through negligence. Legal case
management software might be an obvious example, but it's the tip of a very
big iceberg.

If you have several million users, one-in-a-million edge cases are going to
happen constantly. Weird things happen at scale. We need to treat every
Android permission and every Facebook API call as a potential matter of life
or death. We need the personal courage and the support of our peers to say
"no, I will not implement that feature", "no, I will not store that user
data", "no, I will not transmit that in plaintext", "no, I will not commit
that to production without unit testing".

[https://en.wikipedia.org/wiki/Iron_Ring](https://en.wikipedia.org/wiki/Iron_Ring)

~~~
tener
Don't put the blame on the developers. Too frequently they are put under huge
pressure and cannot say "no" to their managers. There are people which are
responsible for delivery, testing and they are paid way more than the average
developer.

Unlike in engineering, with software you can - and indeed should - test it
before deployment in thousands of ways.

Finally there is also a receiving party that should ensure they are not
accepting broken software. They have responsibility too.

~~~
jdietrich
>Don't put the blame on the developers. Too frequently they are put under huge
pressure and cannot say "no" to their managers.

Befehl ist Befehl. Blaming management is literally the Nuremberg defence. If
we feel that we _have_ to do reckless and dangerous things in order to keep
our jobs, then we desperately need to unionise. It's just not good enough for
us to throw up our hands and blame the PHBs.

Management need to change, but they're not going to do it voluntarily.
Pressure needs to come from users, but also from developers. We need to fight
back against poor practices, as millions of workers have done in other
industries. We're the guys on the shop floor, we're the people who know what
the issues are and how to fix them. If we don't take a stand, nobody else
will.

How would we react if this story was about a plane crash or a nuclear accident
caused by negligence? Would we be so quick to absolve employees of
responsibility, or would we be asking why nobody blew the whistle?

~~~
hackuser
> Pressure needs to come from users, but also from developers

It needs to come from government especially. It's a basic function of
government to protect the public

> How would we react if this story was about a plane crash or a nuclear
> accident caused by negligence?

If there were no government laws penalizing this outcome and regulations
preventing it, I'd be horrified.

~~~
CamperBob2
There are plenty of regulations covering software used in aircraft and nuclear
power control. Move on to the next moral panic.

~~~
subway
Gee Bob, that did a helluva lot of good in preventing these folks from
enduring undue legal hassles.

------
dkarl
_Similar problems have been reported in Tennessee and also in Indiana - where
prosecutors have had a perhaps more troubling issue of inmates being
mistakenly released early._

It seems backwards to describe this as more troubling.

~~~
briandear
Not at all. Do you want a rapist getting out of jail earlier? That would be
troubling.

~~~
brewdad
But a person with a traffic ticket getting jailed and registered as a sex
offender is better? Both are equally troubling. In my mind, the latter moreso.

~~~
daveleebbc
Hm - I take your point. I was coming at it from a public safety point of view
first and foremost. But I can see the other side, sure.

------
awinter-py
Separate from the question of software liability is the court's responsibility
to not make egregious errors.

Historically, courts have been unwilling to assign themselves blame for
screwing up. Judicial immunity is untouchable in american case law because
it's hard to find a judge willing to rule against it.

The standard of due process is high for 'life and limb' cases but low for
misdemeanors & traffic violations. When Fixed discovered that most SF parking
tickets are challengeable, SF didn't fix the problem -- they turned off their
fax machine to make it harder to challenge tickets.
[https://techcrunch.com/2015/10/12/fixed-the-app-that-
fixes-y...](https://techcrunch.com/2015/10/12/fixed-the-app-that-fixes-your-
parking-tickets-gets-blocked-in-san-francisco-oakland-l-a/)

NYC issued fake parking tickets to the tune of hundreds of thousands of
dollars. [http://iquantny.tumblr.com/post/144197004989/the-nypd-was-
sy...](http://iquantny.tumblr.com/post/144197004989/the-nypd-was-
systematically-ticketing-legally)

Jurisprudence doesn't have a concept of 'bulk miscarriage of justice'. You
could put NSA surveillance in this camp too. There's a star trek line about
genocide which says 'we have no law to fit your crime'. That's where we are
with petty crimes mishandled in bulk.

------
danso
> _The software, created by Texas-based Tyler Technologies, costs about $5m
> (£4m) and is set to gradually replace a decades-old e-filing system that
> looks like something a hacker would use in a Hollywood movie._

> _Tyler Technologies acknowledged in a statement that the upgrade process had
> been “challenging” - but said poor training was to blame for bad inputting
> of data and integration with third-party applications that often introduce
> glitches into the system._

Even as someone who spends the majority of the day at the command prompt, I
agree we should always be attentive to user-interface issues. But I've been
less than optimistic that people will have the wisdom to know that a new
"modern" interface automatically means it's more sophisticated/elegant, or
that it's more attuned to the needs of human users.

In particular, it's been alarming to see a rise in unnecessary use of AJAX
(nevermind Angular) in government applications. I'm not anti-JS, it's just
that client-end development seems to have far more moving parts of the kind
that don't get well-tested by workers in a bureaucracy. Especially when that
work has been farmed out.

edit: as an example of how government can do modern web-dev/UX well, I can
think of no better (at least at the U.S. federal level) example than the CFPB:
[https://cfpb.github.io/](https://cfpb.github.io/)

------
panic
_Tyler Technologies acknowledged in a statement that the upgrade process had
been “challenging” - but said poor training was to blame for bad inputting of
data and integration with third-party applications that often introduce
glitches into the system._

People writing software need to take responsibility for how the software is
used, especially when it can impact people's lives to this degree. You can't
just blame third party software or the people entering data.

~~~
noir_lord
> People writing software need to take responsibility for how the software is
> used

There is an old joke that it's almost impossible to get someone to understand
a problem that would result in a threat to their livliehood.

I don't think people need to "take responsibility" I think organisations need
to be _made_ to take responsibility.

In the UK when you purchase a physical good from the retailer the warranty for
that good is between you and the retailer, the retailer then has to deal with
problems up the line from suppliers, it's not perfect but it's workable.

I think software should be the same, I think if I pay Foo for a system then
Foo _should_ be responsible for the system even if it's made of parts from
Bar, Fizz and Buzz, if Foo has an issue with Fizz they need to take it up with
Fizz.

There is _so_ much crap software out there in every field and the acceleration
towards a world run on software continues.

~~~
acbabis
I'm not saying it's not a good idea, but that system's gonna be really tough
on the software industry. Consider what would happen when a famous third-party
API has security flaw. Most third-party libraries' licenses have an "as-is"
clause and even mature third-party software has exploits sometimes (e.g. Java
applets).

If you wanted to use a library in a project, you would have to

    
    
      - a) read the entire thing to see if you can find any bugs that the devs missed, or
      - b) roll your own solution which will probably have even more bugs, or
      - c) find an equivalent library that has a paid version without the "as-is" clause.
    

Imagine paying a monthly subscription fee just so you can offset the liability
when Angular has a security hole. This would run a lot of small shops out of
business.

~~~
noir_lord
I own one of those small shops so I'm not unsympathetic but few other
industries get away with 'as-is' disclaimers,

Can you imagine it in the automotive,aeronautical, transport etc?

The present system is a pass the parcel of blame with no one taking any
responsibility, tonnes of shit, insecure systems written and designed poorly.

Eventually something will give.

~~~
jonnathanson
_" Can you imagine it in the automotive,aeronautical, transport etc?"_

This. This is the most important point I've read so far. The key word here is
"imagine."

I think even we, as an industry, don't quite grok the degree to which software
truly has eaten the world -- and the degree to which it now _is_ every bit as
important, as life-or-death, as things like cars, airplane travel, or even
medicine. We hold these goods to certain standards because we deem them
extremely critical -- too important to be left to the de facto standards set
by the operation of a wild-west market. It's about time the notion of
standards-free software shocked people as much as the notion of standards-free
engineering in any other systemically critical industry.

I'd prefer self-established standards set by the industry to government-
imposed standards set by lay bureaucrats. But the latter is coming eventually
if the former never does.

------
wtbob
Although I don't have any direct evidence regarding this instance, I wouldn't
be surprised if the old system was greenscreen & form-based, while the new one
is some kind of shiny Java-backed web app. It wouldn't surprise me at all if
the old system was faster to use and less error-prone: those old greenscreen
apps tended to be optimised for long-term use, rather than for showing off in
a board-room demo.

~~~
hitgeek
having worked in data entry on a green screen app, as well as the .Net,
Windows Form "upgrade", I agree with this sentiment.

the green screen app was highly optimized for efficient and accurate data
entry without use of a mouse. The modern alternative looked better, but a
large portion of the functionality was much less efficient in comparison.

the only redeeming factor of the new application, was it allowed much of the
manual data entry to be automated. This however required considerable time,
technical knowledge, and industry contacts to develop and implement, which not
all organizational users possessed.

------
ptaipale
The headline is somewhat clickbaity and sensational.

Yes, it is bad that there are clerical errors in justice system.

But, if information is lost due to faulty software or user errors or even user
error helped by bad UI design, it's still fundamentally just a clerical error.
Those errors should be fixed and perhaps some people should be eligible for
compensation for being mistreated due to error, but there is no sinister
"software is putting people in jail" plan here. Just errors.

Embarrassing ones that should be fixed at a priority.

~~~
andrewla
It definitely seems like more of the blame here should be shouldered by the
police and the courts, who, knowing that the system has problems (and
hopefully assured that those problems will be resolved in the future) should
put less faith in them, and double-check (against filed papers, for example)
potentially dubious results.

This is more expensive in terms of people's times, but it's just part of the
cost of adopting the new software, and should be treated as such, possibly by
billing the software vendor for the additional manpower required to work with
the software during the transition.

~~~
ptaipale
Indeed. The "Computer says no" or "Computer says catch him" attitude should
change: particularly if you know it's a new system, check twice before making
drastic actions based on data it gives.

~~~
dsfyu404ed
Even just "computer says this record has been randomly selected for manual
review to verify system integrity, please consult your administrator on how to
proceed or enter a manager's PIN to override" would be really helpful since it
would effectively train people in double checking.

------
eponeponepon
I expect the truth of this lies somewhere in the outsourcing industry. It has
every smell of miscommunicated requirements and half-assed implementation.

If I'm right, I doubt it'll ever be admitted to though.

~~~
briandear
I wish I could upvote this a hundred times. I am currently dealing with an
Indian dev shop and the code has been atrocious -- as if they didn't even read
the requirement. Doing a very basic Stripe integration has taken nearly a week
and it was still incorrect. I could provide days of examples.

~~~
eponeponepon
The people _writing_ your code have certainly not read the requirements.
They'll each be working on one single aspect, and focussing entirely on
fulfilling some knocked-together unit tests written by someone who might've
read the requirements, but only with a view to working out how to make as much
existing code as possible from the last customer fit into your project.

------
luckystarr
Kind of reminds me of Therac25. There a shitty user interface killed people.
Here it "only" affects peoples lives in a dramatic manner.

A shitty user interface ruined somebody's life. Consider that.

~~~
icebraining
For those like me who didn't know the details:
[http://hackaday.com/2015/10/26/killed-by-a-machine-the-
thera...](http://hackaday.com/2015/10/26/killed-by-a-machine-the-therac-25/)

------
mVChr
I'm so glad I don't have to only rely on a journalism source across the globe
and can count on my local paper to warn me about such things.

[https://duckduckgo.com/?q=site%3Alatimes.com+tyler+technolog...](https://duckduckgo.com/?q=site%3Alatimes.com+tyler+technologies+software+upgrade&t=ffab&ia=web)

/s

~~~
daveleebbc
To be fair - it's been covered by the East Bay Times, SF Chron and KQED in
some depth. That's how I heard about it.

------
tremon
Not a system upgrade, but a faulty system. To which branch of the government
should the court system be accountable?

~~~
cafard
Umm, the courts?

I knew various government techies who went to work for the Administrative
Office of the [US] Courts. Honestly, I don't know what the chain of command
was there.

------
FrancoDiaz
Holy Cow! There needs to be some serious consequences for those responsible
for this travesty.

------
threatofrain
Perhaps I'm crazy, but isn't a $5 million contract a bit too low for an
overhaul of the California justice software system?

Business-wise there's going to be a lot of things to cut, pushback against
government asks for software, and a very skeletal plan for maintenance mode. I
sometimes wonder how government models the businesses they do work with, or
whether they work as hard as businesses in modelling the other side.

------
fataliss
I don't get why such software isn't an open source initiative. I wish the
government would give more legitimacy to orgs like "code for america". Who
wants a proprietary closed janky software to rule wether you are a criminal or
not? Nobody, that's who. The only beneficiaries in that story are the people
on the other hand of the contract, making $5M for an half baked piece of
software!

~~~
splicer
While I certainly agree that all government software should be open source, it
sounds like a big part of the problem here was a failure in gathering accurate
requirements (which is a very expensive endeavor).

------
forgotpwtomain
I don't see why people in the comments are blaming tech company (Tyler
Technologies) for this. There was certainly someone over-seeing procurement
and specifications for the government and it was _their_ job to make sure that
the product which was procured and delivered was functional and ready to roll-
out. _This_ person and their department should be held responsible.

------
kazinator
> _had recently finished a six-month drug programme after he was caught in
> possession of marijuana and ecstasy._

THAT is what fucking should not have happened in the first place.

Let's not shift the focus to some glitchy software issues that will likely get
worked out.

------
JumpCrisscross
If you care about the welfare of your fellow Californians, consider sending a
letter to your U.S. Congressperson, your California State Assemblyperson and
Senator [1]. Attach this article as an exhibit. Copy your county court.

Then, and this is very important, set a reminder out one week and call each of
those people, confirming they received the letter and understand your
concerns.

If this is too much, either accept you don't care about the issue (that's
fine) or, if you do, that you may have wrong attitudes about how citizen
influence works in a democracy.

[1]
[http://findyourrep.legislature.ca.gov](http://findyourrep.legislature.ca.gov)

------
jamesvl
I worked as a programmer for a smaller California county court system for
about five years, and have seen something very much like this play out before
(both in my county and others).

I can't comment on the Tyler product or their training directly; maybe they
really are a rock star outfit. But if this is like past attempts, this project
has all of of the worst of aspects of software development risks and none of
our more "modern" methods to mitigate them.

The court employees - most of whom would _not_ be considered very computer
savvy - probably had a lot of training directly with Tyler but are struggling
with a system that a) doesn't meet their needs, b) changes years (decades?) of
ingrained workflow habits and terminology, and c) may be much slower than what
they used to have.

Observations from past projects like this:

* at it's heart it's a database CRUD app, but with hundreds of tables and thousands of fields and business "logic" encoded (in more database fields) to help with validation and workflow

* most of the above fields need to be fully customized for each county, so add in tables and logic to modify your UI on every screen

* this software was not built for Alameda county, but re-purposed from use elsewhere. Terms and concepts for how the law worked in the state this was originally built for may or may not apply here.

* "usability" success metric: "do all 50 fields on the page accept input and save data in less than 60 seconds?" (i.e. no concept of real HCI usability design at all)

* iteration process: waterfall. Vendor sits with court subject experts for 2-3 months, documenting all of the workflow. They customize their product to meet those needs, and a month later show a build that does this. Court can't use it yet (deployment locally would cost way too much), but they've printed out hundred of pages of screen shots to help document how it could be used. Hire external consultants to help with this process. Repeat until a) court money runs out or b) someone's reputation will be tarnished if the system doesn't launch

* There is no staging environment. Deployment is on local hardware only (no cloud). No bug tracker exists that the court can see. Builds are not automated, and "maintenance" may cost the court additional money.

* importing previous cases: worst ETL job you can imagine. Take data from an aging mainframe database that may or may not have any relational integrity at all, and try to plug it into a system as described in point 1

* administrative overhead: your county is given money from the state to do this, and then no choice about which vendor or software to use (because the state wants to roll this out in _all_ counties... each of which is very different from one another, even in CA)

tl;dr This is a horribly difficult software update, subject to the worst
practices in our industry.

Personally, I don't think blaming court employees for "clerical errors" is
fair at all - not that those haven't happened, but (from my experience) these
are hard working people who care about justice yet have really lousy software
that impedes their job.

I'd love to see a company do this software right - custom build, real
iterative development hand in hand with the users. The Courts really needs it,
they've never experienced a high quality product in this area, and the
inefficiencies affect the wider economy (because civil matters are faster to
resolve).

[edit: formatting]

~~~
logfromblammo
Tyler Tech is not a rock star outfit. They may think they are, but they are
not.

I interviewed with them in 2008, and that remains, to date, the worst
interview experience I have ever had. Everything they did before the interview
seemed calculated to convince me to withdraw myself from consideration, and
everything after seemed calculated to discourage anyone else I knew from
applying.

So I felt a little frisson of schadenfreude from reading the article.

------
thinkcomp
This stems directly from the opacity of the California Judicial Council
Technology Committee and the CCMS debacle it has generally made worse. Public
comments are welcome, but secret.

[http://www.courts.ca.gov/jctc.htm](http://www.courts.ca.gov/jctc.htm)

------
ttctciyf
Anyone else put in mind of Gordon R Dickson's (1965!) classic: _Computers Don
't Argue_[1] ?

1:
[http://www.dave.rainey.net/calendars/dystopias/process3.html](http://www.dave.rainey.net/calendars/dystopias/process3.html)

------
codedokode
The problem with incorrect input and mistakes could be solved by entering the
same data two times by two different people. It is probably not that
expensive.

~~~
avmich
It could be reduced this way, agree. But if a particularly bad UI is used in
both cases, I'd assume it's quite possible to have duplication of errors here
too.

I don't think that's enough in all cases.

------
smnscu
Oh, no, people's freedom is being jeopardised? That's terrible, BBC.

[http://www.theregister.co.uk/2016/11/30/investigatory_powers...](http://www.theregister.co.uk/2016/11/30/investigatory_powers_act_backdoors/)

~~~
noir_lord
What does one have to do with the other?

Does the UK passing the investigatory powers act in any way have any bearing
on abuses by the legal system in other countries or are we just playing
"whataboutism bingo" today?

------
sabujp
why is the company the govt not being sued into oblivion?

------
joesmo
Justice is whatever the 'justice' systems says it is. Citizens have no
recourse. The 'justice' system has no checks or balances that can be accessed,
except by the rich. There are no penalties for abuse or misuse, even when
uncovered.

Yeah, that's exactly what I think of when I think of the concept of 'justice'.
One day some people might create a just society, but it almost certainly won't
be in the US.

------
the-dude
tl; dr; No need to be concerned fellow hackers, this only applies if you have
been in the slammer before.

~~~
julianj
The article also references traffic violations - "Minor driving offences were
incorrectly appearing as serious felonies"

~~~
the-dude
Ah, well. At least I skimmed the article. Was worried system admins were
getting arrested about automatic updates and such.

