Hacker News new | past | comments | ask | show | jobs | submit login
The Machine Fired Me (idiallo.com)
1471 points by foxfired on June 19, 2018 | hide | past | favorite | 554 comments

I have a story in a similar vein, although a much less scary one.

In that instance, The Machine Cleaned Out My Desk.

I had a cubicle at the company HQ, but was for some period of time working from home in another state. I still kept quite a few things in the cubicle (notebooks, mugs, etc), which I used when I was in the area.

When I finally came back, I noticed, after a month or two, that my office number has not been updated in the system (from being "HOME OFFICE"), and sent a request to IT to change it.

The next day I came back to see a pristine desk.

With all my stuff gone.

See, my request to update the office number triggered a relocation request. The system, in preparation of the move-in of the "new" tenant (me) into my cubicle, has removed all the belongings of the previous tenant (which also happened to be me).

Luckily, all the removed stuff was put in a box, which I got back several days later, after my manager found the right person in the facilities dept.

Just goes to show that automating even the simplest procedures can be very tricky - and that perhaps it's best to have people on-site manually approve any destructive steps.

> automating even the simplest procedures can be very tricky

automating business processes is actually a similar activity to doing programming!

When it's done competently, the business runs smoother. But if there are bugs (and as anyone who programs knows, there are always bugs), things go wrong. And yet, people who know not anything about complex systems design attempt to write up business requirements for such automation are numourus.

Yes, you can actually program a bussines process using BPMN. Furthermore, there are tools that convert BPMN to the actual machine code automatically.

Or, stop letting the people with business degrees effectively write software via SAP. They're not trained for it, and they don't know how to think through edge cases or reduce fragile entanglements across areas of concern.

Idea: Require every action taken by an SAP or similar system be tagged with the name and telephone number of the MBA who created the action in SAP.

> perhaps it's best to have people on-site manually approve any destructive steps.

Or, almost certainly cheaper, manually deal with the 1 in 10,000 case like yours.

In this case, a simple notes field in the request system would perhaps have sufficed. Perhaps there is/was, and it wasn't filled out because it wasn't recognized as being some manner of exceptional request. Perhaps if there was manual approval, removal of your stuff would still have been approved.

It's literally impossible to catch all of these, because human error will creep in. Sounds to me like your case was dealt with adequately. (Perhaps not from your personal perspective!!)

>Or, almost certainly cheaper,

If my research notebooks actually ended up in the trash, the loss of productivity would have certainly not been cheaper compared to implementing a simple checkmark requiring the approval of the manager of the person whose desk was cleaned out.

Now that think about this, this is a clear example of the famous Frame problem (https://en.wikipedia.org/wiki/Frame_problem)

The next day I came back to see a pristine desk. With all my stuff gone.

Something like this happened to me too at a previous employer, some things I recovered but many were just gone, the cleaning staff apparently help themselves to stuff that “former” employees leave behind, so my fancy headphones for example were just gone. Fucks given by HR/facilities? Zero. One of many similar incidents for me and my cow-orkers. And this was a desk move literally from one row to another!

It wasn’t even an algorithm per se, most of the “machine” at this place was people in India following checklists manually. You could speak to them (tho' they made this very difficult to do) and tell them to stop and they would say “yes” and do it anyway.

I had a friend at another company who was mistakenly terminated, a week later his manager called him at home to find out if he was OK, the conversation apparently went,

Are you sick? What happened?

You fired me you bastard!

No I didn't! Please come back!

Too late now, I have another job.

A paper/protocol machine is still a machine, though. Sucks that it ate your headphones!

I am normally big on Solidarity with fellow Workers but the humans in this loop really should be automated away, because they knew what they were doing was a mistake and did it anyway, so what value were they adding? In fact they were worse than automation because at least that can be debugged, but there is no fix for the bureaucratic mindset.

Especially since you could tell them to stop and they would say "yes" and then carry on anyway...

Another story from the same company, group A would enter their requirements into system 1, group B would pick up work tickets from system 2. Group A thought that group B were idiots who could never do anything right, and group B thought that group A were idiots who could never make up their minds what they wanted.

But the real problem was group C who maintained systems 1 and 2 and "integrated" them with people in India manually rekeying from one to the other with frequent typos. They thought they were saving money but never considered the cost of delays and re-work in groups A and B...

>you could tell them to stop and they would say "yes" and then carry on anyway... //

This was discussed/commented on at length a couple of weeks ago. In Indian culture, apparently, the "yes" is like a verbal tick - kinda - and just acknowledges you've spoken without giving any commitment to doing anything (nor indeed indicating any level of understanding).

Not exactly. If you ask "do you understand ?" and "will you do it?" they will tell you yes as well. It's not a verbal tick. It's just not socially acceptable to say no.

I wonder if western culture has similiar quirks that we don't even realise.

“How are you?”


Tons of people have this exchange out of politeness. The person asking “How are you?” doesn’t really want to know details. The person answering is expected to be brief and positive. It’s only among friends that the same question is expected to get an honest and detailed answer.

As a Finn I typically answer what I really feel at that moment. It creates funny situations, but in general I think if you ask me a question I should give a truthful answer. Why would you ask if you don't want to know...

It's a quirk in English. "How are you?", "How's it going?", etc. are often greetings, not serious questions.

“How do you do?”

—“How do you do?”

tipping of hats ensues

I tried this and it worked poorly; people tended to get annoyed at me for answering the stupid question that they asked. Now I just go with "not too bad" in various tones of voice.

Brief yes, but nothing about the question suggests they want details. As long as you're not making things too personal you can answer non-positively.

I bet you could ask “was that a lie?” and get the answer “yes” too...

I think the moral of the grand OP story was the automated aspects of that machine only enhanced the human actors tendencies to just follow through.

Did you threaten to call the police about theft? Could've followed through with the threat also.

I escalated it to building security, where I had some mates, and they reviewed what CCTV evidence there was but couldn't find anything.

I don't even really blame the cleaners; people left expensive electronics, phones, wallets, whatever on their desks all the time, never any issues at all, I don't think any of them would have taken anything that wasn't from the pile, it was probably at least tacitly sanctioned by their management (who probably helped themselves first too).

Yeah, I think that's the difference. They won't touch anything that's clearly for someone else. But in their mind, the previous tenant had abandoned that stuff. It was free real estate. They didn't think it was stealing because they didn't think it belonged to anyone.

"Ah you admit I was unfairly fired!" how about we settle for 9 months salary or my lawyers will be in touch.

Something eerily similar happened to me. One day I arrived at work as usual and the turnstile didn’t work. I had to request a guest pass and call some colleague to escort me because I was disabled in the security system. I went up and I obviously couldn’t open the door and access my pc. The scary thing was that I was still in the middle of my contract, it would have expired only 3 months later. I contacted my boss and went home. A couple of days later they managed to reinstate my security account, although I couldn’t still access my pc for one or two days. In these days I just helped my colleagues. Once I got back my account I started re-requesting the 10s of permissions that I needed, and while I was doing so I found an open request to recycle my machine. Luckily I managed to contact the guy that was supposed to take it and I stopped him just in time. On the whole I probably lost two weeks. What was the trigger for my termination? Apparently someone managed to input the wrong termination date in SAP for my contract, and that started the havoc. Obviously there were alerts before my termination but everyone ignored them because in the other systems visible to the approvers my termination date was correct. Luckily I got paid for every single day, even when I was at home, but after three months I’m still suffering from random problems in random systems that are probably related to this mess.

> and went home

Did your boss send you home? If not, you should probably stay and be ready for work assignments (even if they don't come). For good measure contact some colleagues each day so they remember you and take a picture before you enter and after you leave.

This is classic modern work culture. Make sure you document your attendance and effort during employer incompetence. Empty your time to exit the turnstile.

No. At a certain level of employee experience and commitment this type of thing is inexcusable. Gaming the game is all that is encouraged these days. Complete corruption.

I told him that I was going home and he was fine.

> I missed 3 weeks of pay because no one could stop the machine.

Why has this company not made their loyal worker whole? He stayed there when they needed him even when their system was trying to lock him out.

They need to fix this. If they don't, they are not a company anyone should work for. Perhaps the worker did not want to risk making a big stink, but a manager should have taken the initiative. Humans were involved by the end and well aware of what was going on.

I agree. It's a theme I run across in many US-worker situations we read about on HN. There is such a lack of basic rights, but also basic norms when relating to workers.

Ethical behaviour, from my perspective, would be to compensate the employee, regardsless of his legal rights. In a more worker-central system, the worker should have easy recourse to an official judgement for his money.

In the Netherlands there is even a concept of culpability in laws regarding firing. Mess up too much, and the employer will have to pay a premium on the disengagement fee. And while our economy is moving towards a lot more 'sole employee contractors' with less worker-rights, you still have rights and a way to affordably enforce them. For them (only) basic contract law holds. That would probably mean paying the full 3 years in this context. A contract _is_ a contract.

None of what you says really applies in this case. According to the article the thing that triggered this was the failure to renew his contract.

If it had renewal intervals, that means he wasn't legally entitled to 3 years.

This is likely why he wasn't paid for the interim either. The best the company could do would likely be to bump up the future rate or offer a "signing bonus".

Being a contractor is not at all like being an employee despite how similar the responsibilities are. You have to watch contract details like renewals like a hawk. This guy and his recruiter should have included compensation for failing to notify when not renewing.

Not sure how this works in other jurisdictions in the EU but in Germany a work contract can be silently renewed if both parties act as if it had been renewed even if the contract itself says it has to be renewed explicitly and in writing.

If you keep showing up to work and your superiors keep managing you (and especially if you keep getting paid) the contract remains valid beyond its stated expiry date.

It does though, failure to renew a contract should be announced in writing a month in advance to allow the employee to search for a new job and take holidays that may be left. If the announcement is late, the employee is entitled to an extra months' pay.

I doubt that's correct legal analysis. This worker is not an employee of the company, and so cannot assert rights afforded employees. There might exist special termination provisions in the contract, but that's speculation.

While I agree with the letter of your legal analysis, it should not be "the best the company could do" to make the worker whole. It ought to be expected that both parties behave ethically.

The worker should not abandon the company during their time of need because of an unfortunate legal SNAFU. The company, in turn, should take it upon themselves to ensure that the worker doesn't suffer economically for having done the right thing by them.

Why wouldn't a contractor leave the company when the company has done everything in its power to show the contractor the door? Aside from doing the job properly, the contractor has zero obligation to the company whose property he's working on.

In the 2-3 days after the glitch, when "there was an emergency on the multimillion dollar tool I was working on", I think it was good of the worker to not get hung up on the contract status and trust that it could be worked out by people in good faith.

Workers and management on a team ought to be able to trust each other to that degree, even if we understand that the interests of all parties are not fully aligned. That trust goes both ways.

Take that too far, tolerating abuse of trust, and you betray your obligation to yourself. But in small amounts, it's admirable.

If they're not being paid, then yes, the worker should abandon the company.

Do you seriously think I'm suggesting that they stay indefinitely without pay?

Or, if you are saying that they shouldn't stay even for a day, even in the face of an emergency, then I disagree. Even if they don't have a legal obligation to do so, I respect those who would, like this worker. And I would prefer to work along such people as colleagues, trusting that if they could help it, they wouldn't leave their team in the lurch.

I don't like the idea that complete lack of trust and hairtrigger hostility should be the default mode of workforce participation.

There's no moral obligation to work without pay, even for one day, even if you've gotten "promises" that you'll be made whole. An ethical company would send him home if they didn't know they could pay him or not, whatever the reason. If they need him to work THAT DAY because of some corporate "emergency", they should pay him cash on the barrelhead or make a similar arrangement, until they can get his regular paychecks going again.

As advice to a worker in this situation, it certainly makes sense to gut it out and take the risk of doing work without pay. It's, for most people, probably a much smaller risk than risking getting "really" fired without first having another job lined up.

What if the worker stayed, expecting that the company would make things right, but when it was discovered that companies simply don't DO that and haven't since the 1980s, files a lawsuit or otherwise takes aggressive action to force the company to treat them fairly?

"I don't like the idea that complete lack of trust and hairtrigger hostility should be the default mode of workforce participation."

Then complain to the employers, who are the ones who created that culture.

I have. Who started this thread?

For what it's worth, I attribute the worst amoral behavior to corporate personhood rather than capitalism. Thus my preferred remedy is to eliminate corporate personhood, rather than capitalism.

The part of being ethical in business outside the legalise still holds?

I would agree with your point of professional contracting: contract for the worst and write out scenarios in contracts.

In this case though the employer acted in a way that signalled renewal and the contractor delivered in good faith. Back to contract law and ethics: are services delivered possibly without contract but in good faith without value and compensation?

Failure to pay employees is "wage theft" in the USA.

The Netherlands sounds a bit employer friendly in the UK mess up at all in firing some one you automatically lose even if the employee is as guilty as sin and was caught bang to rights

He elaborates a bit in the comments - it seems he's entitled to the money, but couldn't be bothered: "So I had to do an appeal, and go through a long process that I did not care much to go through. I had a mind in quitting after all."

Sounds very strange that he wouldn't care to do a bit of paperwork, which he from the sounds of it could probably well justify to do on company time, with the manager and director being supportive. Or get the agency to do it for him, and bill them for the trouble. Or whatever. Something smells a bit fishy about that part of the story.

"Fishy" is a very ungenerous read on the worker opting out of an arduous appeals process and cutting their losses.

And if there's anything in the comments about "the manager and director being supportive", I don't see it -- where did you get that?

The bit about the manager and director is in the main article. There's nothing about pay, but they're presented as entirely sympathetic and angry about the system on his behalf.

As for "fishy", I find it very strange that someone writes an article emphasising twice that they're out three weeks of pay, but then couldn't be bothered to do anything about it. There's breach of contract, this isn't a long and arduous appeals process, it's open and shut, and if it isn't you have a lawyer deal with it, and recoup expenses, too. And if the guy just doesn't care about it, then why does he mention it twice?

And for the purposes of this, and the other sub-thread that is incredulous about this, this is not an American thing: If you're a contractor in Europe, and you don't get paid, and you then don't do anything about that, then you don't get paid. There's no process that automatically fixes things when the aggrieved party doesn't ask for them to be fixed (and yes, even in Europe, there's a bit of annoying process and paperwork to deal with).

The article is written, I think, to criticize the reliance on machines—not this individual company. I don't think he wrote it out of a grudge.

Even if he's legally entitled to the money, bringing a lawyer into the mix is likely to sour his relationship with his client (he's a contractor), and—rightly or wrongly—could give him a bad reputation in the local community, making it harder to find work in the future.

I'll also note that small, claims court might be able to handle a case over just a few weeks pay. He wouldn't need a lawyer for that. The risk and costs of losing are more manageable. He could send some emails about the pay he lost asking nicely citing the work he did while there. Then firmly. Then, once at a better employer, he can take the case to small, claims court.

I'd actually rather people sue over this to establish some kind of case law where companies' legal teams tell their management to make a default policy of giving the missed pay on demand. There might already be case law on it.

> a bit of paperwork

I suspect this is the difference. If this company is incapable of not firing someone, I imagine their appeals process could be equally miserable. Sure, maybe the manager and director want to repay him, but actually releasing funds to pay someone who "wasn't employed" is going involve a lengthy battle with some HR/Accounting computer system that just wants its invariants not to vary. Depending on somebody's financial position, I can see them deciding to just walk away on even a large chunk of money to avoid the mess.

The good thing about places like that is (a) they don't actually care about the money, it's a rounding error. For some smaller firms paying someone for three weeks of not working (regardless of fault) could actually be a problem. (b) They also have a legal department who's invariant is "don't get sued for breach of contract because of a stupid mistake that a small cheque can make go away".

But, of course, the author is perfectly within his rights to just not pursue this, and I have no idea what processes he'd been told to expect for this -- it's just strange that he would emphasise something he doesn't care about in the post.

> a legal department who's invariant is "don't get sued for breach of contract

This is a good point. I was thinking that no matter how trivial the money, actually dragging a check out of the accounting system would be a serious hurdle. But the solution I missed would be to sic legal on it - their machine tends to beat out everyone else's.

“Appeals forms were made available.”

“Made available? I eventually had to go down to the cellar to find them.”

“That’s the appeals department.”

“With a flashlight.”

“Ah, well, the lights had probably gone.”

“So had the stairs.”

“But look, you found the forms, didn’t you?”

“Yes, yes I did. They were made available in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.'”

Surely a 3 year contract means the company is in breach of it by not paying you?

Sounds like he's contracting at a huge company. His contracting company isn't going to make waves over this. I would guess the contracting company is making $300+ an hour off this guy, paying him $50 an hour, and they have 200+ other contractors in the building.

So, wage theft.

Now we're not just talking about a lost opportunity to reward a loyal worker. If you're right, somebody ought to be going to jail.

But nobody will. This is a common story in contractor arrangements---the big, unwritten benefit of the process is that all incentives are aligned to sweep small mistakes (1) under the rug. Contracting client doesn't want to jeopardize a good contract, contractee doesn't want to jeopardize their contract, and employee doesn't want to get fired and black-balled. Who cares if it's a little illegal? :(

(1) "Small" in relation to the whole contract deal; what's one employee's salary in a million-dollar agreement between two corporations?

Its not wage theft. Contractors aren’t humans, from a human resource pov.

> Contractors aren’t humans

Real story from one of the (big) french telecom companies I worked for, as a contractor (a few years ago)...

In that company, a regular employee was given a full desk, contractors were only given 2/3 of a desk (meaning 2 desks put side by side for 3 persons).

Also, contractors were NOT allowed to go to ANY conference. Even when it was about technical stuff that was directly for their project. We had to rely 100% on a transmission of information by our managers/team leads. This led to humorous results, to say the least. Things like: a BIG meeting with plenty of people and managers, who then decided a HUGE technical change on the network, without the ONLY expert because he happened to be a contractor. Once given the results of the meeting, the guy was in a "WTF?" shock. He told me that be immediately binned the request: it basically asked him to disable the authentication server! Not the best decision for a BIG ISP, LOL. Seriously, that's almost beyond belief. (But I have other incredible stories similar to this one).

As a contractor, I have really been mistreated A LOT. So much so that I've sworn never to do it again.

I'm a contractor in France, the treatment depends on where one is working. Canal+ and PSA were bad, but the current situation is good, there's no difference between myself and the employees.

Sounds like departments were using contractors as full members of staff. Contractors should be brought in for a specific purpose, as a consultant to staff, as an extra pair of hands to follow the whims of staff.

If contractors are the only expert in a domain, there are big problems. Of course in corporations this is quite frequent - middle managers have yearly budgets for contractors, but aren't allowed to use that money to increase their headcount.

A company I work for decided to get a contractor in to run a £40m project. This makes sense for a management perspective, they get to blame the contractor (who's already left) if it fails, and they get to claim the glory if it works. It's terrible for a company though.

If I want 500 cables run in a data centre, or even 5 cables if it's a long way, I'll get a contractor in. Saves me a day by not having to do that work neatly and lets me do more important things (like ranting on HN). If I want someone to build me a fancy gui, I'll specify the frameworks, and let them mess around with the look and feel, but ultimately I own the output of the work, I need to be able to deal with it as if I wrote it myself.

I don't really know if there's wage theft going on. I suppose the client company got some free work out of him, but the company can argue he shouldn't have been in the building. His badge didn't work, etc. But maybe the will let him bill those hours retroactively.

If he's working for a contracting company, why would he lose wages? Wouldn't he still have a contract with that company? I work for a contracting company (in the EU), and if the client lets me go, I still get paid until a new project can be found.

That’s not how it works everywhere.

Their account got cancelled because the contract did not get renewed. Sounds like the company fulfilled the terms of the original contract, just (accidentally) didn't renew it. Many long term contracts require periodic renewals and/or options.

>Some of that work included renewing my contract in the new system... When my contract expired, the machine took over and fired me.

> in the new system... my contract expired

As I interpreted the piece, it was a three year contract which was stored in an old system, and not transferred when that system was replaced. So the firing was triggered by the system no longer seeing any valid contract.

If that's true, it would be breach of contract. But if it was a rolling contract expected to end after three years, then yes, it would be a weird situation but not a breach.

That's weird though. Was it a three year contract or wasn't it?

As I read the piece, it was a three year contract which was included in the old HR system, but not ported to the new one. If that's true, it was certainly breach of contract - "we forgot to upload the paperwork" doesn't get you out of honoring it.

If that was the case they would have not said "renew." "Renew" has special meaning in contracts.

With contract staff, the duration is usually a not to exceed. Also, it’s typically a contract to deliver a person with specific skill, not a specific person.

There are exceptions, but they are pretty rare in tech.

It's not weird - the vast majority of long term contracts I've seen (for services my company provides) are several years long but requires the other party to renew the contract at periodic intervals.

So the contract was negotiated at a specific price point for (say) three years but every X months the other party can opt out by not renewing. They only commit for a certain term.

Perhaps he should've written it as an "adding" instead of "renewing" since it's an existing contract with the merged company that needed to be properly added to the take over company.

Probably informally thought of as a three year contact but legally written as a shorter one for tax etc. purposes.

How do you think you will enforce that contract? Hire a lawyer? You might as well go to the ATM grab as much cash as you possible. Then put it all in a garbage can and light a match. It will be far less costly and more productive then attempting pursue that contract.

Sue in small claims court for week 1. Win. Obtain judgment. File lien against company assets. Sue in small claims court for week 2. Win. Obtain judgment. File lien against company assets. Repeat as necessary. Offer settlement in return for not suing anymore.

Is this just wishful thinking that, if something went wrong with your paycheck, the world would be just and there would be an easy legal remedy waiting for you?

From what I understand, you can only sue for $1000 or less in small claims court. An IT contractor who is making $1000 or less per week has bigger problems.

I have good news for you: the small claims limit varies by state and, according to data from this website[1], the average upper limit is $8K (median is $6K, with the lowest being $2.5K).

[1] https://www.nolo.com/legal-encyclopedia/small-claims-suits-h...

Thanks, I stand corrected.

Separate claim for each lost day then.

Or has been done in the UK send bailiffs to the company head office :-)

Of course you'd shop around for the Bailiff company with the worse reputation and get them to send the lads who moonlight at that dodgy night club as Bouncers.

In reality it would be: spend court fees..sue in small claims court. Win. Good luck collecting in many cases. Filing for a lien? Good luck with that pointless venture...

Once you have a judgement, you contact the Sheriff to help you collect. With a judgement, you are able to take things from them in order to make your judgement. Usually once that actually starts happening, the company will cut you a check pretty quick.

In most states, the fee to file in small claims is somewhere between $30-$100. Filing for a lien with the local sheriff is another $100 or so. Well worth it for 3 weeks wages.

Cant you send in bailiffs in to recover the costs?

I don't know, does US not have a small claims court? In UK you'd pay £90 to file a case, you'd get a judgement in about a week and the company would be ordered to pay. Like I really don't understand this entire issue. Did he even ask HR to be compensated for this time? I feel like the whole article is missing some crucial information.

Yes, the US has small claims courts. They are run at the state/county level, so fees and maximum recovery values vary. Fees generally run $30-$100 and max. recovery somewhere around $5k. After winning a case, filing a lien with the sheriff is another $100 or so.

During the 1980s, one idea became paramount in business management schools, books, seminars, etc: The worker must be disposable. Any worker who becomes important to the success of the organization must be fired immediately upon this realization. If you permit that person to remain, they will become even more integral to the organization. Eventually, they will realize this and ask for more pay. Eventually, they will want pay which is greater than industry standard for workers in their position. That can never be permitted to occur (lest the industry standard wage rise), so they will have to be fired at that point. And then, the loss will do great damage. Best to fire them as soon as they become indispensible and deal with the smaller amount of damage then.

This sort of thinking kind of makes sense in a world where almost every company is a big factory company with their workers primarily doing repetitive physical labor. And the business management world has not adapted at all to the transition to mental labor. Their businesses fail and suffer for this, but as it's a universal condition (yes even among Google and such... if the company has a physical office, then they haven't even done the obvious to benefit from the move to mental work) there's not much pressure motivating change.

I had this happen for 3 months, no email no access.

I billed 40 hours a week the entire time.

> Why has this company not made their loyal worker whole?

IMO because the generally demonstrated position of the labor force is that companies do not have to (which I hate, to be clear).

IMO, employees buy into the employer/owner-propogated myth that they are "lucky to be employed." As a result, employees greatly undervalue their labor and give up any leverage to make employers do the right thing.

Having been around the block a few times, I view 'work' as a transaction in which my employer and I need to realize fairly mutual benefits: most people don't. Employees generally see jobs and titles as a way to assess their societal value. And employers, knowing that, can do pretty much anything they want to people, dangling the occasional carrot to encourage higher worker output, but mostly working w/sticks.

This is also contributes to the zero real wage growth in a supposedly tight job market.

TLDR: power dynamics and poor 'game play' on the part of employees

Yeah the company is lucky he wasn't well on his way to finding a job somewhere else at this point.

If you read to the end, he did take "the next opportunity that presented itself".

He still went back to work at the first company though. I guess this depends a lot on his personal situation as well.

Unions take a lot of (rightful) flak but this is one case where an union would possibly have been helpful.

I've had this happen, myself and an employee who I had signed the termination paperwork for shared the same first name and someone put my name in instead of his. It was hell for at least a month as different automated systems kicked in and disabled my accounts, benefits, and payments.

Worse, the first notification email happened while I was presenting to the CEO and the HR contact in the meeting had noticed half way through that I had been fired. Queue jokes of "was the presentation that bad?". No one was able to stop the machine because no one really knew all the different processes or they weren't built to stop midway.

> No one was able to stop the machine because no one really knew all the different processes or they weren't built to stop midway.

This is (in my opinion) one of the reasons why everyone talking about getting rid of dedicated IT ops in their organizations is making a mistake. You can have devs building integrations and automation all day, but you still need sysadmins who can see the whole picture and override them when necessary. Having an outsourced (or even internal) hell desk that goes off a script doesn't take care of situations like this either.

Don't worry, you can still pay outlandish prices for top tier A-Z support, where you can argue for a week with a drone that no, their system is not properly following the HTTP RFC, and here's the TCP dumps to prove it.

Imagine if we didn't have an operations engineer who knew how to trace down the problem using a TCP dump. Or if it had been a developer who was, by the proper "DevOps" hygenics, denied access to the boxes to run TCP dumps.

Yeah, they're fixing it. No, we don't have a timeline.

The worst bit is that someone internal to that business has probably also noticed that the system isn't properly following the HTTP RFC and is fighting the machine themselves to get it fixed with similar results.

I had a situation once, when Symantec firewall stopped working on my company laptop. Not the whole SEP suite, not the antivirus, just the firewall part. After 2 months I got my first angry email from robot, then I started getting them every month with big red text trying to scare me with every possible corporate hell, and it gradually started to CC it to higher and higher management in the company. Meanwhile nobody could fix it and it didn't help that IT support was outsourced to another country (I called them several times with zero results). At some point said managers started visiting me personally (because I assume their inaction was escalated even further). In the end nobody fixed this specifically, but I managed to accidentally do it when fixing virtualbox install - after increasing some obscure "Network Filter" limit in Windows both SEP and VB started working again.

I think wasn't auto fired like OP only because such trigger wasn't implemented in the company, yet. No IT support available can be really bad.

Every so often, I see a 'brilliant' claim like "99% of what IT does is routine, so it should be outsourced or even automated!"

The percentage is debatable, but the more important issue is what that 1% looks like. Because it's definitely not 1% of their value, it's the high-stakes stuff that needs a context-aware human to apply a fix or good decision in place of whatever The Machine is trying to do.

(And if you're going to keep experts on full time to handle the occasional 1% case, there's no longer much reason to outsource everything else.)

>> No one was able to stop the machine because no one really knew all the different processes

maybe the engineer that designed the system got themselves fired while they were testing the system

>>"Thomas Midgley, Jr. (1889–1944) was an American engineer and chemist who contracted polio at age 51, leaving him severely disabled. He devised an elaborate system of ropes and pulleys to help others lift him from bed. He became accidentally entangled in the ropes and died of strangulation at the age of 55. However, he is better known for two of his other inventions: the tetraethyl lead (TEL) additive to gasoline, and chlorofluorocarbons (CFCs).[19][20][21]"

Wow, talk about someone who really never learned the importance of thinking things through.

I had a somewhat similar, but much less serious experience at Google. I resigned just after I had gotten a promotion, but before the date when the promo was effective. Much to my surprise, the resignation (slated to be effective after the promo) somehow cancelled the promo via some automated system.

However, a promo at Google is a huge deal. I really wanted the promotion to go through, so that I would have that level if I decided to re-join Google (or even so that go/epitaphs would match my resume). My manager and HRBP managed to get it sorted out, but it was a pain.

This is interesting.

On the one hand, you have been promoted to a new role, so you have reached a compatible level of expertise. You can put that role on your resume and sell yourself.

On the other hand, you never, ever actually performed in that role, with the new responsibilities. How can you list that on your resume !

I am not sure what is the right answer here... but as your new employer I would take your last promotion with a big grain of salt. A resume is not a score sheet of levels accomplished in a game, it is a list of things your have actually done.

At least at Google, the whole point of a promo is that you've been working at the new level for some time before the promo cycle. And the new employer was not really in the equation, I'd already received my offer from them.

I'm not particularly familiar with the system at Google, but at any somewhat decent tech company in order to get a promotion you have to already perform as you would in your "future" role for some time, with higher-level expectations and responsibilities.

For example, is someone with level I is up for level II promo, she/he has to match level II criteria long before actual promotion is due.

That's general HR pep talk.

In reality promotions mostly happen in organizations due to political lobbying with the powers.

Even in these 'decent' tech companies you will see some people getting rapidly promoted and moving up the hierarchy, while genuine performers are stuck in the process and minutiae. Its just what kind of leverage your manager has with the upper management.


I think everyone should at least experience once to what length a company can go in order to keep someone they really need that threatens to leave. It happened once to me: in a 150k people company that had a well defined promotion model very similar to Google. I gave my resignation notice and suddenly all the HR pep talk was out of the roof. I gained two levels and was promoted to director level.I still left but felt stupid I didn't threaten to leave earlier.

It made me realize that there are two types of workers. The ones that will play fair game and believe the HR pep talk, as we just saw in the previous comments, and the ones that realize that the fastest way to go is to bypass this and play politics in order to fastrack it. It is another type of skill.

A big thing to note in these political systems. People tend to emphasize your interface to the systems as 'Company thinks X about you', 'Company values your work' etc.

Companies are not living systems and in general its just people making decisions.

I've been walked over many times now, but once, after all the work, I was nominated for an award the company's annual meet. My manager, his manager and all the way up assured me that based on what I had done the award was coming my way. I was even asked to prepare a small speech to give on the stage, they even asked for a photo to put up on slide deck with a small bio.

Two days before the meet, my manager and the director called me into a meeting room to tell me that I wouldn't be getting the award, and they didn't want it to be painful surprise to me during the meet. And they had tried everything they could.

Eventually my manager told me during lunch later that big time political lobbying had gone into this, and VP making decisions had no option as he would be cornered politically on other issues, if he didn't relent to demands of rewards from other corners.

Google or any other company. Performance has nothing to do with how you get paid/rewarded in any company.

Curious, do Google regularly give free stuff to their customers (advertisers) hoping that in a few months those customers will upgrade to a higher service?

Yes, they have free trials of many things.

Which is dumb, because it means that the company is getting a higher level of labor than they're paying for.

There is a very clear right answer here provided you consider promotion from a clean point of view.

A promotion is a _recognition_ that you are _already_ operating at a given level. It's an acknowledgement by TPTB that you deserve a given title.

Anything less than this leads to a result where you may not actually be performing at the level the new title requires, which means you could conceivably be promoted and immediately fail to meet expectations in the new role.

> promoted and immediately fail to meet expectations in the new role.

aka the peter principle (https://en.wikipedia.org/wiki/Peter_principle)

At Google, you ostensibly must have been performing at the new level for at least six months in order to get the promotion.

The span at the level lower than your performance is accommodated by an excellent bonus and stock refresh.

> On the other hand, you never, ever actually performed in that role, with the new responsibilities.

For what it's worth - that's not necessarily true. Several places I have worked gave promotions in retrospect - after you have been performing the additional responsibilities for a while. It's sort of like a 'dry run' - they want to make sure you can do it before officially thrusting it on you.

What made you think that you were entitled to something that you were supposed to receive on a date which was later than the date on which you resigned?

I resigned effective the 2nd of a month, and the promo was effective the 1st of the month. See above "resignation (slated to be effective after the promo)". So the resignation was scheduled for the day after the promo.

I actually stayed an extra week to receive the promo.

I'm not going to lie; that type of thing -- resigning right after getting promoted -- seems like something from some alternate reality fantasy world.

Why? Top performers are poached all the time. Plenty of people do excellent work in jobs they may not love. Both promotions and bonuses are usually tied to performance review cycles, and right after bonuses vest is the most popular time to leave.

It is entirely possible they weren't planning on getting promoted.

I was due for a performance review about a week before I handed my notice in at my old job. The review didn't happen because my boss was incapable of communicating with his employees and actually scheduling appointments with them, but had it happened, I was fully prepared to push for a payrise, even though it would only be a couple of dollars an hour, and for only 3 weeks.

You wouldn't turn down a promotion because you know you're about to hand your notice in. You act like everything is normal and you're going to be working at the company for the foreseeable future, right until the day you hand your notice in, which should be at the minimum notice period.

Otherwise what would happen if you decline a promotion, then your new job offer falls through? You'd be left looking like a prize dick.

I had a similar thing happen, except the company was acquired and we thought we were being laid off. 30 days after I started with my new employer, I received notice of being equated to a higher rank in the new company, and a very significant stock grant.

Too late though. :)

It's not that uncommon - promotion takes so long and has so many moving parts - you might as well interview somewhere else, since if it doesn't succeed you'll just be doing the whole thing again in six months.

I have an ex-colleague that was promoted 2 weeks after she resigned, just days before she left the country. The announcements came in the right order, first leaving the company and later being promoted. It puzzled lots of people, but this stuff happens.

One manager told me, he never tries to promote job hoppers no matter how well they perform.

They don't stay, and you end up wasting a perfectly valid promotion while some one who would stick longer with you could have got it.

except that this doesn't work in a high demand market such as the Bay area right now!

Your old manager may have said that to encourage you to stay, but at the end of the day he will promote whoever needs to be promoted in order for that person not to leave. It is all very simple offer and demand at the end of the day.

I was once in a company where the CEO liked to brag that in this company everyone is fairly paid in regards to their competences. You can guess what happened next, we went to have a couple beers with colleagues one evening and started talking about salaries. We realized that we had salaries that didn't really match what the CEO was pretending. The laws of the market are simply too strong, you will always need to sometime pay someone a lot because of circumstances or because that person had another offer to match.

At the end of the day, companies benefit a lot from information asymmetry (they know everyone's salary, but you know nothing). They can claim a lot of things without you having the ability to verify it.

Wouldn't that give incentive to not stay?

Doesn't that exacerbate the problem?

> somehow cancelled the promo via some automated system

Maybe someone just unassigned your employee id to the role, because in reality you weren't actually taking the job!

You didn't answer the question at all.

You should have become a lawyer.

Your story is inconsistent?

> I resigned just after I had gotten a promotion, but before the date when the promo was effective

If I get sick before my insurance is effective...I don't get my coverage.

If I resign before my raise is effective...I don't get my raise.

The point here is not the raise, but the promotion itself. It's leverage for joining other large companies ("I was a Senior SWE at Google, so you'll need to offer me a Staff SWE role"), and as the OP notes, it lets them come back to the new level without having to go through the promo gauntlet again.

If you cancel the insurance effective from next month, you have coverage until the cancellation date.

Exactly. If you resign before your insurance coverage monthly renewal is effective, you don't get your insurance renewed.

All of this is what "effective" means.

Maybe we disagree on the sequence?

As I understand it, it goes like this:

  1) Employee gets promotion, effective in the future (at #3)
  2) Employee resigns, to be effective after promotion (at #4)
  3) Promotion is to be effective at this date
  4) Resignation is to be effective at this date
In my understanding, the employee's resignation is to be effective after the promotion it to be effective, so that the they would resign with the new/promoted level.

I understand the surprise.

"I resigned just after I had gotten a promotion, but before the date when the promo was effective"

I had interpreted that as being resignation effective before the promotion. But I was probably wrong.

Because large companies give the promos as trailing indicators of performance, meaning that to be promoted requires an acknowledgment of prior sustained performance at that level. Entitlement to the level is actually accurate and above board.

A promotion can also be a way of acknowledging an existing reality.

Generally in cases like these, they don't want to waste the promotion(the new position) and they give it to whoever is next in line and wants to stay longer.

A very logical thing to do. In big companies it takes time to build a case for a promotion(position). If you were not going to use it, it was always a good thing to give it to somebody else.

Plus asking somebody to sustain a position for somebody who was promoted and still wants to leave seems like bonkers even from the HR perspective.

Google do not have position quotas. At least they didn't have them when I worked there a few years ago. So getting a promotion is not done by competing with others for the same slot, but by earning the merit from doing a great job.

What is "go/epitaphs"?

An internal Google site that let's people see who has left the company and, occasionally, why.

Err isn't that a potential huge legal liability

I don't know. What do you imagine someone would sue for?


Yes, because getting a raise that is more than 50% of an average full-time worker's entire yearly wages is not a big deal at all.

When a previous company was acquired, everyone got a new job offer generally better since the company wanted everyone to stick around. Well except for one guy. A junior engineer in one of our teams. He was a really good engineer too so we were shocked. It turned out he had the same first and last name as another person at our company so they thought it was a duplicate entry and omitted it. Eventually a offer was prepare after a week of escalation.

Heh, that's the real problem actually with buggy name-based systems. In my current company we have a person officially called "Name Surname 2" for the same reason.

A lot of state DMVs and other government offices apparently use name + birthdate as a "unique" identifier, which works about as well as you'd expect. It doesn't usually cause problems, because both people are accepted as 'real', but it's alarmingly easy to end up with someone else's license suspensions, voter registration, or even credit score.


Because Name Surname 1 was already taken!

Because Name Surname 0 was already taken!

No way they were forward-thinking enough to use zero-indexing.

Don't US payroll/hr systems use SS number its one of the few cases where it makes sense to use it

Was this is the US? There are benefits to having a national, public, unique ID # for each person and this is one of them; it reduces the number of 'they had the same name' mistakes to nearly zero

Doesn't a US employer have their employees social security number, which would fulfill exactly this role if people cared to look at it?

I'm not that knowledgeable but I thought SSN aren't public information and that they are not used throughout information systems to identify people because they are 'sensitive information'

They aren't public, but so many places use it as an identifier. It's kind of dumb for something so sensitive.

Social security numbers are not unique, but combine them with a birthday and name, and it gets pretty close.

Social Security numbers are unique. They don't work as a great identifier because they can be mistyped, can be hard to verify, and people don't want to give them out too much.

SSNs should be unique, the only exceptions are fraud or errors.

According to the Social Security Administration:

> At its inception, the SSN's only purpose was to uniquely identify U.S. workers, enabling employers to submit accurate reports of covered earnings for use in administering benefits under the new Social Security program. That is still the primary purpose for the SSN.

They are not. SSNs can and have been reused

There have been cases where the same SSN was issued multiple times on accident, but they haven't been reused on purpose.

Isn't an SSN only 9 digits? Seems inevitable that you'll hit reuse (not now, but probably before the end or even middle of the century)

I had a similar experience but in reverse.

About six months ago I left my long-time enterprise-y employer for a startup in another city. So we sold our house, packed up the family, bought a new house and moved in.

After 3 months it became obvious I had made a terrible mistake, so we sold the new house, packed up the family again, and bought another new house in the same city we had just left 3 months ago so I could return to my previous employer. They were happy to have me back and I ended up in the same desk and chair I had just vacated 3 months previous.

Only it took awhile. Since I was already in the system as a terminated employee it required manual intervention and code changes to the employee management system to get me added back in. It took about 3 weeks before I could do anything besides go to meetings.

If anything needs disrupting, it’s the employee management systems in use by pretty much all the large enterprise shops. It’s a mess.

Your company's self-image must be either very high or very low: "nobody who quits us will work for us again!"

I think it's just a failure of imagination. Granted, very few folks leave here and no one can remember anyone leaving and coming back. But the team that runs the machine now knows of this edge case and are addressing it.

Maybe next time just try out things at first? i.e. rent, move the family into the rented home after 2-3 months without selling the original house for some time.

Sure, but the GP wasn't asking for life advice.

It's not an advice but things to bring on to discuss why they did the otherwise.

Yeah, my wife would agree. ;) This is probably the most epic failure of my career. I thought I did proper due diligence but obviously I didn't. A learning experience for sure.

It makes sense that the HR IT industry would have some of the worst software developers. It seems obvious to me at least that you can't rely on first name + last name to make a match, and if you do, you have to write code that fails if there are multiple users with such a combination, as well as falling back on soundex or double or triple metaphone as well as nickname support. Second, the screens involved in termination should display warnings if multiple employees have similar names and departments, etc...

I've been in health IT for the last 15ish years so I know a lot about patient matching (wrote at least 4 patient matching MPIs in my time at different companies).. so maybe it's just my perspective..

However, that all being said, almost all processes like this are NOT automated, but instead handled by automated emails telling people to do operations. You're basically then subject to the lowest common denominator logic of people in IT that are in charge of disabling accounts- with no regard or care about who the person is.

Anyway to the person that got fired like this - just be happy you are out of the environment - it is not for you. There are signs in life.

I think you misread the article.

The author assumed that the recruiter accidentally thought that he had been fired after reading the list of fired employees, as there were other employees with the same name.

In reality, he had been fired as his contract had not been renewed. He wasn't actively fired, but rather his contract expired because someone failed to renew it.

A big part of the problem here seems to be rather that there were no warnings of his impending "departure" to the correct people. There were no emails to his manager reminding them that his contract was ending, or to the author himself. Even if it was intended for his contract to end, it would still be a good idea to have that email sent out, just to remind them. I can totally foresee somebody forgetting that their contract is finished.

I think you misread the article.

The author's contract hadn't even ACTUALLY expired. The end date was wrong on the system. That doesn't actually affect his contract.

The article says that his former boss, who left the company, neglected to file the paperwork on extending his contract. There was intent to extend, and everyone thought that’s what happened, except for the fact that it didn’t actually happen.

There's the contract that was signed, and then there's the HR documentation as it appears in the system. The contract was signed, but the HR documentation wasn't.

Why would HR IT have worse software developers than other industries?

Because HR in general has worse employees than other disciplines?

Seriously, in my 20+ year career, I've run across very few legitimately talented people in HR. The department seems to attract the kind of people who have little to offer intrinsically; who happen to be willing to facilitate the necessary protocols, as minimally effective as possible, while claiming "people skills" that end up being little more than office politics.

It’s true in my experience. HR IT, IAM, etc. are dominated by fakes and low skilled workers. Corporate IT is often the same but for some reason HR IT is distilled mediocrity. Definitely some exceptions and exceptional people but they are used and abused by the frauds and leaned heavily on by the unskilled.

I’ve had the joy of being in meetings with folks from every discipline that don’t know their field but HR IT takes the cake. Somehow they find a way to absolve themselves of responsibility by leaning on more technical teams while simultaneously touting their unique technical expertise and importance they use as a club to ignore those very same teams.

Having known a couple of founders who built their companies to a greater than 5000 person company, this is actively done. Founders need control over who they employ, promote, fire, etc. While it doesn't matter much at the entry level stage, as you move up the seniority chain, there is a lot more politics. HRs major purpose is to make sure that things that are done to employees are done in a legally justifiable way. If management wants to see someone gone, they make sure there is a paper trail. If someone needs to be moved up faster, they facilitate that. Essentially, they are there to do upper management's bidding, not to employ their own thoughts. Headstrong people don't survive there.

In my experience HR isn't filled with "bad" employees per say, it's filled just with bureaucrats. They care more about following policy to the letter just to do it rather than actually thoughtfully applying policy which generally involves following the spirit of the law by bending rules rather than the letter of the law.

Two reasons really. HR is not a profit center, and no one who picks the software will actually have to use it. Zed Shaw takes this up to 11 in a presentation he did in Montreal https://vimeo.com/2723800

I had to do a back transaction once in Greece and they did ask for my mother's and my father's name. Strange for a German but it seems to work for them.

Isn’t that a matter of public record in most places?

" I had missed 3 weeks of work by that time, and pay."

The what now? I can only assume this is in US, because in EU he would be 100000% entitled to pay for every single minute of this going on.

Maybe not if he is a Contractor.

Even in such case contract usually has a clause about premature termination and related compensation.

Did you read the article? There was no premature termination here. He was “fired” exactly because his contract expired.

It’s a shit situation and the company should have made him whole, but on ethical not legal grounds.

In many EU countries a contract does not actually end if you continue to show up to do work in mutual (implicit) agreement with the managers at the company you work for. It would constitute an implicit extension of the contract, effectively renewing it under the same terms for the same duration (with a maximum of 1 year IIRC).

I can vouch for this. As a freelancer I actually had exactly this happen: I was working for a client with a time-limited contract. The contract expired on New Year's Eve and they failed to renew it so I expected to have been terminated.

One week later they send me an e-mail asking when I'll be back from vacation. They refused to renew the contract (which explicitly said it required a formal written agreement to be renewed) but wanted me to keep working for them. So I went back to them and they kept paying me.

Several months later we had a disagreement and they claimed the terms of the original contract no longer applied because they hadn't renewed them. But my lawyer confirmed that in fact the contract was still valid because they implicitly extended it and the requirement for a formal renewal was invalidated (they failed to provide a new contract so the extension was still subject to the terms of the initial contract even though it had expired).

We didn't go all the way in court so I can't say whether this would have held up, but they ended up reaching an agreement with me where they pretty much gave up on everything except my legal fees, so I guess their lawyer wasn't too hopeful.

EDIT: For context, this happened in Germany.

By mistake therefore the employers is liable for breach of contract.

You cant get out of "frustrating a contract" by saying its a mistake

I think I would get a compensation bonus without even having to ask out of this (in the EU). Since he got a new contract, I hope he did negotiate a bit on the new terms in that contract.

Would have been clever to negotiate the missing three weeks pay into the new contract, e.g. as a signing bonus.

This is literally an episode of Better Off Ted.[1] In it, the titular Ted is inadvertenly deleted from the company system when trying to correct a misspelling of his last name. Eventually, he is forced to interview for his own job as the system had already put out an ad for his replacement. I think the most striking part of it, and of the true story from the post, is the human factor - the idea that the humans involved looked to the system as an authority and followed its orders blindly.

I wonder what other examples there are of people blindly following technology - people driving into lakes because their GPS told them to, etc. Plus, as our society gets more and more dependent on these systems, we may lose out on the flexibility that human mediators and problem solvers once gave us. The human tendency to defer to authority may never be as terrifying as when that authority is held by an uncaring machine with a couple bugs.

What was once satire has become too real.

[1]: http://betteroffted.wikia.com/wiki/Goodbye,_Mr._Chips

That show was incredibly on the mark, if Black Mirror was a comedy it would have been that show.

The episode about the black engineer who isn't detected by the motion sensors is basically straight out of HP's webcam fiasco[1](although the show takes it to the logical and hilarious extreme).

[1] http://newsblogs.chicagotribune.com/race/2009/12/hp-webcam-c...

Here's that episode on Hulu for anyone who happens to have a subscription and want to check it out:


In my opinion, BOT is one of the best comedy shows of the past decade and that episode is widely considered to be the best one.

This has been a real problem with facial recognition. Faces used to train engines are typically Caucasian, so the ability to distinguish other ethnicities is quite sub-par.


Isn't there a problem with shadow contrast which makes features more difficult to pick out on darker skinned faces? I vaguely recall an East Asian developer giving that reason for why he developed on Western data-sets.

Wouldnt it make more sense then to develop _not_ on the easycase but the hard ones?

Yeah but, you know, deadlines to meet and contracts to win.

gives the best performance metrics as well

MVP presumably.

Most likely it’s an issue with contrast and overall darker skin colour.

Can be solved by illuminating the face with IR though, which should work across all ethnicities.

Yes, but then you need special equipment rather than standard cameras (that usually filter IR); which would prove the point that it's about physical limitations rather than racism.

The racism comes from saying "We recognise human faces" and then not recognising a significant proportion of humans.

That's not racism, that's laziness. Or perhaps just being a bad engineer, if you want to be more cynical about it.

It's not that hard to find both conscious and subconscious examples of systemic or individual racism. Engineers taking the easy path with webcam facial recognition is probably not a good one and serves only to give more fuel to those who claim people jump to the racism cry too quickly.

If you ship a product because it only works for white people and that's good enough, that is definitely racism. Just as it would be sexism if you shipped a voice recognition product that only worked for men.

Laziness and racism aren't mutually exclusive. Historically in America, white people haven't considered black people as fully human. You can read the various declarations of secession or the 3/5ths compromise for that. Or the long post-Reconstruction history of slightly more subtle ways.

Sure, laziness was involved here. But deciding a product was good enough to ship without caring that it worked for black people requires the effective belief that black people didn't really count as people. At least, not people that mattered. Imagine the reverse: if the product didn't work on white men, would it have been shipped? Of course not.

When laziness just happens to have a blatantly racist outcome in a place where there is a centuries-long history of racism, Occam's Razor suggests the explanation is racism. If laziness could not have caused the bad outcome to happen for white people, then it's pretty clear that pure laziness is not the real cause. It's instead white people being differentially lazy when it comes to black people. That's clearly racism.

As a confirmatory example, look at the American justice system. In a lot of places and times, the same nominal laws applied to white and black people. But they were enforced very differently. Serious crimes against black people were ignored. Minor infractions by black people were enforced vigorously. [1] Were the cops lazy? Sure, everybody's lazy sometimes. Was that why there was a racially different outcome? Definitely not.

[1] Examples of this are all over Loewen's "Sundown Towns" for example.

I think it's important to start talking about, and acknowledging, that nearly every second of the average person's life is thinking about something that is within their own universe of their life's experiences.

So plausibly, this really was an innocent and understandable mistake. Maybe they grew up in a town full of white people, and maybe all of their friends and coworkers are white. I understand what they did is technically racist, but that's because of that particular combination of people. Let's use _this_ 'experience' of "racist", but not it's exact definition. For example:

- It was racist you forgot your mother in law's birthday (who you see once a year.)

- It was racists you didn't lift the toilet seat. (Edit: I grew up in a culture where it was offensive to not raise the seat)

Definition: Any unintentional side effect from not thinking about someone* outside of everything you've experienced, and presumably causing harm to that outsider.

* Usually this person is disadvantaged in some way. But almost everyone is disadvantaged in some way, thus neutralizing this particular point IMO.

Does that sound rational? I don't think it does.

The world is full of suffering, and I'm advocating people help and love the people they are surrounded by. If many people did that, it would be easier to recognize each other, where ever you come from. Why? Because hate will push away everything it is unfamiliar with, but love will accept everything it is unfamiliar with.

(Now meta argument, do I expect people to change? No. So I try accepting them instead.)

Again, something can be racism and something else.

> Maybe they grew up in a town full of white people, and maybe all of their friends and coworkers are white.

This is not an accident. All-white contexts in America are the result of personal and systemic racism. People who grow up in those contexts are, unsurprisingly, more likely to be personally biased and will regardless do things that further systemic racism.

People in that condition have a choice: they can either overcome their upbringing or they can continue to support white supremacy. If they do the latter, well, then they've made a choice. You can accept that if you want. I don't.

I don't think it's likely that all the folks in Vermont are racist or that the system of governance there is supremely racist - yet Vermont has an absolutely tiny percentage of non-white folks (3.1%)

Less than 15% of the U.S. is African American so it's not at all surprising that there are areas where there are very few black folks as a result. That's nobody's fault and shouldn't be described as racist because, well, it isn't.

3.1% is about 1 in 30 people. That's not that small. And this isn't 100 years ago, I'm pretty sure people in Vermont have seen a black face on TV/film.

This is historically ignorant. I don't blame you personally, because the American education system is very bad at covering America's racist history. But you're still basing your estimation of "likely" not on real data, but on what you'd like to be true.

If you'd like to learn more, start with Loewen's "Sundown Towns", which describes how hundreds and possibly thousands of towns across the US were turned and kept white:


These were known as "sundown towns" because non-white people had to leave by sundown or face violence: https://en.wikipedia.org/wiki/Sundown_town

These came about not because there were never any black people, but because white people indulged in violent ethnic cleansing. This happened most prominently during the Nadir: https://en.wikipedia.org/wiki/Nadir_of_American_race_relatio...

He proves this quite clearly through census data, where places that used to be racially mixed suddenly got white and then stayed that way for decades.

He doesn't give any specific Vermont examples, but Vermont had a prominent Klan presence in the 1920s: http://vermonthistory.org/research/research-resources-online...

Later, as suburbanization happened, white supremacy was maintained more subtly, through means such as racial covenants: https://www.sevendaysvt.com/vermont/discrimination-in-deed/C...

It's also not necessary for a lot of white people to be actively racist to preserve white supremacy. As is clear from Loewen's book, all it takes is a small number of people willing to be active (e.g., making threats, burning crosses) to drive off new black arrivals. The majority can be passively accepting of a racist system without lifting a finger.

But that doesn't mean they won't end up biased. White people who don't know many black people are more likely to be biased, and all-white towns produce people that are more likely to have racist views. (For more, see Loewen's Ch 11, "The effects of sundown towns on whites".)

This is neat, in a conversation about the technical aspects of image recognition with webcams in low lighting conditions, you're talking about secession and the 3/5th compromise. Let's try to stay on topic.

> But deciding a product was good enough to ship without caring that it worked for black people requires the effective belief that black people didn't really count as people.

Nobody ever said "oh it doesn't work with black people? Who cares LOL." I was going to make a crack about mental gymnastics but this isn't even that. You're just making shit up at this point.

> When laziness just happens to have a blatantly racist outcome in a place where there is a centuries-long history of racism, Occam's Razor suggests the explanation is racism.

No, even using your sentence here you admit the cause is laziness but then try to reframe it to racism because that fits your world view. Just like the fact that we're talking about webcams and you're talking about the 3/5th compromise and the American justice system.

Is there institutional/structural racism in the US? Absolutely. Webcams aren't a good example of it, sorry.

I do not admit that "the cause" is laziness. Laziness is part of the causal chain, sure. Single-cause thinking is a poor way to do a failure retrospective.

We don't know whether or not they knew it didn't work with black people before they shipped it. Maybe they did and dismissed it with technical hogwash, as you are here. Maybe they did and decided it was an ok flaw. Maybe they didn't know because the company doesn't have any black employees and nobody though to test it on black people because the people in charge don't know any. Or maybe they do and just don't care. There are many possible paths to this decision, but any of them demonstrates systemic racism.

And this is on topic, because the broader topic is social implications of technology.

As an aside, your attempt to narrowly define the topic so racism can't even be spoken of is a pretty typical move from people experiencing white fragility: http://libjournal.uncg.edu/ijcp/article/view/249

Seeing someone interact with others after drinking enough kool-aid to fill a pool is actually pretty entertaining. Thanks, I needed a break from work.

Speaking of work, have you tried testing your definition of racism on non-white subjects and products, such as Black Entertainment Television? It doesn't hold up very well.

My definition of racism is pretty standard among people who study it. I'm talking about a societal power system, like capitalism or communism. The US was undeniably racist to start out; black people were property, and only white men could vote. The level of white supremacy has declined some since then, but hasn't been eliminated. And it has been going back up in the last few years.

You could mean two things by "testing your definition of racism on non-white subjects": in the US or elsewhere. Either way, yes, of course I have. When going elsewhere it's necessary to speak of "the dominant racial group", and that can be helpful here too. So taking BET, we can ask questions like, "What effect are they having on non-dominant racial groups?" And, "Do their actions tend to increase or decrease the racial power imbalance?"

I'm not much of a TV person, so I'm not familiar with their shows. But I took a look: https://www.bet.com/shows.html

My first impression is that they are having positive effects on a non-dominant racial group, and are acting to decrease the racial power imbalance. Therefore, no, they are not racist.

Since you are anonymous person with a new account turning up to be a dick about the topic of racism, I presume your question was entirely insincere. But just in case, I hope you find the answer useful.

> That's not racism, that's laziness. Or perhaps just being a bad engineer, if you want to be more cynical about it.

It can be both. An attitude that considers facial recognition software acceptable if it only recognizes light-skinned people is both lazy and racist. Only choosing to test on light-skinned people is racist because doing so assumes dark skinned people are an exception or an outlier, rather than an equally valid part of the set of "human faces."

Whites have dark skin at the end of each summer.

True, but that doesn't really contradict the premise of my comment.

The assumption of ones own race being the default color for humans (rather than an arbitrary value) that leads to facial recognition software being trained on a data set so narrow that the resulting software can't even recognize skin colors that diverge from that norm is as much about implicit, albeit unintentional, racial bias as it is the particular technical problems involved and the need for engineers to cut costs and meet deadlines.

It's not entirely racist, but racism is a component.

> That's not racism, that's laziness. Or perhaps just being a bad engineer, if you want to be more cynical about it.

I wasn't ready to admit the genius of the show before, but I am now: what you said is quite close to a line from the same episode (said by the sociopathic boss, Veronica) when told by the protagonist that the sensors were effectively racist "[...] it's actually the opposite of racist, because it's not targeting black people. It's just ignoring them. [company] insist the worst people can call it is "indifferent.""

I agree. It's a well known problem where the training set isn't representative of the underlying population. While it can certainly be argued that the engineers should have recognized this deficiency and taken corrective action, I really don't understand why all the respondents to your post are so quick to assert racist intent based on clickbait headlines from Forbes.

It's not an active racism on the part of the engineers. It's way more subtle than that.

There could be a hidden assumption on the part of the engineering team that light-skin is "normal" and anything else is a special case. Nobody is saying "I hate black people" or anything of the kind.

Or, as happened with a voice recognition system a former employer used, it was tested on the engineering staff, who happened to be all male. As a result it didn't work well for most women who tried to use it. There was no intentional exclusion of women from the test data, and I'd argue, no intentional exclusion of women from the engineering teams. But it is reasonable to say that systemic sexism that excludes women from engineering careers helped this system fail.

These kinds of problems are difficult to solve, because they aren't active decisions on anyone's part. They evolve out of pervasive conditions, and unconscious biases. At root, the source is still racism, or sexism (or another form of discrimination).

Because it is racism. It is ignoring a large swath of humanity based on the color of their skin. You may not want to think of it as racism because it's not the burn a cross on their lawn type of racism, but it's still systemic racism.

> It is ignoring a large swath of humanity based on the color of their skin.

No, it's an insufficiently sensitive contrast filter combine with too narrow a training set. Screaming racism at everything that is even marginally approaching the topic of race detracts from real racism and ignores the actual issue here - shitty software.

But laziness around something that dispropotionately impacts people based on their race is racist. If you know your system has a harder time training against non-white faces, and you choose to train it only against white faces to be lazy, that's still explicitly racist. This isn't even a case of negligent racism ("oh well I didn't know my system wouldn't work for people with darker skin because only my white coworkers tried it out"). The example here is a case of the engineer explicitly deciding to avoid the harder case of darker skinned individuals, knowing that the results would be poorer for those individuals, and thinking it doesn't matter if the system doesn't work well for people with darker skin. That is explicit racism.

Skin color is not synonymous with race. Every race has people that span a wide range of skin tones, and skin tone fades with age, so you may as well be arguing it's ageism.

That's not racism, that's laziness.

If the system had failed for with people with different hair color rather than skin color would they have been equally lazy?

If it was hitting the same percentages? Very likely yes.

What else would they have been?

Actually, even if it is accidental or subconscious, I'd say this is a perfect example of systemic racism--racist behavior (I'm referring to an algorithm/device specifically here, which you can safely call objectively racist) which is normally within society's acceptance level which, when magnified to a societal scale, is no longer acceptable. Another symptom of this illness is that you might not even notice as a member of this system the system is broken if you're white, but you would if you're black.

Personally, I think it could be understandable people didn't consider race when developing facial recognition technology, especially when we've only had mainstream awareness of this for under a decade and many people live in racially homogeneous or dominated cultures. However, I don't think it's acceptable for organizations, and the time when you can safely say you didn't understand biased learning data will be over soon. There are considerations you need to make scaling your tech from personal project to something the public will consume.

Also, the day will come when computers can point out racist stuff better than the average human can now, albeit with a high false positive rate. I say this because it's relatively easy; even if you only count a subset of tweets talking about racism as not trolling, that's still a shockingly high number of meaningful things about the world many people aren't seeing.

Let's just say that if a system didn't recognize white people's faces, it probably wouldn't be viewed as "done".

If it's not worth the trouble to see if your system works for non-whites, that's beyond lazy.

Like the soap dispensers that don't work for dark skinned people.

It is racism to realize the negative aspects to other races, nevertheless pitch it as universally functional, and push the cost/pain to other races. Laziness is some of it, some of it is the realization that only other races bear the brunt of the cost.

If it doesn't occur to you that there are black people, or you are too lazy to test against then, then that is racism.

No, that is racism. It's not burning a cross on someone's lawn racism, but flat out ignoring a large swath of humanity is racism.

Most cheap cameras actually don't filter out IR; it often shows up as blue or purple. You can use this fact to see if an IR remote is working -- just shine it at your cellphone camera or webcam. Whether there's enough IR sensitivity to be able to illuminate a face is another question.

I find this phenomenon of protecting oneself from being called racist fascinating. Your opinion is not exactly racist, but you had to bring up another minority, in case someone accuses you?

I loved that show so damn much. I was crushed when it was canceled.

Black Mirror IS a comedy. Nothing in it is realistic.

Don't take it seriously.

Just because nothing in it is realistic, does not make it a comedy.

Just because two sentences are next to each other, does not make it conditional on the first.

You're right, it's not always the case, but in that format the second sentence is usually a justification of the first. Also, Black Mirror is not a comedy.

Every time I watch it I'm laughing my ass off. It's definitely not serious, or even satirical.

Most sci-fi has the problem of trying to model a situation with limited social scope. The fact that they never turn out true is because society has social safeguards to prevent it from happening that the writers don't model.

1984 is also a comedy.

We call it a comedy when it _intends_ to make people laugh. Just because _you_ laugh about it, doesn't mean that it also intends this. I know you could now say that they actually intend it. But I doubt that. (X)

Being absurd is being intentionally comedic.

Reminds me of Frank Herbert's ominous (fictional) prophecy:

“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”

Sounds like what Brecht once wrote about the luddites, something along the lines of: "you appear to be dominated by machines only because there are men hiding behind those machines" (can't retrieve the actual quote now, was on a treatise on theater)

Time for Mentats! The Butlerian Jihad seemed so unrealistic when I first read Dune, but now it seems prophetic.

I have noticed that people following machine orders is also used by corporations to force customers to give up on their rights. I have a minor dispute with one of our communication providers and it is a real nightmare - according to the law I have the right, but the system does not include it and people don't know what to do in this case. Some other people higher up just wait until I give up.

A leasing agent for my apartment put the wrong (slightly cheaper) rent amount on our lease, and only realized as we were moving in, well after all parties had signed.

She told us we needed to sign a corrected lease because there was no way to make the rent billing system charge us the lower amount.

We explained to her that the building’s legal obligations trumped what their computer would do. They figured out how to change the amount in their billing system.

I encountered that several years ago. Federal law says that if you request a cable box with an enabled Firewire port, cable companies are required to provide it. I requested such a box from my local Time Warner cable TV company. They had no clue what I was talking about and said their software had no facility to let them enable the Firewire port on my box. So I called the State Utilities Commission in my state. I explained the situation, and that the cable company was violating federal law. The next day I got a call from the owner of the local Time Warner franchise personally. They managed to find a way once they were in danger of having their license to operate in the state revoked.

"Make game of that which makes as much of thee." is a quote from a book thousands of years old (The Rubiyat by Omar Khayaam) but it is as applicable as ever.

Contact the FCC: https://consumercomplaints.fcc.gov/hc/en-us

My wife is a victim of the OPM breach and had someone apply for (and in some cases open) numerous accounts with her data. One of those accounts was with T-Mobile and after three months of phone calls and strongly worded bills we filed an FCC complaint. A T-Mobile rep called us two days later and the issue was resolved (or so it seems anyway, haven’t heard anything in the past few weeks).

Don’t pay the invoice until they either fix it or let it go (they can’t go to court as you’re in the right from a legal point of view).

That’s what I always do with shitty utility providers - it’s funny how the “computer says no” excuse suddenly disappears when they start to be out of pocket.

Careful, automatic processes can give you trouble, cancel your service or hurt your credit rating.

I recommend complying with the process, mentally account for the extra costs when you sign up, stick to defaults and avoid complexity whenever possible.

On the other hand nowadays companies do respond to complaints much better than a few decades ago if you file complaints through their websites. Taking it to social media may also result in quick and drastic measures.

> cancel your service

By the time I arrive to this stage, I’ve already got a replacement lined up and all it takes is to flick a switch.

> hurt your credit rating

From experience, the problem eventually gets resolved and they cancel any black marks on your credit report. But personally I never cared about mine anyway.

The other thing about credit reports is that 1) companies not in the business of providing credit have an uphill battle to claim that you're indebted to them, and 2) you can challenge such things, and the law requires them to show evidence.

Due process has favored debtors for some time now, but the information isn't published, leaving shady "credit cleanup" companies to charge hundreds to do what you can do in many cases simply by logging into a website.

that could pose a problem in some European countries as it is very easy to get a court order (no humans involved, it`s an automated process) to collect the money from you. it would be your duty to prove you don`t have to pay.

Interesting, as the duty of proof is the other way around in the US, but usually it seems to be the EU that favors the little guy.

the burden of proof would be on the plaintiff`s side, but as I said as it`s automated the defendant has to act, if he fails to do so in a certain amount of time the court order becomes effective

That’s very common with badly designed anomaly detection systems. Customer support say the system has detected something wrong in the customer behavior and ask, often order, to stop or fix whatever they’re doing. Neither customers nor support are able to intervene



So true. Recently I was at the pharmacy, the doctor wrote a wrong prescription and my wife was in the car having a hefty migraine. They wouldn't give me the drug even though they could see she has been using it for years and the wrong prescription could be solved afterwards. I think such people will be the first to be completely replaced by robots, I for sure wouldn't notice the difference, in fact I expect a robot to be inhuman so it would be less frustrating.

Okay this example makes sense. No prescription == no drugs (especially for painkillers and other things that people abuse).

If you think this is a bad thing, then it's an organizational problem and has nothing to do with computers.

If you are a human and you can see in the computer that the prescription is regularly updated and you can even call the doctor if you doubted it. You can even check the prescription afterwards and if my story doesn't check out, do something about it. The migraine was pretty bad, imo it was like not treating a broken leg because of a missing insurance card or something.

There are very good reasons why a doctor might decide to change a prescription for someone, and listening to the patient over the doctor in some of those cases would not only be a bad idea, it could be potentially fatal or cause long lasting problems.

For example:

- Evidence of abuse of a drug.

- A second prescription which interacts badly with the first.

- Changing health circumstances (pregnancy, some sort of deficiency, failing internal systems, etc).

The correct thing to do in a case like this is to stop and coordinate with the prescribing physician. Having a pharmacist look at some change compared to how things were done prior and have a patient tell them it's a mistake and they should ignore it without consulting the prescribing physician is almost never a good idea.

You're really blaming the wrong people. Blame the doctor for screwing it up, your wife for not checking the prescription, or the government for creating/enforcing the relevant laws that would have put not only the chemist but their entire franchise underground for "doing the right thing". Laws are laws, broskie, don't expect everybody to break them for you.

If the migraine was really bad (bad enough to cause tangible damage) maybe you should sue the doctor for damages, or if it wasn't that bad, report him/her and go somewhere else next time.

So in effect he's right to suggest the pharmacist should just be replaced by a robot because you would have them follow the rules no matter what with all humanity stripped out. If there was more than one pharmacist available it wouldn't have killed him/her to go and take a look at the wife in the car.

The pharmacist can, and should, use their human judgement to refuse to dispense drugs even when prescribed. But they don't and shouldn't use their human judgement to dispense drugs that weren't prescribed; this is by design and for good reason. It's a two-person rule: you only get the drugs if both the pharmacist and the doctor formally agreed you should get them.

The problem seems to be that the drug has indeed been prescribed, but the writing was erroneous. This is not about "should we freely sell drug", but about "should we use human judgment in addition to paper orders".

Clearly, some commenters are also trying hard to be robot-swappable. :)

The prescription was for the wrong drug, i.e. the wrong drug was prescribed and conversely the right drug was not prescribed. The pharmacist can, should, and quite possibly did use their human judgement to refuse to dispense the wrong drug, even though it was prescribed. But by design they don't have the authority to dispense the right drug without getting a doctor to formally prescribe it.

I really wouldn't be surprised if it turned out a large percentage (over 35%) of hacker news commenters originate from robots / ai / chatbots being trained / tested surreptitiously.

It doesn't matter if the pharmacist thinks -- or even knows -- that they are doing the "right thing", pharmacists cannot prescribe medications. Their life would be over in a heartbeat if anybody ever found out that he or she legally provided a drug to somebody without a prescription. Additionally, pharmacists should use their best judgement to deny valid prescriptions, similar to how a bartender would use their best judgement to not sell alcohol to a customer.

You are right and make an interesting point. Part of the advantage of dealing with a human is that one hopes they can deal with whatever strange problem it thrown at them.

If you can’t deal with corner cases, or have none than leveraging rules/software is the way to go.

I think of vending machines and parking meters as automation of simple tasks, but what to do when they break?

I think google tries this by trying to “automate all the things”. It works most times but when it goes wrong it makes it very frustrating to correct.

When they (the machines) break, a human should be able to override, i.e., I was once stuck in a malfunctioning parking garage (no ability to pay, barrier stayed shut), so I lifted the barrier and let everyone out without paying. This could be illegal, but keeping me in a parking garage without telling me how long it will take is also a crime. Please don't make human un-openable parking garages, ever (meaning: don't let a robot decide whether a human is allowed to leave).

This is a completely different scenario to the one described above. You yourself are potentially breaking the law (if the boom gate is damaged, assisting in potential theft of services rendered, etc) to prevent your car being stuck in the parking lot (which very likely has a clause in the conditions of entry, so may not be illegal).

The two key points are that you are committing the crime yourself, not asking somebody else to, and that it is in fact only potentially a crime, and even then it's relatively minor, and could be argued strongly in court.

You are not asking somebody else to illegally sell you opioids (for example) with no prescription. From what it sounds like, nobody was in real danger, just pain, and you expect people to risk everything they've spent their entire life working for just to prevent an hour of pain for some stranger?

We don't have a suing culture so much here so I don't think that is relevant. Also, I feel that medical personnel should minimize suffering to the best of their ability, they don't take an oath swearing to fully emulate an emotionless robot.

Get a grip on reality dude. Nobody is going to risk their livelyhood to stop somebody being in relatively[0] moderate pain for an hour. The fact that you expect strangers to do this for you is really very selfish.

Not sure how suing is related to the comment you replied to. If a chemist hands out drugs to people without prescriptions and they get caught, they will lose their licence, be barred for life, lose their store, get a massive fine, and in some cases serve jail time. Nothing to do with suing.

[0] Relative to medical emergencies.

If so, these people should be replaced by robots asap.

> If there was more than one pharmacist available it wouldn't have killed him/her to go and take a look at the wife in the car.

Depending on the jurisdiction, pharmacists aren't allowed to prescribe stuff or treat people on the spot. That's business of a (licensed) doctor.

My mother had a pharmacy and she often called the doctor to solve this issues. The doctor is a phone call away, and the pharmacist job is to do whatever he cans to improved the patient's illness.

In which case you blame the programmer who wrote the code, not the employee following the software. Your idea of deferring to the rules over all common sense is the core of the problem. It doesn't matter if we are talking about government rules over which they'll enslave you if you break them or corporate rules over which they'll fire and blacklist you if you break them.

Note the irony in saying the pharmacist should have trusted the machine while ignoring the established rules.

Hell, most humane countries do it like that way already. My pharmacist parents in Norway wouldn’t have a problem with this, based on anecdotes over the years.

That's more about CYA. The pharmacist would risk a lot by giving out the medicine without proper prescription. They may know it's OK, but they won't risk an overzealous prosecution, for example.

Except you are asking that pharmacist to break the law. If they got fired afterward, would you be the one to take them and their family in?

I wouldn't do that either. Because if I did that in the wrong instance, I would be fired. I'm sorry for your wife's migraines, but it's not worth me losing my ability to support my family.

See also the 1985 film “Brazil,” where a fly stuck in a printer (a bug!) sends a SWAT team after one Mr. Buttle instead of one Mr. Tuttle.

A classic! :)

  Mistake? *chuckles* We don't make mistakes!

  *crashing sound as an improvised plug for the new hole in the ground falls straight through*

  Bloody typical! They've gone back to metric without telling us!

Recently we had a lot of tourists drive through the unfinished tunnel Vaðlaheiðargöng (Iceland) because Apple Maps already marked it as part of Route 1. So we had people drive a workers' road past multiple signs banning entrance, and then drive on gravel through a tunnel filled with heavy machinery and workers. Quite funny.

Well, Iceland has a lot of gravel roads so they might assume that this is normal..

In the book Catch 22 the character "Major Major Major" is made a ... Major, I belive due to an IBM computer error.

And he can’t get promoted to captain!

Good Catch

The best there is.

We are already there - "Death by GPS" is a concept today: https://arstechnica.com/cars/2016/05/death-by-gps/?amp=1

The problem isn't that people refer to a machine as authority but that the process has no abort button.

When you design business processes like these, firing, hiring or managing people always keep in mind something can go wrong. Always include an abort button to either stop or roll back the process.

There's a problem with the "human override" option as well. Make it too easy to override and then the system just doesn't matter anymore.

It's basically the issue we're trying to solve, where to draw that line?

The system is meant to automated a certain process, some business process among different people.

It does not loose meaning because it can be aborted if stuff goes FUBAR.

I recommend reading the RISKS Digest. It has been a forum for this stuff for many years.

* http://catless.ncl.ac.uk/risks/

Phenomenal, I'm late to the party. Glad there is party.

> The human tendency to defer to authority may never be as terrifying as when that authority is held by an uncaring machine with a couple bugs.

To this observation I highly recommend E.M. Forster's The Machine Stops, first published in 1909(!).

[0] http://www.ele.uri.edu/faculty/vetter/Other-stuff/The-Machin...

> people driving into lakes because their GPS told them to

That one has actually happened: http://bgr.com/2016/05/17/car-gps-mapping-directions-lake/

It's a little unfair, though. There was heavy rain, fog, and a second article [0] says it was also in the middle of the night. The photo there also does make it look easy to mistake as a road.

[0]: http://torontosun.com/2016/05/13/woman-follows-gps-ends-up-i...

Didn't this happen in an Office episode?

I remember Dwight and Michael arguing in Michael's car over whether the GPS was right as it kept telling them to go right which was directly into a lake.


"Dunder Mifflin Infinity" - Season 4 Episode 3/4

Oh, yeah. From that picture, it took me a second to realize I wasn't looking at a wet road-top and instead water in a lake, which is very close to the same height as the surrounding road.

I'm glad it wasn't me driving there at night and in the rain, I could see myself making the same very bad mistake.

"We were only following orders" suddenly feels even more scary when they come from a computer and not a human.

It shouldn't. Blindly following orders from humans has much the same results.

When the orders come from a human then there is someone who is ultimatly responsible, someone to put through the courts, someone to throw in jail.

When the orders come from a computer. Will any of that happen, will there be someone responsible?

Where the orders comes from has nothing to do with anything in terms of who is held responsible. The person who acts is responsible. That they were following orders is not a valid excuse. This was what the Nuremberg Trials established for us. It doesn't matter if someone told you to do something, you still have the only capacity to make the decision whether you actually do it or not.

Humans are very quick to abdicate their own moral authority, and it is the most repugnant human impulse that exists. If someone puts a gun to your head, and tells you to kill someone or die, you are still responsible for whoever you kill. Others might consider your actions understandable, even forgivable, but you're still a murderer and still one who valued their own life above anothers.

If you do something because you were "just doing your job", that's even worse. That's exactly identical to doing something for money. Which is usually looked very poorly on by society.

yeah, but punitive justice is the least consequential thing in those situations. The major reason you'd punish someone there is to stop someone from willfully repeating the scenario. There exists a path for learning and prevention in automated systems too.

The danger I see is the stifling of potential human subjective decisions. That broad decisions will be made and carried out without a sanity/humanity check by humans.

I think measures should be put in place for a guarantee of human intervention in the case of situational anomaly or system error as well as an emphasis on redress.

Since automation scales and a human workforce doesn't, a guarantee of human intervention might not be practically possible.

Anyone have any ideas on how this kind of thing might be mitigated?

I think it won't be mitigated, given my experience working as a data scientist. It will be ignored because it is profitable to ignore it. If we slowed down and figured out how to make machines mimic human empathy before we make them make decisions it could be mitigated, but we won't do that. Ask your boss about it.

"It looks like the devs didn't do their job properly..."?

But from the other side of the system, it's perfect. Now everybody involved can say they were just following orders!

A lot of biometric technologies are treated like authorities by "our authorities" to a ridiculous fault.

Also very similar to the classic "forgotten employee" story which made the rounds in the early aughts:


Oh man! I just read that, and absolutely loved it. A pretty brilliant piece of writing.

Reminds me of the Google Ultron IT guy story, from 2014 (I think? How do 4Chan/American dates work?)


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact