So you build a grey box neural network that analyzes resumes to find "good" candidates. Underneath it might draw correlations between female or minority names. It might decide that people who lived in certain places or came from country X are "bad employees". Since nobody can explain how it works once you've trained it, you're free to build it using whatever biases you want.
Same with financial regulations. The network could rely 100% on insider trading. Analyzing the HTML of websites for leaked information that isn't visible to humans yet. Again nobody knows this is how it's so good at taking in bytes at one end and buying stocks on the other.
They're literally an attempt to replace humans as the mechanism by which rote paperwork can be done -- that they're used this way by business isn't terribly surprising.
So algorithms aren't the "new bureaucracy", they're the new way of doing the old bureaucracy -- as they were since inception. This is a feature, not a bug.
I’ve also wondered if some elements of brainstorming could be automated, such as reframing a situation in random ways and see what the result is like..
Makes you think about RESTful apis, huh