> TCAV uses directional derivatives to quantify the model prediction’s sensitivity to an underlying high-level concept [...] For instance, given an ML image model recognizing zebras, and a new, user-defined set of examples defining ‘striped’, TCAV can quantify the influence of striped concept to the ‘zebra’ prediction as a single number. In addition, we conduct statistical tests where CAVs are randomly re-learned and rejected unless they show a significant and stable correlation with a model output class or state value.
AI interpretability is a very important and exciting field, and I don't mean to detract from the rest of this article, or from the speaker's work. However:
1) Technology is neither good nor bad in and of itself. It is a tool that can be used for one or the other.
2) If it's useful for something, some people will use it, despite the protests of others. Have we ever collectively decided to "drop it" in the past, where "it" is a powerful technology?
> Because CFCs contribute to ozone depletion in the upper atmosphere, the manufacture of such compounds has been phased out under the Montreal Protocol, and they are being replaced with other products such as hydrofluorocarbons
Maybe this success is at least somewhat due to the fact that there is an alternative technology that has very similar utility and cost, but without the significant negative effects.
The absence of such an alternative might explain why Asbestos hasn't had similar success. From https://en.wikipedia.org/wiki/Asbestos#Usage_by_industry_and...:
> Some countries, such as India, Indonesia, China, Russia and Brazil, have continued widespread use of asbestos.
One possibility is that a bad AI decision could create huge liability costs on whoever is running the algorithm. If the algorithm can't explain that its decision was reasonable, courts could get... testy. And that would drive people towards at least explainable AI.
That is, it might end up being cheaper to have some racist employees than a big monolithic AI making racist decisions (or other human failings like being a bad driver).
> Macaque monkey: (2017) First successful cloning of a primate species using nuclear transfer, with the birth of two live clones, named Zhong Zhong and Hua Hua. Conducted in China in 2017, and reported in January 2018. In January 2019, scientists in China reported the creation of five identical cloned gene-edited monkeys, using the same cloning technique that was used with Zhong Zhong and Hua Hua – the first ever cloned monkeys - and Dolly the sheep, and the same gene-editing Crispr-Cas9 technique allegedly used by He Jiankui in creating the first ever gene-modified human babies Lulu and Nana. The monkey clones were made in order to study several medical diseases.
From Pluripotent stem cells: the last 10 years
> Over the past 10 years, technological advances and innovative platforms have yielded first-in-man PSC-based clinical trials and opened up new approaches for disease modeling and drug development.
But for a neural network with hundreds of mini logistic regressions built in... seems tough. Interpretability isn’t just knowing how decisions are made but also how much irrationality is being built into it. If one factor is being marked as a huge negative influence, what does that actually mean about the problem space? Maybe nothing. If you actually want an interpretable neural net model, you should probably hand craft layers of domain specific ensembles that you can individually verify and are closer to being self evident. Maybe you won’t know how it determines stripes or horselike, but if you feel good about those two models individually then it’s a much easier task to follow the last step which is: it’s a zebra if and only if it’s horselike and has stripes.
Human rationalisation has it's issues eg “So convenient a thing to be a reasonable creature, since it enables one to find or make a reason for every thing one has a mind to do.”
Wonder if AI will do better.
Seems like an obvious technique. A few months back, someone posted a similar linear reverse engineering technique to make a customizable face generator from already trained GANs.
Language shouldn't be necessary if your feature can be conveyed through examples.
But yes, it's a big assumption to say that all features can be isolated as linearly decidable subsets of the activation space.
I would guess one could get better results with stronger, non linear classifiers combined with more abstract generalizations to directional derivatives.
Can it be emulated with our current tech / knowledge ? maybe, maybe not.
In the end it all boils down to: is intelligence / consciousness 100% material.
If it's the case it would be theoretically possible to replicate it. In practice it's much more complex.
If these principles can't be explained by materialism I think we'll have even bigger questions to answer.
also see: https://computing.dcu.ie/~humphrys/philosophy.html
I remember reading that people who worked on "flying machines" said it wouldn't be possible to have heavier than air planes before decades or centuries (or even ever), little did they now that they already existed for a few months but the news didn't came to them yet.
Turing machine is only one kind of computing device. It just happens to be really good at simulating many other kinds.
Turns out, most differences between computing devices don't really matter.
- Edsger Dijkstra (EWD898).
Understanding why a machine learning model makes certain decision is very relevant in practice.
Am not going to argue whether a machine can “really” be alive, “really” be self-aware. Is a virus self-aware? Nyet. How about oyster? I doubt it. A cat? Almost certainly. A human? Don’t know about you, tovarisch, but I am. Somewhere along evolutionary chain from macromolecule to human brain self-awareness crept in. Psychologists assert it happens automatically whenever a brain acquires certain very high number of associational paths. Can’t see it matters whether paths are protein or platinum.
I understand this, however it's not a fair point. Without proper understanding of 'swimming' and 'flying' these technologies wouldn't be possible. So the link between thinking and AI is not totally trivial in my opinion.
It's very interesting that there is no problem to describe the action of planes as "flying" (because we humans do not fly), but OTOH it feels very weird to say that a submarine swim... becase WE do swim...