Say you want a model that takes in a picture of a person and tells you what emotion they are feeling - sad/happy/disgusted/whatever.
But you also want to know why it classifies a person as sad/happy/etc.
Then you make the model have 2 outputs. The first is just a classification of the emotion as you'd have normally.
While the second is which parts of the image contributed most to the classification - e.g. a heatmap over a smiling mouth in one case or over the squinted eyes in another.
I could take a unet and plug a classifier to it hmm. I tried something like that before but it did not work that well. Maybe what I did sucked and there is a better way
Not sure how that would work with a segmentation algorithm like Unet though. It makes more sense in showing where in an image that the network was activated to give the image a label. AFAIK Unet gives ever pixel a label so I don't see how you could do what the parent describes.
That is activation mapping. I understood that what he was suggesting is using another objective that is optimized to define why network decided what it did. But need loss function for that
For let’s say identifying a car we could come up with a labeled dataset that not only has classes but also different parts of the car labeled that we think differentiate car from a motorcycle or a whatever. Then model has to output both class and also segmentation or bounding boxes of the parts
Then we combine both losses from these outputs and train.
I tried using unet by attaching layers to it in different ways to interpret the segmentation. I could not get it to work well though
But you also want to know why it classifies a person as sad/happy/etc.
Then you make the model have 2 outputs. The first is just a classification of the emotion as you'd have normally. While the second is which parts of the image contributed most to the classification - e.g. a heatmap over a smiling mouth in one case or over the squinted eyes in another.
You can do this with pretty much anything.