"Racism" does not have exact definition. It's just a label that means slightly different things in different contexts. It's an abstraction over a specific type of offensive / hateful behavior. In more in-depth discussions, I think it's better to operate beneath this abstraction, on a lower level: i.e. how much harm saying or doing something does.
In most situations, saying "dumbblack" would be more offensive for black person than "dumbwhite" for white person. Especially in this case. It's like if you had a couple of bad experiences with the police, you could start saying offensive things about police officers in general. You could be accused of "policism" but in fact, it would be quite understandable reaction. When he said "dumbwhite", he was kind of "revenging" their racism, which is very different from saying "dumbwhite" just because he disrespects white people.
In most situations, saying "dumbblack" would be more offensive for black person than "dumbwhite" for white person. Especially in this case. It's like if you had a couple of bad experiences with the police, you could start saying offensive things about police officers in general. You could be accused of "policism" but in fact, it would be quite understandable reaction. When he said "dumbwhite", he was kind of "revenging" their racism, which is very different from saying "dumbwhite" just because he disrespects white people.