There's another definition, which is talk that's designed to maintain & reinforce the institutionalised power structure among races. Right now, if modern USA life was a video game, "white male" would be an easier difficulty level than "black male". There are statistically less problems for the "white male" group. Racist talk is talk that re-enforced that imbalance. This definition is harder for some people to accept because it means that you need to look at yourself and think about what power imbalances you might be benefiting from, and it means you can't just do s/white/black/g and make it just as racist.
So no, it's more racist to say "dumbblack" than "dumbwhite"