I wouldn't say racism is the problem, but it is one of the problems. It can be the case that white people are disadvantaged by systemically racist policies originally intended to target black people. That urban planning sometimes had an explicitly racist element in its execution, such as American transportation infrastructure being laid out in such a way as to disenfranchise black communities[0], is a simple but unfortunate historical fact. The ripple effects of racism baked that deeply into infrastructure don't just go away when society becomes desegregated.
[0]https://www.kqed.org/news/11943263/americas-highway-system-i...