The rules are actually pretty strict for what can constitute legal refusal of a fare [1]. 54-20(b)(2), "The Passenger is carrying or is in possession of any article, package, case
or container which the Driver reasonably believes will cause damage to
the Vehicle" and 54-20(b)(9) "The passenger is disorderly or intoxicated" are really the only rules that have a subjective component.
...and yet, it's only four letters to spell 'race'. And its not there, anywhere. So easy to have said 'drivers will not discriminate on the basis of race, religion' or some such. And they didn't say it.
However, if the law says "You must pick up a passenger under [SMALL LIST OF EXCPETIONS]", then you basically are banned from excluding people on the basis of their race, right?
*(NB I'm not USAian, and don't know much about US taxis)
[1] http://www.nyc.gov/html/tlc/downloads/pdf/rule_book_current_..., see S 54-20