No, he's referring to back before Congress told private businesses that they weren't allowed to refuse to serve black people. When Congress took away the right of family owned restaurants to exercise their freedom of association with only white people.