Then pursue the customers and force them to actually report their rightful use-tax when they file their taxes at the end of the year. Amazon isn't not collecting CA sales tax because they think it shouldn't be paid. They're not collecting the sales tax because they have no legal requirement to.
