The bar for success is different too. A rocket which safely delivers its cargo 99.99% of the time would be a huge step forward. A car which safely delivers its cargo 99.99% of the time would be taken off the roads.
It’s funny you mention 99.99% - because 40,000 people are killed by car crashes in a year in the United States, which is actually .012% of the population. So cars actually do a bit worse than your stat...
If a majority of the population used a rocket daily then the numbers would be different. The number of people killed per space flight is probably a little higher than the number if people killed per drive in a car.