For single projects, it's possible to build such caches and it's done by some, e.g. Firefox uses it.
But for the "any project" use case, the combination of Rust's compilation model and how crates.io works makes this hard. Due to Rust's compilation model, if crate A depends on crate B, then the version of crate B it's been compiled for can't be changed: you have to use that version. It also means that the features of the crate that have been enabled can't be changed. "Normal" Rust projects can quickly get into hundreds of crates.io dependencies in their DAG. Those are the very dependencies you want to use ssccache for. However, due to how crates.io works, basically every day you get a new updated crate in your DAG, and this invalidates all the crates that depend on it, including your own.
In a single project use case, this is no problem because everyone uses the same Cargo.lock so they use the same dependencies and the same features of those dependencies. But if you don't share a Cargo.lock, the problem is too wide.
There are some solutions like freezing crates.io according to a schedule and only pushing out updates once per week or so. But this still leaves the feature issue unsolved. Plus it is probably against the idea of crates.io to have updates available immediately. Another solution would be API only dependencies, but this is harder to pull off.