We’re using the Orin AGX for edge ML. Not the same setup (Ampere) but it’s a similar situation. The GPU is excellent for what we need it to do, but the CPU cores are painful. We’re lucky… the CPUs aren’t great but there’s 12 of them and we can get away with carefully pipelining our data flows across multiple threads to get the throughput we need even though some individual stage latencies aren’t what we’d like.
