Even if bandwidth weren't an issue and all users had compatible hardware: You'd ...

lxgr 3 months ago | parent | context | favorite | on: Serving 70B-scale LLMs efficiently on low-resource...

Even if bandwidth weren't an issue and all users had compatible hardware: You'd still be offloading a (semi-)trusted computation to user hardware, which is usually completely untrusted.