This is a great ideal and user friendly as well. Has the potential of converting multiple old devices overnight from being useless. However, I wish they had provided some results on tok/s, latency with some example setups.
We didn't expect this to blow up so quickly. A lot of work needs to be done on getting different setups working. I have made an issue here: https://github.com/exo-explore/exo/issues/11
This is great work! I will keep an eye (and maybe even try to contribute).
Looking back at the beginning of Google, I think their use of hardware and hardware agnostic platform likely contributed to support growth at lower cost. We need more of that in the AI era
Thank you for the support! I agree on the cost point, and personally I don’t want to live in a world where all AI runs on H100s in a giant datacenter controlled by one company.
