Hey HN! Alex and Zack from Nexa AI here. We are excited to share a project our team has been passionately working on recently, in collaboration with Jiajun from Meta, Qun from San Francisco State University, and Xin and Qi from the University of North Texas.
Running AI models on edge devices is becoming increasingly important. It's cost-effective, ensures privacy, offers low-latency responses, and allows for customization. Plus, it's always available, even offline. What's really exciting is that smaller-scale models are now approaching the performance of large-scale closed-source models for many use cases like "writing assistant" and "email classifier".
We've been immersing ourselves in this rapidly evolving field of on-device AI - from smartphones to IoT gadgets and even that Raspberry Pi you might have lying around. It's a fascinating field that's moving incredibly fast, and honestly, it's been a challenge just keeping up with all the developments.
To help us make sense of it all, we started compiling our notes, findings, and resources into a single place. That turned into this GitHub repo: https://github.com/NexaAI/Awesome-LLMs-on-device
Here's what you'll find inside:
A timeline tracking the evolution of on-device AI models
Our analysis of efficient architectures and optimization techniques (there are some seriously clever tricks out there)
A curated list of cutting-edge models and frameworks we've come across
Real-world examples and case studies that got us excited about the potential of this tech
We're constantly updating it as we learn more. It's become an invaluable resource for our own work, and we hope it can be useful for others too - whether you're deep in the trenches of AI research or just curious about where edge computing is heading.
We'd love to hear what you think. If you spot anything we've missed, have some insights to add, or just want to geek out about on-device AI, please don't hesitate to contribute or reach out. We're all learning together here!
This is a topic we are genuinely passionate about, and we are looking forward to some great discussions. Thanks for checking it out!
