I often found myself wishing I had better insights or responses during crucial moments in business meetings. Sure, I'd review meeting transcripts afterward and use LLMs to analyze them, but I kept thinking, "What if I could harness this power during the meeting itself?"
That's when the idea for Hedy (https://hedy.bot/) was born. I wanted to combine real-time meeting transcription with the analytical prowess of LLMs, creating an AI assistant that could provide instant, contextual insights. Hedy runs on your phone during your meetings and classes, offering real-time answers, thought-starters, and ways to position yourself as the brightest person in the room.
Now, I can fire up Hedy during a Zoom meeting and have it provide direction to work through complex situations, analyze what's been discussed so far, and suggest ways to build better relationships with other attendees.
But it's not just about making professionals sharper in meetings; it's about leveling the playing field for students with learning disabilities, non-native speakers in international settings, or anyone who's ever felt that sinking "I should know this" feeling mid-conversation.
From a development perspective, building the on-device Speech-to-Text engine was the biggest challenge. I wanted to keep all audio processing on-device for privacy purposes and to avoid cloud computing costs, allowing us to offer the app with unlimited meeting time for a fixed cost. In the end I was able to build an STT engine does performs very fast on iOS devices by taking advantage of the Metal architecture. The Android version is still in development.
Specs:
* LLM: Anthropic Claude 3.5 Sonnet
* STT: OpenAI Whisper
* Backend: Google Firebase
