Yes. Doing the same. What is the advantage of this new feature? Tmux/Tailscale/Termius give you full control of your terminal.
Or mainly to save the end user the hassle to set it up correctly?
Oh lots of people will not be comfortable with tmux approach. The anthropic feature makes sense. But it's Max only and doesn't work well according to other comments.
Ease of setup is the biggest reason. I use this setup as well, but there are other UX niceties that would be a lot better with a dedicated mobile app: push notifications when Claude needs your input (I use a hook for this that connects to Pushover, but that's another service and extra setup), voice input, autocorrect that's right for this context, etc.
I have a hook in my claude.json that fires on "Stop", it calls a shell script (written by Claude, of course) that calls the Pushover API: https://pushover.net/, which lets you send push notifications to your device. It's paid, but just a one-time fee when you install the app on your phone.
The shell script takes a message which includes Claude's message, but unfortunately there's no deeplinking back to my ssh app (for obvious reasons, the notification just routes you to the pushover app), so instead of tapping the message, I know to just open my Blink shell app to respond to Claude.
This is also quite noisy when I'm just sitting at my desk working, but I usually turn off phone notifications while working anyway.
Which program does support it to allow streaming? Currently using spokenly and parakeet but would like to transition to a model that is streaming instead of transcribing chunk wise.
This is actually pretty impressive. What kinda phone are you using? Are you noticing any drain on battery heat?Do you think it's possible to get this working with Flutter on iOS?
2-3 years old Android flagship phone with 8 GB RAM. When I looked for an app for parakeet, I think I also came across iOS apps. Don't recall it since I use Android.
Seems light on the phone/battery. Don't observe any drain but I also only record shorter transcripts at once.
Side note: Parakeet is actually pretty nice to do meetings with oneself. Did that on a computer while driving for an hour (split in several transcript chunks). Processed the raw meeting notes afterwards with an LLM. Effective use of the time in the car...
Unfortunately, Parakeet doesn't support streaming like Moonshot does (as much as I know). Would be perfect to have sth of the size of Parakeet but supporting streaming. Still hope Nvidia releases a V4 with that feature :)
Otherwise, I think STT is basically a solved problem running locally on edge devices.
I think there is a streaming version of Parakeet. It is often referred to as Nemotron, though.
I tried comparing Parakeet streaming with Moonshine streaming. Moonshine is smaller, and I felt it was subjectively faster with about the same level of accuracy.
This is actually really good. I'm writing with it right now. It's just not the best setup as a keyboard. Because for example you cannot easily switch back to uh the normal keyboard with keys.
With the remote setup of Claude Code (e.g. [1]) I can know (vibe)code from my phone. But typing has been a pain. Didn't find a great speech to text app for Android featuring Nvidia's Parakeet v3, which is the (?) leading STT model.
Found this repo. It's working flawlessly. Checked it with adb (I am always a bit suspicious). And it's really fully local. Highly recommend :)
I had not heard of Parakeet until earlier today with Handy [1].
I've previously had good luck with FUTO's keyboard and it's companion voice input app [2] on my Android, both of which are local-only after downloading the model. I'll have to try this one out and compare them.
FUTO Voice Input is currently based on the OpenAI Whisper model, and could theoretically support all of the languages that OpenAI Whisper supports. However, in practice, the smaller models tend to not perform too good with languages that had fewer training hours. To avoid presenting something worse than nothing, only languages with more than 1,000 training hours are included as options in the UI:
<List of supported languages>
Language support and accuracy may expand in the future with better optimization and fine-tuned models. Feedback is welcomed about language-related issues or general language accuracy.
Went into a rabbit hole and found this: https://github.com/notune/android_transcribe_app
Solid app that uses Parakeet V3.
With these random apps on the internet I am always a bit sceptical. Checked it with adb and it is really fully local. I now have a voice keyboard that is a lot better than Google's and has local multilanguage support. I am stoked :)
reply