Hacker Newsnew | past | comments | ask | show | jobs | submit | traktorn's commentslogin

Which model are you running locally? Is it faster than waiting for Claudes generation? What gear do you use?


That's the fun part, you can use all of them! And you don't need to use browser plugins or console scripts to auto-retry failures (there aren't any) or queue up a ton of tasks overnight.

Have a 3950X w/ 32GB ram, Radeon VII & 6900XT sitting in the closet hosting smaller models then a 5800X3D/128GB/7900XTX as my main machine.

Most any quantized model that fits in half of the vram of a single gpu (and ideally supports flash attention, optionally speculative decoding) will give you far faster autocompletes. This is especially the case with the Radeon VII thanks to the memory bandwidth.


Not OP but for autocomplete I am running Qwen2.5-Coder-7B and I quantized it using Q2_K. I followed this guide:

https://blog.steelph0enix.dev/posts/llama-cpp-guide/#quantiz...

And I get fast enough autcomplete results for it to be useful. I have and NVIDIA 4060 RTX in a laptop with 8 gigs of dedicated memory that I use for it. I still use claude for chat (pair programming) though, and I don't really use agents.


I love Dynomights writing. I highly recommend subscribing to his newsletter.


The page looks nice and I like the catchphrase, it would be nice to see some samples.

Have you built anything using this yourself?


yep for now it is focused on telegram:

- simple todo/link collection bot with telegram custom keyboard

- markdown from pdf message

- bot that returns top 5 posts on hn

- simple settlement bot

- friend manage to build a bot that after getting a url to recipe generated image about it

- "cookie clicker" app

- basic shop app


Cool, looking forward to seeing a "gallery" of these.


The same for the Testimonal-link in the menu. Not working.


yes sorry will update this in future, landing for now supposed to be mostly basic info and redirect to telegram.

regarding pricing: mostly number of messages per month/ number of bots

- free: 10 msg, 2 bots

- starter: 50 msg, 5 bots - 10$/month

- developer: 100 msg, 10 bots - 20$/month

- pro: 250 msg, 20 bots, voice messages - 50$/month

- scale: 500msg, 40bots, voice messages - 100$/month


Messages/month? Is that the amount of messages the bot does in Telegram? Or the amount of messages sent to the LLM when building it?


amount of edit messages in plutonic bot, so yes messages sent to llm


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: