I'm always a bit wary about doing write up on things like this, as I kind of just muddle things together and definitely don't follow best practices.
I used OpenClaw a while back, I did find it useful, but it wasn't as reliable.
This time around, I found hermes a lot easier to use and configure. One thing I think has improved it is building apps to "pre-compute" data.
For example, with my email, instead of accessing my email servers directly, I have an IMAP server running locally; with mbsync running to download all the mail from all my email accounts locally. Then, when I ask hermes for something in my email, it's much quicker, doesn't hit any rate limits, and gets the full email (gmail sometimes doesn't send the full body of the email).
With my travel and purchase tracking. I parse an e-mail once as it arrive (I use a frontier model to generate a script which can parse e-mails from that merchant in the future). The parsed data is added to a SQLite database.
I ask hermes what's the address of the place we're staying at next week, It generates a SQL query, and queries the database.
Much cheaper/quicker/reliable than searching my e-mails for booking confirmations, and then parsing the e-mail.
My (unscientific) theory, is that doing things like that is much more important than the model or agent you use.
For me, the concern with Hermes, etc, is the security aspect. I was running OpenClaw in a dedicated VM and I carefully controlled what files it had access to. But I'd rather not go that route again.
The other major headache was it trying to configure itself, and then crashing.
Eventually I wrote my own bot that communicates with me on Telegram. I've manually enabled a few capabilities ("Add to my TODO list", "Add to my calendar", "Get weather" (in format I prefer)). The hope was I'd just add functionalities as I needed, and know how the whole flow works.
I didn't find value in things like soul.md, heartbeat, memory, etc (as in, they never really "worked"). I think most of the stuff you described are things I could easily add to my custom solution - I think that may still be a better approach than using hermes and wondering what the heck it's doing.
What do you use to allow the LLM to get content from web sites? I don't need it to do stuff - just simple navigation. The most complex thing would be "Here's my question - go to this Linux forum thread and navigate all the pages and see if anything there addresses my question."
For browsing, I use camoufox, it tries to hide the fact that it is an AI running a bot and seems to be fairly successful.
I set the browser window dimensions as the same is my phone, and added a VNC server. so if there are things I need to intervene on (like a captcha), I can log in and manually take control.
The agent will tell me in the WhatsApp chat, and send a screenshot.
I have a VNC client app on my phone, so I log in and take control. But my next step is to setup a web based VNC client, and have the agent send me a link to it (with time based with), so that I can do it from within WhatsApp
I used OpenClaw a while back, I did find it useful, but it wasn't as reliable.
This time around, I found hermes a lot easier to use and configure. One thing I think has improved it is building apps to "pre-compute" data.
For example, with my email, instead of accessing my email servers directly, I have an IMAP server running locally; with mbsync running to download all the mail from all my email accounts locally. Then, when I ask hermes for something in my email, it's much quicker, doesn't hit any rate limits, and gets the full email (gmail sometimes doesn't send the full body of the email).
With my travel and purchase tracking. I parse an e-mail once as it arrive (I use a frontier model to generate a script which can parse e-mails from that merchant in the future). The parsed data is added to a SQLite database.
I ask hermes what's the address of the place we're staying at next week, It generates a SQL query, and queries the database.
Much cheaper/quicker/reliable than searching my e-mails for booking confirmations, and then parsing the e-mail.
My (unscientific) theory, is that doing things like that is much more important than the model or agent you use.