Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)

[muttled]

https://airic.serveo.net/

I trained the 65b model on my texts so I can talk to myself. It's pretty useless as an assistant, and will only do stuff you convince it to, but I guess it's technically uncensored? I'll leave it up for a bit if you want to chat with it.

I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion:

LLAMA 1 65b

Rank 128

5 epochs

Batch size 1, 256 cutoff

Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora

Loss around 1.5 seems to give the most coherent results

Trained on raw text dumps that is then parsed by a crappy Blazor Server app I threw together in a few hours. Text format is just "Sender:The Message\n"

Trained on 2x 3090

Training took about 16 hours at a 90% power cap on the 3090's

Trained on ~30k texts (I talk a lot, that was just 2 years)

There's nothing telling it that it's a robot, though it sometimes seems to know

It was largely inspired by the Unreal Engine lora tutorial

I generated a list of fake names and addresses, pulled a list of my contacts, and then scripted out swapping the names and addresses for fictitious PII. I don't really send other sensitive data through text and my account is so thoroughly associated with my real name/location that the data leakage risk is manageable for the short period of time I'll have this available. It tends to halucinate fake PII as well which I think is partially a side effect of the data scrubbing. You'll notice it says things like that I live at 420 Ligma.

I'll need to mix in some actual assistant tasks to the dataset before it will actually be useful as an assistant. Right now it's largely just for idle conversation.

It's pretty ADHD and will randomly go off on its own tangents. I don't think it's the model. I think I just talk like that.

Let me know if you have any questions or comments. I built it for myself, but figured I'll let the communities that have taught and entertained me so much play with it a little, too.

Note: it says some pretty unhinged stuff. There's absolutely no guardrails. It also tends to talk like you're already friends with history.

[coreyp_1]

Cool. But please tell me that your childhood phone number wasn't 708-642-9XXX (censored in case it's real) because it just told me that "fact"... and we were chatting about trivia night!! lol

It also told me that your mom was an astronaut and that you were born in space. I love this thing!

[muttled → coreyp_1]

Nope, not mine! I scrubbed out the PII and all the chats I've seen so far, it's been generating fake info.

[coreyp_1 → muttled]

I am sad to know that you were not actually born in space. lol

[kinduff]

This is so interesting. I've been wanting to do this, not to chat, but to generate coherent answers to real chats - like pre-baked answers to quickly reply. Looks pretty complicated, though. I have no idea where to start.

[clarkmcc]

According to this, your boss Matt Cain is the most annoying person you know. This would be a much more exciting chat if it hadn’t been scrubbed haha

[quickthrower2]

It is a cool app but it might spill secrets? It told me some things like someone’s name who you know, but it could be making it up.

[muttled → quickthrower2]

It's probably making it up. I scrubbed the PII vs a list of fake names and info.