Local LLM Models: Are They Useful?

[peab]

https://www.maximepeabody.com/blog/local-llm-models

[mojomark]

I mean, short answer is that of course they're absolutely useful if you sit behind security firewalls. Is that not enough?

[peab → mojomark]

Ah, that's another good use case. RIP to whatever company doesn't allow their workers to use the LLMs though

[mojomark → peab]

Think larger - that is, governments. You simply cannot have people just typing whatever they want willy-nilly into cloud-based LLM's. For anything other than fully unclassified (and that is a lot - for any country) that must be done on-prem.

[mojomark → mojomark]

Well, I should clarify... today that's done on-prem. Of course if you had a secure network and host, LLM interaction could certainly be done in the cloud.

There are also other reasons to host LLM's locally. That is systems that leverage contextual reasoning power of LLMs, but are unable to readily access the internet. A planetary rover (e.g. Mars), for example.