Hey hacker news,
We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project.
We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data.
The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information from other systems and inserting them into the LLM’s context window via a prompt. This gives LLMs information beyond what was provided in their training data, which is necessary for almost every enterprise application. Examples include data from current web pages, data from SaaS apps like Confluence or Salesforce, and data from documents like sales contracts and PDFs.
RAG works better than fine-tuning the model because it’s cheaper, it’s faster, and it’s more reliable since the provenance of information is attached to each response.
While there are quite quite a few “chat with your data” apps at this point, most have external dependencies to APIs like OpenAI or Pinecone. RAGstack, on the other hand, only has open-source dependencies and lets you run the entire stack locally or on your cloud provider. This includes:
- Containerizing LLMs like Falcon, Llama2, and GPT4all with Truss
- Vector search with Qdrant.
- File parsing and ingestion with Langchain, PyMuPDF, and Unstructured.io
- Cloud deployment with Terraform
If you want to dive into it yourself, we also published a couple of tutorials on how to deploy open source LLMs for your organization, and optionally give it access to internal documents without any data ever leaving your VPC.
- How to deploy Llama 2 to Google Cloud (GCP): https://www.psychic.dev/post/how-to-deploy-llama-2-to-google...
- How to connect Llama 2 to your own data using RAGstack: https://www.psychic.dev/post/how-to-self-host-llama-2-and-co...
Let a thousand private corporate oracles bloom!
