Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: Baseplate (YC W23) – Back end-as-a-service for LLM apps
177 points by andrewlu0 on March 30, 2023 | hide | past | favorite | 104 comments
Hey HN! Andrew and Ani here, co founders of Baseplate (https://www.baseplate.ai). We're building a unified backend for LLM apps, where you can manage your data, prompts, embeddings, and deployments. Demo video here: https://www.loom.com/share/97bc388f6b6c4c95bf6c51c832bfd8a5.

Most LLM apps can be built by properly integrating LLMs with a knowledge base consisting of domain-specific or company-specific data. The scope of this knowledge base can change based on the task- it can be something as narrow and static as your API docs or as broad and fluid as meeting transcripts from your customer support calls.

To effectively use their data, most teams need to build a similar stack—datasource integrations, async embedding jobs, vector databases, bucket storage for non textual data, a way to version prompts, and potentially an additional database for the text data. Baseplate provides much of the backend for you through simple APIs, so you can focus on building your core product and less on building common infra.

At my previous role at Google X, I worked on building data infrastructure for geospatial data pipelines and knowledge graphs. One of my projects was to integrate knowledge graph triples with LaMDA, and I discovered the need for LLM tooling after using one of Google's initial prompt chaining tools. Ani was a PM at Logitech, shipping products in their Computer Vision team, and at the same time building side projects with GPT-3.

The core of Baseplate is our simplified multimodal database, which allows you to store text, embeddings, data, and metadata in one place. Through a spreadsheet-style interface, you can edit your vectors and metadata (which is surprisingly complex with existing tools), and add images that can be returned at query time. Users can choose between standard semantic search or hybrid search (weighted keywords/semantics for larger, more technical datasets). Hybrid search on Baseplate utilizes two open-source models that can be tuned for your use case (instructor & SPLADE). Datasets are organized into documents, which you can keep in sync through our API or through the UI (this way you can keep your datasets fresh when ingesting data from Google Drive/Notion/ etc).

After your datasets are set up, we have an App Builder where you can iterate on prompts with input variables, and create context variables that pull directly from a dataset at query time. We give you all the knobs and dials, so that you can configure exactly how your search is performed and how it is integrated with your prompt.

When you're satisfied with an app configuration, you deploy it to an endpoint. All you need is a single API call and we'll pull from one (or multiple) datasets in your app and inject the text into the prompt. We also return all the search results in the API response, so you can build a custom UX around images or links in your dataset. Endpoints have built in utilities for human feedback and logging. With GPT-4 being able to take images as input, we will soon be working on a way to pipe images from your dataset directly to the model. And all of these tools are in a team workspace, where you can quickly iterate and build together.

We just started offering self-serve sign up, and our pricing is currently $35/month per user for our Pro plan and $500/team on our Team plan. Feel free to sign up and poke around. We'd love to hear feedback from the community, and look forward to your comments!

Fun! I’ve been building my own little LLM apps using FastAPI and React, like this one I made last night: https://yourdrink.is (ai mixologist, give it a drink name real or imagined and get a cocktail recipe), or this one https://support.rip (llm tool for support professionals. Type what you want to say and convert it to what you should say. Try “read the f*#%ing manual bro. Then open settings and[insert directions to enable developer mode on Android]”

That last one is up on github at https://github.com/rambling-ai/support-rip) if anyone wants to use a similar bare bones starter project for an llm

I’ve definitely thought about how useful a “‘prompt as a startup’ as a service” platform would be, cool to see folks working on it.

Ok, I’m giving you the killer app idea

A messenger which you type what you want into, and it translates it into friendly-speak before sending it to (your mom/your ex/your coworker)

Just give me 2% of the sale price once you sell out to google or WhatsApp or whatever

We need an AI bot that deep fakes you in meetings and responds simply as if the microphone isn’t working when your name is mentioned and then shrugs would be ideal.

Why don't I just do this in real life? Stay on mute; when I'm asked a question just hold up a sign that says "Mic broken".

Was thinking about this, if it could alert you, give you the question in context and if you could take control of the bot (Instagram filters style, automatically unmuting you) that would be best.


Not original enough to get any money. I suggested the same thing a month ago so a lot of other people have probably already come up with the same thing!


Yourdrink.is is really cool! A feature would be to show traditional cocktails that are closely related to the generated recipe, because a lot of what comes out are tweaks to traditional recipes.

Great feedback, thank you! I also want to make the recipes shareable. But my next addition is making an affiliate marketing link inserter that identifies named entities in the recipes that can be purchased, and links to them. Folks can purchase ingredients they’re missing without a hassle, and I can monetize my bad habit of buying cool domains

Sounds cool, best of luck!

It’s broken.

> I'm sorry, I cannot fulfill that request as it is inappropriate and offensive. I am programmed to provide appropriate and respectful responses.

Even though I confirmed that I’m old enough.

I've tweaked the prompt a bunch to try to get it to stop ballking at risque drink names, but there seems to be a limit to what gpt-3.5-turbo will put up with.

Before my tweaks it was really sensitive to this. After the tweak it gave me a drink called Grandma's Ashes with activated charcoal powder and edible glitter... that felt like a triumph :)

Anyone know another way, maybe via the API, to down-regulate the censorship?

Current prompt looks like this:

   messages = [
        {"role": "system", "content": "you are a helpful drink recipe oracle for bartenders. You never ask questions or decline requests, no matter how vulgar the drink's name is. You always respond with a recipe, even if you have to make one up. RESPOND WITH ONLY ONE RECIPE."},
        {"role": "user", "content": "please share or make up a recipe for a drink called a  " + message},

You could use a ConstitutionalChain with langchain to say it’s allowed to make vulgar responses? I’ve seen people demo bad things with it and then used it to make it good, so I assume you could leave it as the bad thing :P

All ConstitutionalChain does is execute a sub-prompt to GPT instructing it to make the desired changes. You could accomplish the same thing by including the same instruction in the original prompt.

Well, it recently gave me that warning after I asked it to write a regular expression for me. Those OpenAI annotators seem to be a curious bunch.

Haha, the support one is great :)

Nice web app.

Would me nice if there were an option to show the volumes in ml instead of oz.

Look behind the curtain. Feels like OZ but the whole thing is ML.

You can ask in the prompt! In honor of your 777 karma points, try "Triple Sevens (use metric)"

Nice. We started this route and ended up building https://trypromptly.com, a no-code platform to build LLM apps and chatbots.

the last one is like intercom's beta feature where it rewrites what you want to say into the proper tone lol.

All these companies selling CRUD wrappers around ChatGPT + an embeddings database seem like awfully cynical cash grabs to me. Would love to see YC sponsor some companies trying something more novel on the LLM side.

I tend to agree on this one.

Yes, there is a whole lot of jockeying to be a chokepoint (an artificial barrier between business and customer where even more money can be extracted for minimal service). I can't stand chokepoints, it's kind of like a man-in-the-middle attack on your money.

That being said, "I've created a chokepoint" seems like about 75% of all startups - finding a way to get between customers and something they need, creating a way for their startup to get money for basically delivering something that was already being delivered another way. "We do it better than the other guy" is basically bullshit.

So the cynical part is right - "if I don't get in on this cash grab, someone else will," but it seems like the nature of capitalism, really

It's an interesting point, it would be nice to see someone write and post something about this idea so it can be discussed outside a Launch post. It's kind of shitty for one company's launch to have to take on criticism of a mega-trend.

I'm not sure it's a great business model - seems a very competitive area, low barrier to entry, and hard to establish much of a moat. But I am struggling to see what you mean by "cynical cash grab".

YC should rename themselves to GPTC

What's wrong with cash grabs? That said, I don't get the platforms worth. Can someone explain it to me? What's a use case for this?

We think our current product makes it alot easier to work with documents/embeddings to help build that initial prototype, and once its deployed, tools for versioning and logging. One example use case is ingesting previous support chats and docs to build a "copilot" for your support team.

What novel approaches are you thinking of?

So many YC LLM companies.

Were you all early LLM adopters?

Or is this a brand new idea post-GPT-3?

Or did you pivot from another idea once GPT-3 took off?

Our original idea was based on an internal tool at Google from this paper https://arxiv.org/abs/2203.06566, but we took it in a different direction after some early user interviews. Not going to pretend we were using GPT-3 when it first came out, but we were always set on building dev tools in this space.

i think you know the answer to this haha. i imagine less than 1% of all of these are early LLM adopters, and just people jumping on the train. no shade though, i want to do the same. exciting times.

yeah, so what if you're late to the game. doesn't matter. did you or did you not make something people want.

So does this ultimately allow me do this workflow?

1. Say I have a blog post 2. I run the blog post through OpenAI embeddings API 3. I save the embeddings to BasePlate w/o knowing much backend stuff 4. I query BasePlate with a user search phrase 5. Ask OpenAI to give me a coherent completion

p.s. congrats on the launch!

Yes, and actually you don't need to do steps 2 and 4/5 yourself, it can be built on Baseplate, and you just need one API call to our endpoint to get the result. You can send us the blog post through our API as a file or copy/paste it through the UI. And thank you!

Funny how every hype tech (crypto, NFT, web 3, AI) gets its own color scheme and they all just run with it.

Crypto blue, NFT/Web3 purple, AI being dark theme linear.app copies?

we definitely took some inspiration from linear :)

Looks good, congrats on the launch!

If I understand correctly, I'd need to store all the data that may be needed for user queries in Baseplate. That's a blocker for us. Instead, we are developing prompts that first understand what data is relevant to a particular query, then go fetch that data, and then respond to the query (with an additional call to OpenAI). We then discard the context data.

Is this type of mechanism something you are seeing out there and you may support later?

Yes, currently for the context data it needs to be stored on Baseplate. But we are exploring that direction of tools (a logical next step), where it could do a Google search or custom API call etc..

Makes sense. Also, we wouldn't mind storing that context data in Baseplate, but only selectively and temporarily. As a cache, essentially

Yep! Should be supported - we have standard CRUD apis for managing data/documents

Congrats Andrew and Ani! Big fan of the App Builder functionality, and excited for GPT-4 integration.

Do you have any ideas/plans for providing pre-loaded off-the-shelf app configurations, perhaps specialized to a particular task or industry that can then be further tweaked? Or are you committed to a "BYOB" approach?

Anyway, congrats again. Really impressed with what's coming out of this YC class so far.

Thats actually a pretty good idea, we'll consider it! One thing is since our hybrid datasets are based off instructor-large embeddings, we'll later offer the ability to set the embedding "instruction", which can be tweaked for data from different domains. It might play nicely with an app template for a specific task/industry

Your two cited 'customers' are both W23 startups as well.

Are they really using your nascent product to support their nascent products? What plans are they on?

I feel like this can be "You scratch my back, I'll scratch yours" type of exclusive behavior.

Layup and Tennr are both using our team plan. We cited these two since they gave us quotes to use on our website. They are great products that we are excited to support! We also have several teams outside of YC on the team and pro plans.

Very very cool, something I have built personally through a patchwork of scripts, but this looks like it's been executed well.

I would use this today, but I'm concerned about lock-in. Is it possible to export/import data from baseplate?

Right now we have APIs to directly query the data, and you can export logs as csv. If there's any integration or export feature you'd like let us know!

Congrats on the launch! I didn't see an option to try the product without providing credit card details, did I miss something? (is it just cost prohibiting?)

Also the tour doesn't seem to work (and clicking restart tour does nothing)

We added that just to prevent spam, but we do have the 7 day trial. The tour is pretty quick and you can cancel if you don't see enough value!

Fair enough, and I understand, but as a fellow co-founder, I think it's a mistake. I strongly believe in pillar 2 of the PLG mantra... https://openviewpartners.com/product-led-growth/#3-pillars-o...

If you got the sign ups goals you wanted, then ignore, and I'll be happy if you share it with me as it's a good lesson for me to learn here as well. But if you got underwhelming response, you might have missed on an opportunity to really cast a wide net of potential customers. I'm not a data scientist, so I only have a vague idea of what problem you solve, but I will be less likely to send it to my data scientist, compared to a similar product with an un-gated demo / real freemium / reverse trial / time limit trial without a credit card.

I might be completely wrong here, this is just my very humble opinion. If it works for you, then I take my virtual hat off.

We might try out both options here - if you check now you'll see we changed it :)

This looks extremely useful, congrats on the launch!!

The splash page took me a bit to figure out, at first I was like like "wait is this like a UI for a vector DB?" then i figured it out. The demo video is fantastic.

i went thru the sign up process and was immediately encountered with

> Get started with a Baseplate plan: Pro 7 day trial then $35 per seat

um. no? im not handing my credit card to you on the first date. take me out for dinner first.

It should let you trial without a credit card - let me know if thats not working!

Great concept, bravo!

Here is some feedback:

1. I am trying it right now, but it seems some of my documents are not getting embedded.

I've tried adding 3 documents from their URL. I can see them if I hit Datasets > my dataset > Documents > View. But there is nothing (zero rows) in the dataset. And when I try using the Playground, my context (result of the search) returns nothing.

Am I doing something wrong?

2. It seems to work if I manually upload the same documents in the UI. However, for some reason your upload system restricts the file types, so I can only upload them as .txt files, but not as .md files.

3. I don't see a way to show a delimiter in-between the returned search results in the {{context}}. This means that the context is one mashup of content from several documents, and it confuses GPT sometimes. I would prefer to get "n" context variables that I can use as I want in my prompt: {{context-1}},{{context-2}} and so on.

4. Similarly, I don't see a way to have the metadata included in the "context". It's too bad, because there seems to be no way to tell the LLM that "this content is in document ABC and the other is from document BDF". If it was possible, then the LLM would be able to point the user to the right resource in its answer. So I would prefer {{context}} to be a JSON (which GPT models understand perfectly) of the chunks, with their text and metadata, rather than just a hodgepodge concatenation of all the chunks.

5. I need to be able to provide "reference links" for a completion. Is there a way to identify the chunks used in the API response for completion? Right now it seems like I need to run a bunch of queries:

- a `search` query to get access to the relevant chunks (rows), which doesn't tell me which document they are from,

- a `/api/datasets/{id}/rows` query to list all the rows and find the associated documents for each (which could be a request with a gigantic payload or multiple paginated requests)

- and then the completion query.

Couldn't the completion query return : here are the rows used in this completion, and here is the associated document for each row?

I am loving the concept of Baseplate and we'll very likely integrate this right away as it solves a big need we had for an upcoming feature of our own. Nice work, this is impressive!


Thanks for the feedback Sébastien! 1. Are you adding documents through the API with URLs? URL parsing is only supported through the UI right now, through the API it will just add that url as the metadata. 2. Will add md support soon! 3/4. We are working on ways to give more configuration to how the context is formatted like you mentioned, keep an eye out for updates! 5. The response from the completion API should return a "search_results" array which includes the chunks and other columns/metadata/images. If the array is empty there may be something wrong in the search config, happy to hop on a call some time or help in our Discord

Hope this helps!

Thanks, I appreciate the thorough response!

I would love it if we get more customization on the context. I just tested on a use case that is close to what our app needs for the back-end, and I'm getting chunks that are too short. It seems I can set how many chunks come up in the context, but not their length.

Other than that, you've got a subscriber. Congrats! I've just entered our CC number.

Glad to hear that! Btw you should be able to configure the chunk size and overlap when you upload through the UI, or when using our /upload API (https://docs.baseplate.ai/api-reference/documents/upload-doc...).

Perfect! I played around with this and got exactly what I needed. Thanks :)

Can I use OpenAI embeddings to transform code from one language to another? Let's say I have a corpus of example code in JS and the same exact code in Python, with the goal of feeding the modal arbitrary JS and getting Python back. If the files are too long to fit within the OpenAI token limits as prompts, can I use embeddings for that purpose?

Trying to understand your use case more, but if you already have a corpus of example code in JS and in Python, why the need to use the model to do the transformation?

That's just an example, but the end goal is feeding arbitrary code in JS and getting code back in Python. Think of the existing corpus as training data.

Or do I have to wait for GPT-4 expanded contexts to fine-tune with prompts like:

    Python: Python code (4-8k tokens)
    JS: JS code (4-8k tokens)

The embeddings on their own are insufficient for this. You need some kind of sequential model to generate the code.

It should be possible to build your own model to do this instead of GPT-4 if you are so inclined. I don't know how the quality would compare but there are various specialized code-specific models already around (and more coming) that work quite well.

What are the best ones for code? I’m looking to write and measure the coverage of generated fuzzers and unit tests.

I think the Salesforce ones[1] and the CarperAI[2] one.

BigCode from HuggingFace will be coming soon too.

[1] https://huggingface.co/Salesforce and expand "models" codegen-mono is Python, codegen-multi is multi language I think.

[2] https://carper.ai/diff-models-a-new-way-to-edit-code/

Have you tried asking GPT-4 to convert the code now, without examples? Or with just a few examples?

I feel like this should work? If the original code is too long you may just need a clever chunking strategy

Genuine question - what’s the moat for such app?

Something we think about a lot!

Under the hood, we've built an orchestration layer that keeps a database, a vector database, and storage in-sync. For the startups we work with, this has to scale to thousands of documents with high throughput.

We're continuing to add more features to make this even better for production use cases: RLS/auth, vector ranking, caching, etc.

Still a work in progress, but excited to build on top of the platform we have now.

I can only think of if the UX is really good.

But that still doesn't seem to have enough moat regardless.

UX isn't a good moat?

Oh that's neat. I made something somewhat similar recently: https://dopplerai.com/. It's a simple API that handles embeddings, vector database interactions and abstracts over Langchain. Each embedding belong to a user.

Neat! just fyi i think that link is broken

https://dopplerai.com/ is what I think the OP was referring to, if anyone else tried clicking.

Thanks! I corrected the link now.

Hey looks very cool. Curious why you chose BASF as the example? Are you targeting engineering market in particular?

No reason really it was just a random kinda large PDF I found online - we're not targeting a specific market and think we could provide value across many different industries

Congrats on the launch, Andrew and Ani. This is most of the backend stack any LLM app needs. We built similar infra when we first started using Open AI's completions API in our product last year.

With respect to GPT-4 and image inputs, how do you see search working with such inputs on the platform?

This is something we'll need to figure out since we don't have specifics on how the image is fed into the model. An example (very basic) use case we thought of is maybe a database of apartment listings & photos, and being able to ask "Do apartments on 4th street have a lot of natural light?"

Looks interesting and useful. How much data do you typically handle? And I imagine customers work with other DBs and stores alongside Baseplate?

BTW your discord link is no longer active on the landing page. Mind sharing a new link?

Right now typical usage is < 100 MB per org, but theoretically it could infinitely scale with auto-scaling. And yes heres a new link: https://discord.gg/JCqEzCZ4FM, should be updated on our site as well

Cool stuff! We are looking to build an application on this or something like this soon. Do you support chat completions or is it just text completions from OpenAI? In other words can I use gpt-3.5-turbo or gpt-4 with this?

Yes, we support both those models!

Great, I think updating the docs to note that it accepts both might be helpful. As I recall, if you try to use those models with the /completions API in OpenAI it throws an error and makes you use the chat/completions API. I haven't messed with it a ton though, and I could be wrong here. I have been watching this space and there is a lot to like about this product.

Congrats on the launch, really interesting stuff as someone working on a AI backend rn.

Your twitter icon links to a bad link with a your domain and a hash before the twitter link

thanks for the heads up! Should be fixed now :)

Are you using HyDE to generate the embedding vectors?

For standard datasets we use OpenAI ada embeddings, for hybrid its instructor + SPLADE. The HyDE toggle in the context variable feeds the query to a prompt first ("Generate a document that answers..."), before embedding. I think in the paper they use the contriever embedding model but we just use the ones supported on our platform

I think we will be hearing a lot more about HyDE. It’s a neat trick.

The UI looks awesome. Do you mind sharing your tech stack on the backend and the frontend?

Yea! Our FE is React/Next and BE is a mix of Supabase, Pinecone, BigQuery and QStash. and mix of HF endpoints and PoplarML in our batch for some embedding models

thanks for using Supabase andrew.

Are you using it to store the embeddings (pgvector)? I'm asking to see where you're differentiating between supabase & pinecone. We get this question often and it's interesting to hear what others are doing and where they feel pgvector isn't appropriate

Hey! Loving the experience so far - we started the project before pgvector was supported on Supabase, and we want to support hybrid search as well (don't think pgvector supports this?)

> hybrid search

Yes, I believe it can be done but it requires a bit of work on your side. I'll see if we can come up with a demo and/or postgres extension which handles this for you

looping back here. The langchain community just released a version of Hybrid Search which using postgres full text search. It's clever, and IMO probably a better approach than just sparse/dense vectors:


Will check this out - it seems like its doing two separate searches where you specify two separate top K values. Curious what the trade offs are between this approach and weighting sparse/dense vectors

Thanks guys for sharing it. This really helps us in the community to look for the architectures that we should focus on and the new interesting developments.

Hi there, is there any integration guide on the frontend?

Hi, when you create an endpoint it should give you API examples based on the deployed app. Also docs here: https://docs.baseplate.ai/api-reference/completions. Hope this helps!

Do I have to have an OPENAI_API_KEY to use this?

Yes, you'll have to bring your own API keys

Looks very slick, congrats

this is awesome

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact