Some quick context: I was inspired to build this by this HN post earlier today [1]. So thank you glorf for making the recipe dataset available.
Thought this would take me 1-2 hours to build, ended up taking about 6 hours - engineering estimates and all!
> Details about the Tech Stack:
The dataset has 2,231,142 recipes and is indexed on Typesense [2], an open source alternative to Algolia/ElasticSearch that a friend and I are working on.
The UI was built using the Typesense adapter for InstantSearch.js [3] and is a static site bundled using ParcelJS.
The app is hosted on S3, with CloudFront for a CDN.
The search backend is powered by a geo-distributed 3-node Typesense cluster running on Typesense Cloud [4], with nodes in Oregon, Frankfurt and Mumbai.
Jason, sorry to ask here rather than read your GitHub docs, but how does Typesense fare against non-romance languages that can't be segmented by whitespace?
I'm guessing you meant to say logographic languages. We don't yet support tokenization for logographic languages (like Chinese, Japanese, etc) but it's on our medium-term radar: https://github.com/typesense/typesense/issues/86
Basically...but I didn't know to use that term! So thanks for teaching me. Then there's also languages like Thai that are not whitespace separated on the word level, but that use an alphabet. So...I meant more 'non-Latin' but I think that's not actually a tight category. It's actually quite difficult to come up with the right term. I guess I was trying to be too clever, the best term is probably "non-whitespace delimited languages". Thanks for your response, and awesome speed to index the dataset and have it up and running in the same day.
Could I ask you a few more questions? What was the dataset size? What was the size of your index? How long (and how much RAM) did it take to index the dataset and what machine (and how many cores) did you do it on?
> Then there's also languages like Thai that are not whitespace separated on the word level, but that use an alphabet.
I did not know that! Good to know.
> What was the dataset size?
2.2GB in size, with ~2.2M records
> What was the size of your index?
2.7GB
> How long (and how much RAM) did it take to index the dataset
It took about 8 minutes to index that data. Typesense stores the entire index in memory, so the index took 2.7GB in RAM
> What machine (and how many cores) did you do it on?
It's running on a 3-node cluster, with each node having 4vCPUs and 8GB of RAM. The nodes are distributed across data centers, so search requests are served by the closest node (like a CDN).
Yeah I meant some concept like "non-Latin derived" or "non-Roman alphabet" languages but then there's Cyrillic, etc. I was pretty sure "non-Romance" sounded like that right term, but not totally sure. I looked it up after and yeah, it wasn't. Actually I have no deep idea of the terms in this and just grabbed the first term that came to me. I thought I did pretty well and I appreciate the learning experience!
You have a UI bug: dismissing the modal recipe popup isn’t entirely reliable, and the site can get stuck in a state that doesn’t allow user interaction. This even survives the back button.
Thanks I was just looking for a cheap alternative for a search engine just today (Algolia is cool but very expensive if you need to index millions of records) - I will check out Typesense.
Type sense is looking great! I was going to use it on a side project. Not sure why I didn’t but it must have been missing something I needed. Been using meilisearch but I’ll definitely be checking out typesense again.
Typesense would indeed be good for a general web page search system, just like Algolia. In fact, even Algolia stores web page data as structured JSON entities.
Did you use CloudFormation to crate the infra? If not, I'd love to hear some details on how you did this. Any API Gateway being used? Seems to be offline at the moment.
We run similarly on CloudFormation. Interesting with the ‘aws s3 cp’ command. I started using ‘aws s3 sync —delete’ nowadays after having issues with pre cleanup required for ‘cp’.
In my case since the assets have fingerprinted filenames and index.html references them, if I delete old files with each deployment, then a user who has a cached version of index.html will see a broken page. So I just leave old asset files as is.
Thought this would take me 1-2 hours to build, ended up taking about 6 hours - engineering estimates and all!
> Details about the Tech Stack:
The dataset has 2,231,142 recipes and is indexed on Typesense [2], an open source alternative to Algolia/ElasticSearch that a friend and I are working on.
The UI was built using the Typesense adapter for InstantSearch.js [3] and is a static site bundled using ParcelJS.
The app is hosted on S3, with CloudFront for a CDN.
The search backend is powered by a geo-distributed 3-node Typesense cluster running on Typesense Cloud [4], with nodes in Oregon, Frankfurt and Mumbai.
Here's the source code: https://github.com/typesense/showcase-recipe-search
[1] https://news.ycombinator.com/item?id=25356156
[2] https://github.com/typesense/typesense
[3] https://github.com/typesense/typesense-instantsearch-adapter
[4] https://cloud.typesense.org