By the way, to "bootstrap" a business is short for "pulling yourself up by your own bootstraps" (yes, it's an old phrase) and means not raising any money.
Just a to bootstrap a computer is to start it when it has no code loaded or running. Nowadays it's in ROM so the term is rather obscure but we used to toggle a little bootstrap program in from the front panel or, if you were lucky, load it in from paper tape. That program was very short, just enough to get some more code loaded and start it.
> to "bootstrap" a business is short for "pulling yourself up by your own bootstraps" (yes, it's an old phrase) and means not raising any money.
Today I learned! Technically Kagi was bootstrapped up to this point (I've invested $2MM+ of my own money to get us this far). This is the first external fundraise (as noted in the annoucement) and what makes it very special to me is that people participating in the round are Kagi users who were already a part of our journey.
and it seems like many actual users of Kagi (and Orion?) have contributed, so this isn't a (completely) a "big VC firm will enshitify this". This gives me hope. I wish I would qualify to invest in things like this...
If you have the money to participate and lie about having/earning more money then you qualify.
There is no constitutional way to prevent you from investing based on net worth, so the law places consequences on the companies offering investments.
And the only time this affects the company is if they were relying on a regulatory exemption that also allows unaccredited investors, in which case you should have just participated as an unaccredited investor and they were supposed to verify.
Otherwise, the law says “self-certify”, which is the state sanctioned term for lying.
It doesn't need to be a protected class and is not how I would argue
If a fine or sanction from the government was applied to the investor themselves based on net worth, I would argue 5th amendment and 14th amendment equal application of laws, and 1st amendment's freedom of speech citing the Citizen's United case where money movement is speech
I think that would be resilient in most circuits and certainly scotus
$670k investment is not "bootstrap money". It's literally the opposite of bootstrapping. What is the value in this attempted narrative? Is it the entrepreneur equivalent of "I don't work with big data, only 500 billion row sets but..."
The typical formula is that it costs double a person's salary for the total budget. So really you're saying 5.5 years and 2 people is a low number so maybe you build a team of 4 and then the runway is an excruciatingly short 2.75 years.
This made me happy, too. I've been delaying becoming a paying customer because I wanted to see if they were going the VC route first. This is a signal to me that they aren't, so I've signed up.
I think this is because our eyes-brain system is optimizing the read operation by scanning the words and maps them against a list of words we already familiar with, thus, sometimes it got the wrong meaning. This trick was used by some brands to mislead people to buy some products with a familiar brand name.. like “Adibas” and “Reedok”.
> makes me hopeful that Kagi will stay close to its original mission
I really hope so but no one can guarantee this would be the case if the company get bigger with more money being thrown there.
In its first years, Google’s tagline was “don’t be evil“ but they couldn’t deliver the promise.
Just that it was a smaller number than would you are expected to sell your soul for with VC cash. Only raising what you need to grow "naturally" and retaining control is a good thing in my book, but that isn't what a lot of people want to do in that situation.
I know a lot of people here thought Kagi wasn't worth it, but I'm an early adopter who has been a happy customer for some time now.
Search is really important to me, and it isn't just about looking up businesses or Wikipedia pages. I read lots of academic papers and just want it to be easy to separate the wheat from the chaffe. Kagi makes that really easy with its lenses. Sure, Brave search has something similar, but I like Kagi's interface better and having to pay them gives me the sense their service won't just be deprecated or significantly changed at the whim of a CEO trying to satisfy advertisers or whatever. Having insight into how many ads and trackers on a page is nice too, as well as being able to demote and block certain domains. It's nice that I can block a domain and it's also blocked on my phone.
What's especially important to me is the ability to wrap my queries in quotes for literal string matches. Granted, no search engine is perfect at this anymore because the web has become so huge, but Kagi seems to be the only search engine I use that gets string matches right more of the time and doesn't give me fake results when there are no actual matches. That's what pisses me off about The Google. And though I once used DDG, I found that the quality of search started getting worse and now it seems like quotes barely even work with that search engine now.
I hope Kagi succeeds. It doesn't need to compete with The Google. Just be good enough for those willing to pay for it.
I forgot I signed up for a free Kagi account a while back and reading some of these comments has reminded me to start using it. I don't need it to offer a huge leap in quality as long as it's actually good and doesn't degrade in quality.
G search is right out for me because I avoid any G products whenever possible. Also it's quite bad these days. So I primarily use DuckDuckGo, except it's honestly not that good either. And I'm very suspicious of how much money they have for advertising with a free, privacy-focused product. It just doesn't feel sustainable. I could be wrong! Maybe their non-personalized ads are more sustainable than Kagi's subscription model. But the search kind of sucks. Actually, sometimes I do end up using `!g` to improve my results (rather: to actually get relevant results), and it feels bad.
So I'm going to switch over to Kagi and see if that can deliver the results I want/need. If so, I'll happily pay. I might even pay $25/mo... it's a lot for a subscription, but it's not a lot when compared to my monthly income, so if it makes me job easier it will feel like a fair trade.
It's basically a way to restrict search results to particular websites. So you could create a "programming" lens including Stack Overflow and Github, for example, and a search using that lens would only return results from those sites.
Can lenses be shared with other users? And is there some kind of shared repository for them where we could crowdsource for example a lens with the urls of all Python docs of popular packages?
There's actually a built-in Academic lens they provide, although I haven't really used it much, favoring my own "Research" lens. It's not very sophisticated, but since I'm mostly into research related to health, I use this a lot:
"Goggles enable any individual—or community of people—to alter the ranking of Brave Search by using a set of instructions (rules and filters). Anyone can create, apply, or extend a Goggle. Essentially Goggles act as a custom re-ranking on top of the Brave search index."
The other day I wanted to make a quick Venn diagram as a joke, and I was sure hundreds of people have made a simple tool for this and put it on the web.
I did a search (on kagi) and all the top results wanted me to sign up or subscribe. The joke I wanted to make wasn't worth it for that.
I was reminded of google in present times, loads of results that are almost what I'm looking for, but filled with frustration instead.
I don't feel entitled here, but I do know that in the old web, back when I fell in love with google, I'd have found some hobbyist site that did this without trying to get anything from me.
Anyway I then clicked "non-commercial" and was instantly teleported to the old web, the first result was a free simple tool from some hobbyist.
Can someone help me a little? I tried Kagi and when my 100 searches ran out I stopped using it. I really didn't like the feeling that I needed to meter my searches, it added a barrier to every search I did (should I use Kagi for this or should I just use Google?). It sounds stupid but literally every search I did I had a pang of worry that I was "wasting" a search.
I also didn't experience a huge increase in search quality that I noticed. I didn't really get into the Lenses stuff even though it seemed cool because I just don't think about things like that when I'm searching. I guess 25 years of googling has conditioned my way of thinking about search.
Kagi feels like something I should really like and I'd be fine with paying for it but I guess I need some more "tips" about how to get the most out of it and/or how to change my way of thinking about this.
I've been using it since the beta, and am still paying for it. I shared your concerns at first, but in practice I've never run into the limit. In fact, checking my history, I've never actually gotten close to the limit. I pay $10/mo for 1500 searches, and most months I do less than 500 searches.
I don't think the search quality is anything special. I have never figured out how to use Lenses, and I don't even care to try them again. I like it for privacy, and I like that you can give weight to different websites, and even block ones you don't want to see again. I know there are Google plugins for blocking sources as well.
I don't have any tips. I think Kagi is about as good as DuckDuckGo, and the advantage it has is that it costs money. I'm at the point where I don't even trust a nominally privacy-focused search engine like DDG not to drift into the unethical. For some reason—maybe naivety—I think a company that takes money in exchange guaranteeing my privacy is theoretically more reliable, if only because it would be too audacious to outright lie about that.
They can claim what they want; the fact remains that I have to trust them. If I use Brave or DDG or even Google, I can do things to obfuscate my identity from them. Kagi is the only search engine that requires users to authenticate, and ties that authentication to a method of payment that can be trivially tied to the user's identity. As far as the user is able to verify, that makes them the least private search engine.
Even if they're telling the truth, if they receive a national security warrant, they'll start collecting that information secretly. Hence my interest in a warrant canary.
That is fair. Few things we are doing that I'd like to point out:
- Users can register with any email. We really do not verify or need this apart from having a way to have an account, which we need to authorize payments
- Some users suggested using crypto to obfuscate identity but this is not as straightforward and does not work as good with subscriptions
I initially worried about surpassing my quota at first, but I've never come close to it (I'm on the grandfathered $10/1500 plan).
For me, I notice better search results than Google (not night and day, but noticeable). The ability to block/promote/demote certain sites is fantastic, and I like the URL rules you can create. For instance, I've set Reddit links to open in old.reddit.com, and I've set Youtube links to redirect to an Invidious instance.
It's got some other nice things—I like the universal summarizer a lot, and the search result summarizer has come in handy many times. But mostly, I just like using something that isn't Google without feeling like I'm compromising (which I did feel with both DDG and Brave).
It’s the personalisation element combined with slightly better results that makes it so much better. Plus, I don’t have to block any ads or tracking. Google without an adblocker has become harder and harder to navigate.
On the other end, I’m seeing Google results get really really bad. Mostly just missing results. When I search the same term on bing there are pages and pages of results. I’ve no idea what’s going on at Google. I don’t use it logged in and don’t know if that affects the results.
I felt the same way about query metering and ended up cancelling my Kagi subscription once they raised their prices. I still occasionally check their Discord server for updates and it sounds like they're trying to bring the cost of unlimited search back down to around $15.
Regarding search quality, I don't think it's a revolutionary improvement over Google, but for me it's been noticeable during the time I used it, where as Neeva and Brave have been very noticeable downgrades. I think Kagi shines once you lean into the tools they provide and start upranking and downranking different domains.
Keep in mind it's been a couple of months since I last used it and they've added the search result summarizer and some other AI tools since then that might help tip the balance their way.
$15/unlimited would be acceptable to me, $25 just seems way too high. I never played much with the summarizer since it was metered as well and I get squirrely about using up limited things (see also: my inventory at the end of every RPG, so many potions that "I might need later").
The ranking and lenses seemed very cool to me but I burned through my free tier before I could really use them. Maybe I'll try again at the $10/mo price.
I'm completely the same as you so I did the following: I took the $10 plan and watched my usage for a week. I seem to hover roughly at about 30-40 searches per day on average so I upgraded to the Family plan. Yes, I know, it sounds weird but now I've got 1400 searches and just didn't invite anyone. And I don't have to worry about if it's going to be enough because my own data says that it's going to easily be enough.
Lenses are actually why I subscribed mostly. I search for organic data often while trying to find help for a product or the opinions on a service or something like that. Regular search is ok, changing to the Forum Lense is mind blowing because I only get discussions from real people.
There is no problem, except in your imagination. Each additional search costs a couple of cents, why would you worry about such a small amount? When you're driving a car you're using cents of fuel every minute, do you worry about that when you're driving, counting the cents?
You've literally managed to find the most insignificant thing to worry about in existence.
But if quality is not impressive to you there's no reason to use the service.
Wow, not really sure where the hostility is coming from. I'm literally asking for suggestion/motivations/reasons to pay because I feel like it is something I want but I need help getting over the hump of paying for search. I also outlined that I didn't experience an increase in quality and posited that I might be using it wrong or not taking full advantage.
> You've literally managed to find the most insignificant thing to worry about in existence.
It's clearly not "insignificant" since others here have voiced the same concern or annoyance. I think we have enough research/studies to show that metering/limiting has an effect on how people use something even if the overage fee is low.
I used the trial and I wasn't sold on it, I was asking for people to tell me how they get value out of it, you're doing literally the opposite and being a jerk about it. If anything your comment makes me want to just forget about Kagi.
I really didn't intend to be hostile in any way, and I apologize if that's how my comment looked.
We as humans tend to trick our own minds into worrying about insignificant things and sometimes need somebody to help us snap out of it. I was trying to help you with that, but instead you think I came across as a jerk. It's not worth your time to worry about a couple of cents for a search. I know many people have the same concern and worry as you, it is still not rational.
> I used the trial and I wasn't sold on it
Then there is no reason for you to use it - even if it was free. The only reason to use paid search is to get better quality results. There is no special maneuver to use with Kagi, it's just typing the query into the box like on Google.
Neeva - cofounded by an ex-Google exec also provided a paid search experience with similar features. They have since shut down and in their blog post pointed out that the issue they faced was not convincing people that they should pay for search but rather the fact that distribution of their service is difficult (i.e. being browser defaults, at work, on people's phones).
What's Kagi doing differently to be able to succeed?
That's hardly a relevant criteria. HN consists of a niche audience of technical users already interested in startups and search technology. The real criteria is whether Kagi will be able to scale beyond a niche market, and establish a sustainable mainstream business. OP's question is valid, but it can't be answered by outsider speculation. Hopefully someone from Kagi is able to address it.
This investment is a good sign, though. I'm rooting for them. We need more competition in this space, and a proper business not hijacked by advertisers is always a good thing.
> HN consists of a niche audience of technical users already interested in startups and search technology.
Isnt that how firefox got popular, techies started to use it and then convinced others to switch. Of Course in the end it was no match against Google marketing for Chrome.
True, it could be. I'm just not sure it can survive as such when its main competitors are billion-dollar corporations.
But maybe there's room for smaller search engines with alternative business models to compete with each other. marginalia.nu is another promising startup.
You're forgetting that the "niche technical audience" has something marketers and influencers and other kinds of pushers don't: competence and credibility. And (not relevant here, but to plenty other products) control or influence over procurement process in SMBs, corporations, universities and government facilities.
Nearly all of us in the "niche technical audience" are the personal IT support departments of some subset of our family, friends, and their friends. They come to us asking for advice, or to set their computers/phones up, or to fix them after they "caught viruses". However begrudgingly we do that, it puts us in a position of power - they listen to and trust in what we say, and accept uncritically what we do to their machines. And, our interests are mostly aligned with theirs - even if we don't care about particular friend-of-a-friend's happiness, we'll still do them good so they don't have to come back with more issues any time soon.
This is how Google Chrome spread. This is how Firefox still survives. This is how several brands of anti-malware software spread - a mistake that's now difficult to undo. This is how AdBlock Plus became a thing, and how uBlock Origin is now replacing it. All these trends and more, I participated in first-hand. People still remember and follow advice I gave them over a decade ago (which isn't always good - see the anti-malware stuff).
And so is the case with Kagi, to a degree. I'm paid user for 1.5 years now, happy with the service. I recommend it in relevant discussions, I mention it to people who spot it - but since it's a paid product, it is a tough sell with general population. Still, I try to spread the word, like I do with any other good and non-user-abusive tool.
That said, this news makes me somewhat reluctant to recommend Kagi. I'll keep using it because it provides me immediate value for reasonable price, but taking investment often is a Faustian bargain. In this case it's not as clear as with regular VC backing, so I guess we'll see where this goes.
I think you're overestimating the impact technical users have on the survival of a business. Word of mouth and technical influence inside SMBs and corporations can only do so much, especially for a business that doesn't have a corporate sales strategy.
> Nearly all of us in the "niche technical audience" are the personal IT support departments of some subset of our family, friends, and their friends.
How has that influence worked out for you so far? By that logic, all our family and friend circles would be running Linux, using OSS, and be more mindful of their privacy. IME my attempts at convincing others to use the tools that I use has mostly been met with lukewarm response, or even arguments in favor of the tools that they already use. Convincing someone to change their computing habits is not just a matter of being considered an expert in the field.
Google Chrome spread because it had the resources and influence of a billion-dollar corporation, and the marketing budget to reach millions of users. Firefox is barely surviving, and most of it is due to its corporate contracts, not because of its technical audience.
> How has that influence worked out for you so far? By that logic, all our family and friend circles would be running Linux, using OSS, and be more mindful of their privacy.
Asking normal people to use Linux and OSS is a bit too much, but I think that Macbook sales have had at least a bit of help by being endorsed by the technical crowd.
And iPhones and iPad, too. They are what you recommend to your family members after one too many requests to "fix" a "virus" on their Windows PC or Android device.
Kagi is not convenient in the sense of browser search engine awareness. They do plug-in type thing. Needs work from the browsers for better integration.
> What's Kagi doing differently to be able to succeed?
Define success? It outperforms every other search engine in terms of quality of results and at least from what I understand, they're not too far off from breaking even on their salaries.
Neeva was a VC funded abomination and it showed, they even had deals with "big brand" companies like LastPass to offer bundles, but their core product sucked. Neeva search results were often worse than Google. That's not the case with Kagi.
Its not hard to define success. It means having more revenue than their cost, and paying all employees market rate salary(including the founder himself).
Everyone has their own definition of success. For many, a "successful" tech startup is one that eventually lands a $100M+ buyout, or gets millions of paying customers, or some other metric that's far beyond "simply" arriving at a sustainable business that you (and I) mentioned.
Unfortunately, it's a common bar for the continued existence of companies. I'd say successes if the company still offers a service in 5 or 10 years. Unless the operators an employees are all independently wealthy, you need to break even
I tried Neeva and immediately hated it - it felt slow and clunky. I tried Kagi and immediately loved it enough that I've been paying ever since they let me. I think the big thing that Kagi is doing differently to Neeva is just being good.
I know right! I can't remember the last time I had to click through to the second page of Kagi unless my search was a futile one in the first place (ie: I hadn't understood the problem enough to know what to search for)
They have their own browser, the Orion browser, to funnel people onto their search engine.
Their search engine is also better than google for me for most use cases already, it has a clean and smart design with well thought out features that add value.
I hate that Orion is not cross platform and limited to apple ecosystem.
With that said it is native and my default choice for macOS and iOS.
No telemetry, ad blocker built in, performant and supports extensions from chrome and firefox - on iOS too.
Hands down the best native tree style tabs of any browser on the market on top of that and easy tab syncing.
It’s still in development but it’s stable enough for me to be my main browser.
These two products show some serious technical ability from a small team. I bet on companies who invest in and show technical ability over those with VC money, big names focused on marketing and growth.
At least for iOS/Mac but 99.999% sure for other browsers as well, they could write an extension that rewrites "https://www.google.com/search?q=some%20search" into their own search url right? Then keep Google as default search engine. Installing an extension sounds like a hurdle people paying for search are willing and able to jump over.
I love kagi but I hate this. It's not their fault, it's just a limitation of the os but it's such a hassle when you have to solve a Google captcha to get kagi results. I'm considering just removing that from my phone and switching back to Google there.
You can switch default search engine to DDG or Ecosia, Kagi extension will still redirect you away. Or use extensions like xSearch or xEngine, they seem to be better at redirecting
That's exactly what I do. I feed searches to Ecosia and the Kagi extension handles the rewrite. Works well on Mac and iOS. I only use Google explicitly with the !g operator (which I use sparingly, and often get the same results or slightly worse) or for Google Images (which are better for now)
I didn't realise Neeva had shut down. I tried it for about a week, but stopped using it because it wasn't as snappy as Google. In fact, I always seem to return to Google from every other search engine I try because of its snappiness — decades of tuning seems to have paid off.
Kagi is the first alternative search engine that I actually use regularly instead of Google.
I used DDG for years, but more than half my queries ended up being prepended with !g because their results just weren't very good. With Kagi, I fall back on Google maybe once a month, and usually Google doesn't end up finding anything better.
Neeva did only one thing right: they cared about privacy, but otherwise they really did not do a good job. Their search results were worse than google, their load times were even slower than google, they tried making too much money too quickly, and they jumped towards fads too quickly.
"Kagi is building a novel ad-free, paid search engine and a powerful web browser as a part of our mission to humanize the web."
With 670K?
Cuil went through about $30 million to develop a standalone search engine.
And that was fifteen years ago, when search was simpler.
There may be a market for a search engine company that profitably runs a low-cost operation with very few ads and makes real efforts to keep out spam. Everybody is fed up with Google and Bing. There's a great opportunity here to disrupt the industry and destroy a trillion dollars in market cap.
I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important.
The absolute worst way you can arrange a search engine project is as some sort of manhattan project with a humongous budget and an army of professors and experts.
History is littered with bold and ambitious Google killers that went nowhere.
You can throw almost any amount of money at an operation, and it will gobble it up. I think search in particular is very prone to bloated R&D budgets and various forms of mission creep.
Yet the underlying reality is that software development scales very poorly with organization size, and the larger your organization is, the harder it is to steer and the more difficult it is to make the right calls. A squad of 3-4 motivated and talented guys is absolute peak get-shit-done.
>I think a small operation is exactly the sort of outfit to do it.
A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users.
As a consequence, they will always be at the mercy of Google/Bing. The range of things they can innovate on will always be limited. And the situation can only get worse as people start to expect more AI functionality.
I doubt that you can build a sustainable niche product if the effort you have to put into it is just as big as if you're building for billions of users. Having few users is not what defines a niche. Niches are defined by specialisation.
> A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users.
What do you reckon the cost of doing this would be?
Just doing the napkin math for say a Mojeek sized index (couple of billion docs) doesn't seem to justify a particularly astronomical budget.
>What do you reckon the cost of doing this would be?
That is indeed the key question. I tried to find out before commenting but the information I found is very vague. Internet Archive spends millions per year, but their index is updated far too slowly for a search engine. I have no idea what it costs to create a Google-size index.
Do you think the Mojeek index is good enough to compete with Google?
Google started becoming an answer engine from ~2015 with the introduction of "People Also Ask", and arguably earlier than that [0]. Mojeek.com is a (information retrieval) search engine, and we resist the temptation to also become an answer engine. So you might say we do not compete with Google. afte all we have a very different business model and proposition.
As for the index; this underpins mojeek.com and our API; which customers use for search and/or AI. Common Crawl is ~3.5 billion pages and underpins LLMs. Our index is ~7 billion. Who knows what (else) Google, and Bing, do with their index ;) ?
>Google started becoming an answer engine from ~2015 with the introduction of "People Also Ask", and arguably earlier than that [0]. Mojeek.com is a (information retrieval) search engine, and we resist the temptation to also become an answer engine. So you might say we do not compete with Google.
I'm sure you know your users well after so many years in the search engine business, but having read your article I must say I find your approach risky. You seem to be betting on search engines and answer engines continuing to be complementary rather than substitutes.
But we are not the ones making this decision. Users will be making the decision in light of the newly available AI capabilities, and they will be making it with complete disregard for the health of the web, as is their nature :)
The "funny" thing is that big publishers are as happy right now as I haven't seen them in the past 25 years, because it is so completely obvious that chat AIs will destroy the web unless big tech starts making big payments to big publishers. As you rightly say, small businesses and publishers will be collateral damage.
But how do you make sure you're not collateral damage as well?
It's basically the safest position you could be in.
An LLM to digest results of a classic search index is greater than the sum of its parts. An LLMs that is not permitted to brush up on the relevant literature before answering a question generally doesn't produce very good answers, is prone to hallucinations etc. A pure LLM design isn't even a serious contender in the answer engine space.
Why would publishers allow you to crawl their sites if you're not sending them any traffic?
The big publishers certainly won't let you do that as they are selling their data to Google, Microsoft, Facebook and whoever else has the money to train a fully fledged LLM, which is certainly not everyone.
Because it lets them sell data to other parties than Google and Facebook? That's actually pretty great. Only having a 1 or 2 customers kinda sucks.
A search engine partnering with an answer engine may not send traffic, but the answer engine is a potential customer for the websites the search engines direct them to.
>Because it lets them sell data to other parties than Google and Facebook?
Only indirectly by charging search engines for access to content. It would be an entirely different business model that requires a complex set of agreements between publishers, search engines and LLM providers.
Granted it's not impossible and certainly worth considering if you have search engine expertise but no money to train an LLM.
> Do you think the Mojeek index is good enough to compete with Google?
As a back-end for something like Kagi, it's sure getting there. Most of what sets Google apart is their exceptional level of user profiling. The actual indexing technology is likely on par with most of their competition.
Of course very little of that enters into API queries.
Internet archive snapshot the whole page while google only index the first few kb of text data on every page (forgot the number, was it 100kb max per page?), so maybe it'll cost lower to build a search engine index compared to an internet preservation project?
Assuming they'll need to index 820 billion pages (the number of pages preserved in the internet archive), at 100kb each, and assuming they use a database with 0.3x text data compression efficiency, they'll need at least 24600 TB to store those text data. Assuming $300 per 16TB disk, then they'll need to spend at least $7,380,000 for disk alone. This is a lot of money just for storage and we haven't included stuff like replication and backup, indexing metadata overhead, etc.
This seems like a good use case for some sort of spam filtering. Maybe the web is that big, but what percentage of it is data someone will ever care about? I wonder if you could make a good-enough search with agressive filtering of pages before they enter the index.
A search engine doesn't index the HTML code. You're looking at a few Kb per document. You also don't need multiple historical snapshots of the document like WM retains.
So you're looking at maybe 20 bn docs, 4 Kb each. 100 Tb, before compression.
True, but then Google doesn't just download the page source and index that. They run JavaScript in some cases to get to the actual content. This must come at a significant cost. Their index is enormous as well:
"The Google Search index contains hundreds of billions of web pages and is well over 100,000,000 gigabytes in size."
Sure, download and run the javascript, but then you can snapshot the DOM, grab the text, and discard all the rest. The HTML and js is of little practical value for the index after that point.
Google's index is likely very large because they don't have any real economic incentives to keeping it small.
>... but then you can snapshot the DOM, grab the text, and discard all the rest
Yes, absolutely, I didn't mean to imply otherwise. But first you have to figure out what you can discard beyond the HTML tags themselves to avoid indexing all the garbage that is on each and every page.
When I tried to do this I came to the conclusion that I needed to actually render the page to find out where on the page a particular piece of text was, what font size it had, if it was even visible, etc. And then there's JavaScript of course.
So what I'm saying is that storing a couple of kilobytes is probably not the most costly part of indexing a page.
> When I tried to do this I came to the conclusion that I needed to actually render the page to find out where on the page a particular piece of text was, what font size it had, if it was even visible, etc. And then there's JavaScript of course.
Are there open source projects devoted to this functionality? It’s becoming more and more a sticking point for working with LLMs. Grabbing the text without navigation and other crap but while maintaining formatting and links, etc
For my specific purposes it has always been good enough to apply some simple heuristics. But that wouldn't have been possible without access to post rendering information, which only a real browser (https://pptr.dev) can reliably produce.
There are many software libraries that can output just the text from HTML or run JS. For C# there's HTML Agility Pack and PuppeteerSharp, for example. I did use them for web scrapping.
You don't need to store it indefinitely though, and there's not much point in crawling faster than you can process the data.
The couple of kilobytes per document is the actual storage footprint. Sure you need to massage the data, but that almost entirely CPU bound. You also need a lot of RAM for keeping the hot parts of the index.
You don't index HTML code, but you have to process HTML and eventually run Javascript to get the text content. Then you have to compute the word frequencies.And that means you have to use more compute power.
They changed that limit somewhere in the mid-2000s. Just as well, there's some CMS's out there where there's several hundred kilobytes of inline JS and CSS before any body text.
A hash table is not a good backing structure for a search engine.
Hash tables almost guarantee worst case disk read patterns. You use something like a skip list or a b-tree, since that's makes much better use of the hardware, and on top of that allows you to do incredibly fast joins.
That's true, but I was referring to a hash table as a mere mental model, not an actual implementation. A better name would be a dictionary instead of hash table.
You're ignoring that Kagi runs on subscriptions, so this funding is not all there is. If the subscription covers basic user costs (which it should, because they just adjusted the pricing) or at least covers most of them then the 670K can be used for growth.
Such a waste. MacOS users might be open to paying (for Kagi in general) because they're used to paying for a bunch of things other OSes get included or as freeware, but still, the market share is small (depending on the source, 10-30%). And even if many of those would enjoy a Mac-native app, there are at least some, like myself, that refuse to use single platform tools. I have a bunch of devices on a bunch of different OSes, I'm not going to use a very special browser on one of them, losing sync, history, muscle memory when switching. That's the reason I can't stay on Arc even if I quite like some of it's goals and structure.
You always need to start somewhere, Kagi is now (just) a 15 person team. There are enough Macs (200M+) and iPhones (1B+) in use to justify selecting this as the first ecosystem to target for the browser. Really it comes down to resource management and allocation in the early stages.
Well Cuil has already been mentioned, there's A9.com and Quaero; on the open source side there's mozDex, notably Jimbo Wales failed twice with Wikia Search and then Knowledge Engine.
Yeah, there's a lot of truth in this, and it matches my experience, with the caveat that the 3-4 guys need to stay very clear eyed and pragmatic with their technology choices. Kind of covered by the "talented" part but I've seen so many otherwise talented people fall into holes because of this, and I'd guess it applies doubly for building a search engine.
There's also interesting things happening in the server space that doesn't get talked about enough. Not only are the latest generation of Epycs very good, the price of RAM and especially SSDs has absolutely plummeted.
If ever there was a moment where horizontal scaling looked promising, this is it.
Perhaps the business is already sustainable and that $670k will just let them hire an additional person to speed things up a bit?
There's a lot of wasted effort in this industry. Slack has 1500+ employees just to make a chat app. Granted, it's a damn good chat app, but Mozilla is maintaining a browser with half as many people, with not everyone focused on Firefox at that.
My friend is now in a project where he billed for three weeks until all the issues with his dev account were sorted out. It remains unclear when he will be able to start actually working.
I spent two years building a web app + gRPC server that perhaps could have been just the latter.
I could go on. Point is, you can blow through $30mln easily but that doesn't mean you have to.
Per TFA, this money was mostly raised from Kagi's existing customers.
They aren't building a web crawler. Kagi is a search "client" (it is one way to build on a shoestring budget, alright), augmented by an in-house small-scale just-in-time crawler.
> From here, we take your query and use it to aggregate data from multiple other sources, including but not limited to Google, Bing, and Wikipedia, and other internal data sources in order to procure your search results.https://help.kagi.com/kagi/privacy/privacy-protection.html
I like Kagi's general vision for LLMs + Search. There's a real chance small competitors can compete with Google with clever use of LLMs' zero-shot summarization, categorization, intent recognition, and answering abilities.
Especially when google continues to be worse off year after year. Just search for my damn keywords, like you did 10 years ago, especially when I have already put quotes around them because you Re useless!
As someone who has already replaced Google with Kagi, as far as I'm concerned they've already done it.
I've never paid Google for anything (apart from with my data) but I've been paying Kagi for almost a year now. The model is different. From what I've seen so far, it's better.
The amount of money you burn through is not a good indicator of whether you'll end up with a good product or viable business at the end, IMHO
There's real restrictions, contractually, that Google has on accessing any data the a customer generates, and of which carry hefty fees for Google.
This is one of the reasons on why some Google products never launch with support for Workspace accounts. There's just too much red-tape that a team doesn't want to deal with.
Source and Disclaimer: I was a Technical Solutions Engineer for Google Cloud.
> I have to admit though, ChatGPT did make me consider dropping the subscription. Mr Chat has weakened the value of Kagi to me.
I felt the same, but I do still need web search, and I do still want it to be as good as possible, so it's still worth it to me.
And actually, as I type this I'm less sure about how much ChatGPT has reduced the value of web search for me. I start with ChatGPT for almost everything that's not "news". But I still frequently do web searches branching out from what I learned via ChatGPT. It's fewer total searches than I've done before, but each one might be more valuable. And since I'm flowing from one tool to another, not having to wade through ads and bad ui and SEO spam also becomes even more valuable.
As a subscriber to both ChatGPT and Kagi, agreed. You may be interested to know that Kagi also has https://labs.kagi.com/fastgpt. It's been pretty good in my tests so far, nothing magical just fast and similar to a search input field.
We beg to differ on the need to raise large sums. Like Kagi we are on a marathon not a sprint. We have built a no-tracking and completely independent crawler search engine and infrastructure from the ground-up having raised £3m from angels only. Cuil, Blekko, Quaero and Neeva who raised 10s/100s of $m may have come and gone; meanwhile we have been slowly building since 2004, with a user and API customer base that is also growing healthily.
Have you tried kagi? Because I have and it is pretty good. I was sold when they summarized several "listicles" (ie your typical Top10 blog spam) into one concise listing. "Yes these results would pop up for your search ... but you probably don't want to look at them".
At their current pricing, about 10k paying users would get them 600K/year in revenue. Which is what they just raised. Enough for a small team and modest infrastructure. You don't need a lot to serve 600K users.
All they need to do is nail enough value add for those users. The wider goal of disrupting Google/Bing is a different game. But getting a small company to 10K paying users might be doable given enough of a value add.
Of course they are based in the Bay area, so this kind of money doesn't have a huge runway there.
Kagi has been around since 2018. They're privately bootstrapped (not by a VC firm). 670K is not the only money that's gone into the business. 670K is how a bootstrapped company responsibly raises money.
I'm happily paying $10/mo for Kagi, I find it genuinely better* than Google already. And yes, it's genuinely amazing that it's a pipsqueak startup that's pulling this off.
* For core search, that is, obviously there's vast slabs of services like maps, translate, etc etc where they're not even trying to compete.
This demonstrates to me how distorted searching has become. The early days of the web. The standard thing was to show someone yahoo which was a human curated list of sites. The other site was a search engine from one of the colleges I think it was Lycos.
This changed with ads. Suddenly search engines were used to find people searching.
There's a huge difference between typical VC funding and what this is. Kagi is already a self-sustaining business with a working and effective search engine. This is extra funding on top that they have available for improving Kagi. That's great. You don't need 30 million for that.
That $30 million might not have yielded a great search engine but it did produce Cuil Theory, a branch of mathematics with the potential to change the world:
I have no idea how you could justify a $30 million budget. $670k seems much more reasonable to me. I can imagine where this budget goes, the numbers make sense.
Just to clarify how this works. We didn't 'aim' for any number. The investment vehicle we used for this round (Angel List RUV, very good and easy to use product btw) requires to specify a goal amount and we estimated it the best we could based on the number of people who reached out over the time expressing the interest to invest in Kagi (around 260 people). However when the rounded eventually opened most were not able to participate this time because of the accredited investor status requirement for this type of the investment vehicle. We would be happy with any amount and $670K is more than enough for the next phase of the company.
Their revenue in September was $318k/yr, what is it today? Let’s say it’s $600k? $1M? $40M is a really high valuation for that revenue, I’d not have invested either. It’s not 2021 anymore.
When you raise money with an SAFE, you are postponing the negotiation about how much your company is worth. It could easily convert at well under $40M depending on how well they are doing when they next raise.
Finance language for million. Mil Mil aka thousand thousand. Mil is Latin for thousand hence why CPM is cost per thousand, not cost per million impressions.
I'm not Vlad (the person you asked), but I use Lenses almost all the time. They have changed how I search for user generated content completely. I used to add "site:reddit.com" to at least 2/3 of my searches and the Forum Lense does the same for me while including many other discussion boards. I love it.
As an example: I'm playing Diablo 4 at the moment and getting info about the convoluted systems can be a chore. For example: I didn't know that I have to do at least a Tier 3 Nightmare Dungeon to unlock Sigil Crafting. None of the usual pages mentioned that. The Forum Lense saved me a lot of time because users discussing it mentioned it quite a lot.
Kagi has successfully raised $670K in a SAFE note investment round, marking our first external fundraise to date. This was made possible with the participation of 42 accredited investors, most of whom are actual Kagi users.
That is a dense paragraph.
1) What is a "SAFE note"? Google tells me:
A "Simple Agreement for Future Equity" note is a way that startups can raise capital. The SAFE note is a legally binding agreement that allows an investor to buy a specific number of shares for an agreed-upon price at some point in the future, usually when the startup has a subsequent funding round.
2) What are "accredited investors"? Let's assume US.
Net worth over $1 million, excluding primary residence (individually or with spouse or partner)
Or:
Income over $200,000 (individually) or $300,000 (with spouse or partner) in each of the prior two years, and reasonably expects the same for the current year
For context, you're reading this on Hacker News, a forum hosted by YC.
What Google didn't tell you is that SAFEs were invented by YC ten years ago in order to provide better angel/seed investing framework than the then-current bespoke convertible notes.
"Accredited investor" is basically someone who is supposed to be sophisticated enough that they won't first give you (a startup) some money, then later on say they didn't know that your startup was risky and sue you. Accepting investments only from accredited investors is a way to shield yourself from that scenario.
Both terms are used fairly often on HN (and really throughout the startup communities anywhere).
To be clear, I don't like them from the perspective of equity crowdfunding, which has an alarmingly high percentage of low-information, follow-the-crowd, "dumb money" retail investors.
SAFEs muddy the waters. It's just another thing that already overwhelmed retail investors need to take into account when considering whether to invest.
As long as there isn't a secondary market on which to trade, then no, they aren't any worse. And right now, there isn't a good secondary market in the USA.
Ever since they upped the search limit (before it's pay per search), I've been a happy camper.
On the other hand, it did teach me the true cost of searches, and I use a lot more !bangs now, like !mdn or !archwiki (although the search powering both of these are probably funded by donations)
> On the other hand, it did teach me the true cost of searches
It really does change the dynamic of how you search, if you know that the next search will cost you 1.25¢. Right now I use Ecosia, so I'm actually more motivated to do extra searches, to increase the number of trees they plant. Kagi is the other way around, search has a cost, so be mindful about what you search for. It's a great way for them to have a sense of their load, while also perhaps being ever so slightly more sustainable.
You can cut out the middle man for those searches, at least on the desktop versions of Firefox and Chrome. You can add a new search engine in the browser settings and add a keyword. If you've done that you can type that keyword in your address bar, followed by the search term you want.
Another happy paying Kagi user here. Several commenters have mentioned concerns about metering their searches. I'm grandfathered into a $10/mo 1,500 search plan, and there have been a few months where I do indeed hit that limit. The UX is quite good, it makes it clear when you're close, when you hit it, and allows you to set further limits on subsequent searches (you pay $0.015/search after the limit).
I've also gotten quite a bit of utility out of their API, with just a thin wrapper for querying via the command line [1]. I think I'd still prefer using the Search API directly (I currently use the GPT-enabled API), but that's only available for Teams at the moment [2].
> I am glad there's alternatives to Google and Bing.
Kagi is just Bing under the hood with an OpenAI summarizer and a lot of tuning. It is fully reliant on the quality of search results it can fetch from Bing's databases.
I know there exists Qwant (a french company) which claim to have an independent engine, but they keep turning their Bing backend on for a lot of searches. Only truly independent engine from Google/Bing with relevant results that i know of is Brave Search. Not a great company, but they do know how to make a good search engine, and are practically alone in the space fighting against the Bing-based alternative search engine business model.
I'm not sure what percentage of the results are Bing (perhaps the majority), but I think it's inaccurate to say that Kagi is just Bing under the hood:
> Our searching includes anonymized requests to traditional search indexes like Google and Bing and vertical sources like Wikipedia, DeepL, and other APIs. We also have our own non-commercial index (Teclis), news index (TinyGem), and an AI for instant answers.
> I'm not sure what percentage of the results are Bing (perhaps the majority)
Zero right now (we dropped Bing completely after their API price hike). We need to update the documentation to reflect this, thanks for pointing it out.
A bit of a non sequitur, but every time kagi comes up I get a twinge: "kagi.com" had a long, robust life in the earlier days of the internet as an e-commerce facilitator. They handled the credit card transactions for hundreds or thousands of small businesses in the '90s and '00s, including mine, and I somewhat knew the founder, Kee Nethery.
The internet changed over time, of course, and Kagi changed with it. Then something bad happened (I don't remember what, and archive.org is being unhelpful finding the announcement) and Kagi shut down. Then I guess Kee sold the domain, and here we are.
And every time the new kagi.com comes up, I think of Kee.
I don't think I've seen this sentiment shared by anyone else, so I'll go ahead and share it:
I need a way to pay anonymously. For me, this usually means via Monero. Searches are simply too personal and too habitually unrestrained for me to trust to anyone who can de-anonymize them.
Honestly, in retrospect I'll acknowledge that my intense quest for privacy is more ideological than practical, and it wouldn't likely be a big deal if my searches were leaked or whatnot, but still. I've wanted to pay for Kagi and looked for a way to multiple times, but until I can do so anonymously, it ain't happening.
Mullvad also notably accepts Monero, and if/when emulated, we should encourage and celebrate their comprehensive scope as much as any individual method
But then I saw the “K” and grinned. This is proper bootstrap money, and makes me hopeful that Kagi will stay close to its original mission.
Congrats. I am really pulling for you, Vlad.