
Show HN: Auto-indexing key-value store - edward_rolf
https://github.com/kreeben/resin
======
edward_rolf
Hi, this is what I would like the dotnet core community * to start using
instead of looking for the Lucene team to port several years old java code to
core. We already have a search engine on dotnet core. It's called Resin. I'm
only a few minor optimizations away, plus one or two features I'd also like to
include, from having a robust and full-featured Lucene replacement, leaner,
faster, with better compression and with readable code (sometimes even object-
oriented). Time to bury Lucene.Net people. Come help out.

*) Is there such a thing? I mean, you're awfully quiet and compared to the java community, which I know exists, you seldom help out in open-source projects. What's up with that? Haven't found the right project yet? Search, i.e. information retieval and NLP, is about as about as interesting as it gets, at least to me. General AI will be bourne out of IR and natural language processing. Not interested yet? There's a really cool Trie and some pretty sweet tree traversing algorithms that can perhaps be optimized. Have a look at my poor-mans [Levenshtein automaton over in-memory trie]([https://github.com/kreeben/resin/blob/master/src/ResinCore/I...](https://github.com/kreeben/resin/blob/master/src/ResinCore/IO/LcrsTrie.cs#L259)) and the same [over a disk-based trie]([https://github.com/kreeben/resin/blob/master/src/ResinCore/I...](https://github.com/kreeben/resin/blob/master/src/ResinCore/IO/Read/TrieReader.cs#L60)). Sentiment analysis comes later on. It's great fun!

~~~
flukus
> Is there such a thing? I mean, you're awfully quiet and compared to the java
> community, which I know exists, you seldom help out in open-source projects.

Of course there is such a thing, although Microsoft did it's best to kill it
about a decade ago, it's never recovered from that. As for this project, I
suspect the contributors to be small, it's simply the sort of thing that's
outside the skill set of most .net developers.

~~~
edward_rolf
>Microsoft did it's best to kill it about a decade ago

A decade ago I started learning C#. I did not notice this event. What was it
in particular that they did?

>outside the skill set of most .net developers

But javaland has many of those devs? I'm really wondering if anyone outside of
the Lucene team and Google even has a clue about how to build a modern search
engine.

If it is true that .net devs are unskilled then this project is the perfect
starting point for all those who wish to transcend from being web and business
application developers making tons of money for their employers, to ML devs,
thus (hopefully) securing for a while their hireability, since businesses in
fact have realized they need to become smarter than their compertitors and
that they need ML skills (and big data capabilities).

It's the perfect starting point because in this project, I've taken all ugly,
wierd and awkwardness out of IR, because there is no place for such things in
IR, because IR is not ugly, wierd or awkward, it's very, very straightforward.
The code needed to express an IR system needn't be complex, at all. Half the
fun of doing this project is showing everybody how easy the domain of
information retieval is.

~~~
flukus
> A decade ago I started learning C#. I did not notice this event. What was it
> in particular that they did?

There was a fairly vibrant OSS community around .net at the time filling many
of the gaps. ORM (nHibernate), MVC (monorail), build (nant), IOC (Autofac and
many others), unit testing (nUnit and others).

MS decided to go all NIH and create their own, almost always worse version of
all of these tools rather than work with the community. The message they
effectively sent was "don't do OSS on our platform, we'll crush you're
successful project with an inferior clone".

>But javaland has many of those devs? I'm really wondering if anyone outside
of the Lucene team and Google even has a clue about how to build a modern
search engine.

I think you're right, there probably aren't that many with the know how. The
even bigger problem is there aren't many with the awareness that tools like
lucene exist, it's database queries or nothing to most developers.

~~~
edward_rolf
> it's database queries or nothing to most developers

Perhaps for some but not for most, I think.

I believe dotnet devs suffer from a lack of confidence and from identifying
themselves as dotnet devs, instead of just devs.

I want to be on dotnet core because I want to be cross platform and code in
C#. My options, if I cannot get a community around this piece of code, is to
go to golang or java, because I really want to be cross-platform and code in a
decent high-level lang.

But it seems silly that I should move from the preferred platform to one where
I am less productive because Microsoft were being the Microsoft they were back
then, when they put so much effort today into copying the success of java.
Prior to an open-sourced dotnet core, that vision was unreachable. Now it is
reachable, for real.

I don't trust Oracle or Google any more than I trust Microsoft. I trust they
will be what they are.

I do think dotnet core, technically, outshines java, by miles and miles.
Haven't tried go yet.

I need my tech to be built on the best tech in the world. Because I'm out to
beat those who are not.

We'll see how much traction this project gets. Maybe none. I'll beat the
Lucene team single-handed if I have to. That's not my vision, it's just the
first step towards creating what I'm actually here to create: tech, open for
everyone, powerful enough to beat Google with, if you just add the right
business strategy to the mix.

This is not a Lucene-killer. It's a Google-killer.

Yeah, I'm megalomanic sometimes. What a dull world to live in though, where
Google is unbeatable ;)

~~~
flukus
Good luck, I'd suggest focusing on a great API/developer experience, that's by
far the most painful part of lucene.

I don't think the megolomania/arrogance is a bad thing, most projects start
with an "I can do it better" approach.

~~~
edward_rolf
Thank you, I agree and those suggestions are valuable to me. If you have more,
let me know.

