On: Show HN: Fist – Persistent Full Text Search and Indexing Server Written in C

[taneq]

I'm not overly familiar with search as a field - why is stemming "critical to FTS" rather than a nice enhancement? I would have thought the defining criterion for "full text search" was that it searched the full text of the database?

[ComodoHacker → taneq]

The key is "with support for many languages". For many languages (other than English) with heavy use of suffixes and/or prefixes, FTS without stemming is kind of useless.

[microtonal → ComodoHacker]

For many languages, it is even useless with stemming, you would want lemmatization plus a certain amount of fuzzy matching.

[ianai → ComodoHacker]

I didn’t quite understand what you meant. I saw your comma and thought you were developing another sentence and thought entirely. I understood you meant FTS search needs stemming to be useful after skipping he comma. Just FYI.

[ComodoHacker → ianai]

Yes, you understood correctly. English is not my native, sorry.

[ianai → ComodoHacker]

I used to be a heavy comma user. Still am. But try being ‘radical’ and not using a comma more often. (It’s not actually radical at all)

[itronitron → taneq]

you don't need stemming for full text search, although it is a benefit for some text data.