Hacker News new | past | comments | ask | show | jobs | submit login

Speaking of filesystems- is there one that allows me to add tags? Or alternate streams (like NTFS does), so I can add my own tagging system on top?



As someone who has spent a lot of time on this problem: you don't want to store tags as attributes/resource forks/streams/etc in a traditional filesystem. The single most important characteristic for a tag-based organization system is that search operations must be extremely fast and cheap. Walking a hierarchical filesystem and stat'ing a bunch of files to discover their tag information is anything but cheap.

What you want is to store the tag metadata heap in an easily indexed structure (i.e: a search tree.) That heap then just contains pointers into content addressable storage. You can kind of build such a thing on top of existing filesystems which support hardlinks, however I'd advise against that for the metadata itself. Once you have the power of tagged content a natural evolution is querying that data using set operators. You will inevitably re-invent (some approximation of) SQL once you arrive at this discovery.


It might still make sense to keep the canonical metadata in xattrs so that tools can interoperate on it and build a search index from it when you actually need to do search.

On a NVMe drive rebuilding the search index shouldn't be too costly, even on hundred thousands of files.

I think there were also some ioctl additions for btrfs that let you sequentially walk the filesystem by extents or inodes instead of by directory tree, which makes indexing a lot easier.


That's a great point. Which rules out using alternate streams as a tagging system.

But not tags themselves- so I'll still like a file-system that supports tags natively.


I think extended attributes is what you're looking for (at least in Linux land).




Applications are open for YC Winter 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: