Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The Future Needs Files (jenson.org)
168 points by polm23 on Sept 3, 2021 | hide | past | favorite | 71 comments


This is a big complaint I have about iOS. One day I get a notification telling me that my cloud drive is almost out of space so I go to clean it out using the finder, and I find like 4 empty folders. An hour later I realize I have a bunch of photos synced, but I they're only viewable/removable through the photos app (somehow this includes images that I downloaded). It feels like such a poorly designed system.


It's as if developers have this internal conversation:

"Hmm, I need a way to store user- and system-provided data persistently to disk. They need to be looked-up by name, and maybe searchable. Some kind of 'system' for storing 'files'. Hmmmm... If only the computer had something like this... Oh, I know! I'll put them in a sqlite database!"

Facepalm


It feels like such a poorly designed system.

Or how about a system designed with intentional obfuscation as a means of selling more storage?


1. Claim users can't be trusted not to screw things up. 2. Prevent users from learning how to properly use their device. 3. Exploit the resulting user ignorance. 4. Profit!


Yes, that sounds like the plan.

Fortunately, I was exposed to this early on when a family member wanted to move their music from iTunes to a non-Apple phone.

You would think storing music would be fairly simple and straight forward --- sorted by artist, album and track or something similar.

Not so with Apple.

What came from iTunes was so arcane and obtuse that I was convinced it could only be intentional obfuscation. And I decided then and there to avoid playing in their sandbox.


I am curious about how far Apple might delve into the iOS-ification of their MacOS line. Mac is a much smaller amount of revenue, probably at a much lower margin than everything else they make money on. I think for least at some leaders at Apple, MacOS is "the thing that runs Xcode to make apps for iOS".


Until Xcode is sucked into Github Codespaces.


You mean Xcode Cloud. :)


How does that work?


Yes, the finder predates cloud syncing and is not designed for managing storage space.

iOS has a built in tool for managing storage space, which is much better.


It might be much better, but it's atrocious! I open up storage and it shows 10+ GB under "other" with zero transparency and no method for cleaning anything out.


Presumably that is system stuff that can’t be deleted without causing problems.

I don’t see what is wrong with reporting that to you.


In my experience usually its cache from installed apps. It would be nice to clear it or see which applications it belongs to easily.


Well, clearing it depends on whether the app is designed to handle that. Since iOS gives app developers the option, we must assume that what is in that area is not designed to be cleared.

I agree it would be nice to see more detail about it, but for almost nobody is thus actually relevant.


> we must assume that what is in that area is not designed to be cleared.

> it would be nice to see more detail about it, but for almost nobody is thus actually relevant.

It is relevant to anyone running out of storage.

And it is always actionable to know which apps, services or system features are using how much.

Also, transparency, as usual, applies constructive pressure on developers inside and outside Apple to be more efficient when users can look over their shoulders at the user resources they are taking.


> Also, transparency, as usual, applies constructive pressure on developers inside and outside Apple to be more efficient when users can look over their shoulders at the user resources they are taking.

You can’t be serious that this is a major concern for either users or developers.


Knowing how much storage an app uses, or how much battery power an app uses isn't relevant to users or developers?

Have you heard of mobile computing? Of course people care.


What tool is that? It sounds useful.


I think they're referring to Settings > General > Storage or something like that


Some counterpoints:

(1) Just "files" are not going to save you -- they must also be in the standard format. It may be pretty simple for text-only notes with no support for formatting, but anything more advanced gets complex fast, especially if you expect files to be editable.

("Editable" is the worst, actually.. Imagine a pen-based note-taking app which stores data as PDF in user-accessible location... users would expect to be able to drop any random PDF and have it show up and be editable. This will make the code immensely more complex compared to simple proprietary format)

(2) The file interface is pretty limited (no easy way to detect new files, can only query by name), so you need indexes and secondary metadata. You might need a non-trivial amount of metadata in app-specific format for advanced functionality, like "get past version" or "find similar objects".

If you are allowing user to mess with your files, this gets much harder -- you now need to be able to re-scan the whole folder, and try to infer renames. This is certainly possible -- many old-school MP3 managers did this -- but it is a significant additional effort.

(3) A lot of times, it is easier to provide alternate mechanisms rather than trying to shoehorn the data model into filesystem (this applies less to mobile, and more to other non-file-based apps), for example:

- The (cloud based) note-taking service I use offer "export" option -- your notes are stored in whatever cloud database, but there is a 2-click way to download entire thing as .zip full of text + opml files.

- There are also APIs -- the same service offers API to download/upload notes. I run a small script from crontab which downloads all the nodes periodically to my computer.

- Local only: "git" does not expose past revision as files -- you use various git commands to see that data.


> If you are allowing user to mess with your files […]

That's the problem. They shouldn't be your files, they should be the user's files.


As a software author, they are "my files" as long as I am responsible for them. Think like a school teacher saying "my class" -- it does not mean the kids belong to teacher, but it does mean that as long as teacher is responsible for them, the kids should be following teacher's instructions.

The same goes with the files -- sure, they belong to user, but as long as you want to process it in the program, you need to follow program's rules. How complex and onerous those rules are depends entirely on the program. For example:

- Text editor like emacs can handle pretty much any file you throw at it -- but OpenOffice will fail with "file damaged" error if you just change 1 byte in the file header

- Many programs require filenames to have certain suffix

- Some programs require files to live in designated "projects" dir

- A lot of programs have some sort of internal database and will fail if user edits it in a non-trivial way.

- Some programs require you to do special manual actions anytime you rename the file using third-party file managers (for example, git needs "git add"/"git rm")

Let's not pretend that allowing arbitrary file renames is some sort of ideological standpoint which separates "user files" from "non-user's files". Every program requires something from user to work, and "do not allow arbitrary filenames" is just one point on this spectrum. OpenSSL does not impinge on user freedom just because it requires a very rigid naming structure in /etc/ssl/cert.


> As a software author, they are "my files" as long as I am responsible for them.

As a software author, you are only responsible for those files insofar as the user allows you to be. The user still owns those files and can do whatever they want with them.

If they have damaged the files your software uses, that's on them, not you; you are not responsible for that, unless it was your software that did it.

Also, it should be pretty simple for a piece of software to tell the user why it can't open a file it needs to open, including a damaged header, a wrong suffix or filename, or a file not being in the correct directory. If the user caused the problem, telling them the problem should make it easy for them to fix it.

In addition to that, if software does not have flexibility in where they put files (such as using environment variables), I would argue that that rigidity does, in fact, impinge on user freedom.


> If the user caused the problem, telling them the problem should make it easy for them to fix it.

Ever shipped consumer software?


What's your point? Consumers still own their files.

Claiming that consumer-oriented software should "own" the files created by the software is the same as claiming that consumers are too stupid for their own good. Personally, I would not want to use software made by a company that believes that.


Apparently not.

I am not against files, actually, I am a big proponent of files. But your statement I quoted is just naive.


No, I have not shipped any, but I've used plenty, and I hate how such software treats users.

Also, I said that it should make it easy, which I understood to mean that there will still be some people for whom it won't help because they simply don't know how or what they did that destroyed their files.


This is a weak analysis. There are large numbers of ‘shoebox’ style applications which pre-date iOS, and they exist on every platform. It’s a mistake to attribute their existence to iOS.

The point of a shoebox app is decouple the user model of organization of data from the file system model even as files are used for storage.

This is done because the filesystem user model is inadequate or inappropriate, or because the storage model is expected to change.

iOS allows apps to expose their data as files within the Files app without requiring them to expose the underlying files themselves.

A good example of this is Quip, which is a collaborative document editor. All of Quip’s data is stored in their online database, but it is also exposed through the files app as a set of PDFs.

Notes could expose a filesystem style view if Apple prioritized it. Of course then we’d have the issue of file format versioning etc. to deal with. If I were them I wouldn’t be rushing to make that trade-off.


> This is done because the filesystem user model is inadequate or inappropriate, or because the storage model is expected to change.

This is a weak analysis. I agree reasonably with your overall point, that this isn't entirely iOS's fault. But I see no evidence or cause to believe any of the alternate anti-file approaches have anything better to offer, present better. You yourself argue that better interfaces are presentable atop the filesystem interface, so it's obviously not a limiting factor in data-representation & data-modelling.

Rather, to me, I see cloudification requiring adaption. Files didn't apply, weren't connected, where-as the digital entities we we began trying to work with were connected, cross-system. That doesn't outmode the filesystem, but it does require adaption & reconsideration. It requires connecting. It requires going further.

I'd point out that we have no replacement for the filesystem. There's no words, no descriptor we can use to describe the alternate, the pervasive, cloudy objects that we interact with everywhere. There's nothing generic, or user-manageable that's replaced the idea of the file.

Files were a convenient reification of the digital to sacrifice, to make computing harder, and because we didn't know how to make the notion of a file more online.


> But I see no evidence or cause to believe any of the alternate anti-file approaches have anything better to offer, present better.

I think it’s fairly clear that the iOS photos app is easier to navigate than a folder with tens of thousands of files. Even the notes app presents the notes in a more useful way than a folder would. Other obvious examples are things like notion or roam research, or any number of systems where the file metaphor just isn’t natural.

Many people today use a web browser to view pages of text and multimedia connected via hyperlinks. This technology hides the underlying implementation which may be files, or may be databases or other kinds of store. It is more successful than the earlier file based protocols such as FTP and Gopher.

> I'd point out that we have no replacement for the filesystem.

It’s not clear what this is supposed to mean.

We have many other kinds of data store, e.g. relational databases, graph database, time series databases etc.


The web uses files extensively. Its not like downloads aren't files. Any page is a file.

I don't think arguing that 'the main ui is not a file browser' is enough to show that a web browser is a shoebox app.


> The web uses files extensively.

Not these days.

Almost all web content is dynamically generated from one kind of indexed data store or another.

> Its not like downloads aren't files.

The fact that you can download some files is similar to the fact that shoebox apps can export files. It may be true but it doesn’t change anything.

> Any page is a file.

This isn’t true. But even if it were, I doubt you’d convince rektide that this would meet his definition of ‘files’.


>> The web uses files extensively.

> Not these days.

> Almost all web content is dynamically generated from one kind of indexed data store or another.

Which are generated from files.

>> Any page is a file.

> This isn’t true. But even if it were, I doubt you’d convince rektide that this would meet his definition of ‘files’.

You are correct here, but every web page is generated from a file or bunch of files.

And there is still static websites.


> Which are generated from files.

No they are not.

>> Any page is a file.

>>> This isn’t true. But even if it were, I doubt you’d convince rektide that this would meet his definition of ‘files’.

>> You are correct here, but every web page is generated from a file or bunch of files.

This is simply untrue. Hacker News for example is made of web pages which are not generated from a file or bunch of files.

> And there is still static websites.

So what? Nobody said that no websites are based on files. Only that most web pages are not.


>> Which are generated from files.

> No they are not.

An indexed data store is still some file or files on a filesystem. Databases are not exempt from creating files unless they are working with raw disk.

> This is simply untrue. Hacker News for example is made of web pages which are not generated from a file or bunch of files.

Where is the information stored for Hacker News? I suspect it's in a database that stores its data in files. Every HN page is generated from data, which is stored in files.

In general, pages are generated from data which is stored in files.

>> And there is still static websites.

> So what? Nobody said that no websites are based on files. Only that most web pages are not.

"Most" is doing a lot of heavy lifting. Is it "most" as in, most visited? Or most pages by volume?


> In general, pages are generated from data which is stored in files.

So what? Nobody is argue that computers don’t store data in files. Only that web pages are not files.

>> And there is still static websites. > So what? Nobody said that no websites are based on files. Only that most web pages are not. "Most" is doing a lot of heavy lifting. Is it "most" as in, most visited? Or most pages by volume?

It doesn’t matter. It’s true of both.


> So what? Nobody is argue that computers don’t store data in files. Only that web pages are not files.

You conceded that all data (modulo a little) is stored in files, so you seem to be arguing for hiding that fact from users. To me, hiding the fact that files exist from users smacks of the same sort of elitism that argues against Right to Repair. Even if a user does not know how to handle their data, they should have the ability to get a third party who can do it for them. Locking down user data is, in my mind, one of the unethical things our industry does.

> It doesn’t matter. It’s true of both.

Honestly, almost all websites either use JavaScript or CSS, which is stored in files, so even if a website isn't static, they almost all still use static files.

Also, I would argue that even a page generated from a single PHP file qualifies as a webpage being a file.


All computers use transistors. Should we expose them to the user?

Should we expose to the user the system libraries used by the OS?

Should we expose the source code of the OS? Because the OS is also made from a bunch of files.


Yes on all accounts. For users that want such.

QEMU exposes the transistors. There are even better OpenFPGA grade simulation tools that we are making to even more detailedly model the transistors. Many chips have very extensive debug interfaces we can use to expose the inner workings of systems. Users should have access.

We should absolutely expose the system libraries. On Android this has been an essential tool for many users to improve their daily existence. It's a huge point of controversy that Google has hired the Magisk developer who did just that, and that now the library is giving up some of it's former skills to help users manipulate system libraries. https://9to5google.com/2021/08/29/magisk-android-changes-goo...

Yes, we should definitely expose the source code. I'd love it if more programs were actively debuggable. Debian for example makes it fairly easy to download debug packages, with all the debugging symbols. The kernel should be accessible. Treating users like end users is not cool & keeps humanity from getting good, from learning & understanding the world. Even if few do take up the option, the option to explore the world about us is something computing ought value.


Did the user create the OS, libraries, or the transistors? By the user's actions, their files were created. They own that data. We should expose it to them.

The other things, they probably did not create. We don't need to expose that.

Nice whataboutism, but there is a difference.

Edit: Also, yes, we should give the user access to the OS and system libraries. They own the computer, and we should allow them to do with it what they wish.


> Databases are not exempt from creating files unless they are working with raw disk.

There's no "unless"; /dev/sda is a file (as is eg /qemu/vm1234/disk/sda).


Good catch. I'll take the L for being too cautious on that one.


>> I'd point out that we have no replacement for the filesystem.

> It’s not clear what this is supposed to mean.

i was super clear. what else have we presented users as a consistent cross system abstraction/means to handle data? nothing. you have nothing.

i dont understand your arguments on ios photos or notes. google photos is one big folder full of files, and some photos can be tagged or organized. this would fit into either a filed symlinking paradigm or a use of extended attributes. i have zero clue ehy you think these apps are anything other than files and folders, but obfuscated.


> i was super clear. what else have we presented users as a consistent cross system abstraction/means to handle data? nothing. you have nothing.

The web.

> i dont understand your arguments on ios photos or notes. google photos is one big folder full of files, and some photos can be tagged or organized. this would fit into either a filed symlinking paradigm or a use of extended attributes. i have zero clue ehy you think these apps are anything other than files and folders, but obfuscated.

I have never used or mentioned Google photos. It sounds like you haven’t looked at iOS photos, which is what I mentioned.

If you had you would never claim that extended attributes or symlinking would replace the experience.

You also simply ignored the other examples, like notion, or roam research.

None of these things are done better by simply having folders full of files.

If files were better, that’s what we’d be using, since we had networked filesystems before we had the web.


> The web.

this is intangible & ungraspable. it compares in no way to files. if you think it does compare to files, explain how.

chiefly i see the web as in no way offering the user-directed malleability that a personal computer has. the web never achieved the vision of writability that Tim Berners-Lee imagined. it should. but it hasn't.

> None of these things are done better by simply having folders full of files.

this is not what files are though. they're also symlinks and extended attributes.

and again i'd argue that most every system is better when it has files under it. tools like Roam are better when there's user-manageable files underneath: the excellent open source Foam (https://foambubble.github.io/foam/) does just that, and is easier to manage for it, while retaining the non-file-centric linked-web-of-documents feel that Roam/Zettlekastan-notes is known for. the alternates you cite aren't alternatives: they're all doable, easily, as files and folders. to not afford the common-man's offering for these applications is a travesty & a crime.

my father greatly prefers managing photos by folders. it's the only system that has made sense to him. he's found every alternative jarring & frustrating. there's a common ground, something tangible & real about the OS having constructs & tools for managing digital things, and my dad gets it, & can use it. not being subject to each application's intangible, abstract digital entities, having "real" digital entities is empowering.

> You also simply ignored the other examples, like notion, or roam research.

still not sure what any of your arguments are. you cite applications but don't provide even an elementary review of how or what characteristics are different, how the experience is different. your parade of other applications is just adding more chaff. i think your analysis is weak.


> this is intangible & ungraspable. it compares in no way to files. if you think it does compare to files, explain how.

The web allows people to accomplish computing tasks without needing to deal with files.

Of course it’s not a 1-1 comparison. That’s the point. Files are not necessary.

> chiefly i see the web as in no way offering the user-directed malleability that a personal computer has. the web never achieved the vision of writability that Tim Berners-Lee imagined. it should. but it hasn't.

I agree, but files don’t solve that problem.

>> None of these things are done better by simply having folders full of files. > this is not what files are though. they're also symlinks and extended attributes.

None of these things are done better by simply having folders full of files with symlinks and extended attributes.

> and again i'd argue that most every system is better when it has files under it. tools like Roam are better when there's user-manageable files underneath: the excellent open source Foam (https://foambubble.github.io/foam/) does just that, and is easier to manage for it, while retaining the non-file-centric linked-web-of-documents feel that Roam/Zettlekastan-notes is known for. the alternates you cite aren't alternatives: they're all doable, easily, as files and folders. to not afford the common-man's offering for these applications is a travesty & a crime.

It’s not the common man’s offering. Almost nobody wants files. That’s why systems that are not file-centric have vastly outcompeted file-centric systems. Files add an additional cognitive burden.

> my father greatly prefers managing photos by folders. it's the only system that has made sense to him. he's found every alternative jarring & frustrating.

He’s obviously in the minority now.

> there's a common ground, something tangible & real about the OS having constructs & tools for managing digital things, and my dad gets it, & can use it. not being subject to each application's intangible, abstract digital entities, having "real" digital entities is empowering.

I agree with this in part, and I don’t personally like the way the web works. However I don’t think files are an adequate solution, and I am not denying the reality that the web and fileless apps do in fact work better for most users. Files are no more ‘real’ than any other digital entity. They are just old and have a lot of tools around them, but they also have limitations.

Also see other people’s comments about the issues of file formats.

> You also simply ignored the other examples, like notion, or roam research.

> still not sure what any of your arguments are. you cite applications but don't provide even an elementary review of how or what characteristics are different, how the experience is different. your parade of other applications is just adding more chaff. i think your analysis is weak.

You seem to understand that there are Apps that don’t use files and present what you call ‘abstract digital entities’.

You seem to understand that these have prevailed because people like them.

You seem to understand that this has happened even though they don’t expose files to the end user.


> You seem to understand that these have prevailed because people like them.

that's not in any way why I would say anti-files have won. cloudification and applicationization, stealing the common material & putting it into something closed/proprietary/untouchable has happened for a lot of reasons (some of that is simply that we haven't yet figured out how to make online objects as personal/accessible as files were; projects like Tim Berners-Lee's Solid or the older remotestroage spec attempt to address this), and I don't think the users liking it has much to do with it.

if there were more examples of online, connected software that had malleability, that had a file-like option, i'd be way more willing to negotiate what users want. right now that feels extremely premature. right now the technical gaps, to get as good as files were, are too high to begin to say.


> some of that is simply that we haven't yet figured out how to make online objects as personal/accessible as files were;

Here you admit that files are inadequate.

> if there were more examples of online, connected software that had malleability, that had a file-like option, i'd be way more willing to negotiate what users want. right now that feels extremely premature. right now the technical gaps, to get as good as files were, are too high to begin to say.

I agree with this, but you are continuing to concede that files simply weren’t good enough to meet users needs, so they chose something else. From the options available, users have not chosen files.

It’s not unreasonable to speculate that users would choose something better than files that nobody has successfully engineered yet.

I would very much like to believe that is true.


There's almost no cross-system paradigms that have survived the cloudification of software. Each app builds it's own intra-verse of ideas, it's own tools. The only thing left that has any meaning is the URL. Which I for one like, but it feels deeply insufficient.

This article is great. Great topic, great coverage, good range, solid points. Thanks.


Not sure how is this related to "cloudification"?

I mean, my Nokia N9110 communicator back in 2000 (which had no associated cloud service) had the same setup for note app -- single file in proprietary format, with all notes stored there. And no mass export tools either!

And before that, in the middle school, we used "At Ease" system [0] which was alternative system interface which exposed no files -- there was one "tab" with objects, but no folders or renames.

[0] http://toastytech.com/guis/atease.html


It’s also probably worth mentioning that the iPhone borrowed some UI conventions from Palm OS. I recall correctly, not only did Palm OS not expose files to the user, but it lacked a file system entirely. Instead it had a database associated with each application to store user data. As the linked article pointed out, this worked very well for the use case of taking quick notes or checking a grocery list but quickly becomes limiting for workflows with multiple programs.


> the iPhone borrowed some UI conventions from Palm OS

This isn’t the case. iOS is closer to “At ease”, which pre-dated palm os by a decade.



Why would you think I was joking. At ease was a simplified interface from the regular finder. It’s primary change was to flatten the file hierarchy into a limited set of panels. It pre-dated both palm os and iOS, and informed the design of both of them.


Because there are files, folders, windows, and the entire suite. This resembles MS Bob (as the linked webpage says) more than it does resemble PalmOS / iOS.

Initial versions of PalmOS / iOS had no files whatsoever, at least visible to the user. If you had say a document editing app on iOS/PalmOS, and you uninstall it, your documents are absolutely gone from the device unless you backed them up somewhere else. The app "owns" the documents. On iOS, this is because apps can only access their private storage space with a few exceptions; on PalmOS this is because there is no filesystem whatsoever so all your apps can do is store stuff on your application's private database, also with a few exceptions (e.g., PalmDOC standard for ebooks). There are no folders, just "categories" or tags at most, there's no way to "browse files" because there aren't, just "apps" and a app launcher which you reach through a home button, and there's no way to share information between apps except through the clipboard.

This "at ease" is clearly showing a shared filesystem where apps can load/save random documents to, and in fact it even shows you can create folders and subfolders... http://toastytech.com/guis/ateasesubfolder.png


Besides the sibling replies, IBM mainframes are known for not having a file system, they use a database and catalogs for storage.

The Newton also used a database for storage.


I agree with the author on the merits of the file abstraction, but I think the concept should be updated for networked devices. We need file formats that support both offline usage and seamless sync over the network.

For example, here I use a merkle DAG-based file format to represent CRDT-like types:

https://www.hyperhyperspace.org

The resulting abstraction can be universally looked up using a hash (or short sequence of words), can be modified offline and synchronized flawlessly. It's still WIP (for example, you still can't export it to an actual file, hehe).


I am interested in the distributed computing/P2P space and study it.

Developers dont want to rewrite their data models to reflect a strange API or weird model. I dont want to pollute my domain objects with MutableReferences or extend HashedObject. So while I think the ideas in Hyperhyperspace are good, the programming model needs revisiting if you want people to actually use it. You need to use plain old objects and spider them for references.

ORMs dont require you to use MutableReferences, they let you use your actual data model as is.

If we can get the backend storage to be P2P then any application outside the backend storage can be written traditionally.

I am personally interested in distributed SQL databases and have written a simple distributed SQL database that could in theory be used as a P2P application backend with distributed joins

I think a combination of event sourcing and CRDTs could be used to provide arbitrary synchronisation and merging between peers that handles bad actors through web of trust.

I have started a discussion of P2P storage backends on Infinity family which is a community of inventors (I can provide invites). I have mentioned Hyperhyperspace there.

https://0oo.li/intent/74001/distributed-data-storage#1630701...


Thank you for your feedback!

There are other projects that take an approach similar to what you're describing (Automerge and its derivatives, mainly).

I guess there is a distinction between distributed databases and permission-less p2p applications. Having a data center where you can trust the compute nodes running your distributed database is very different than having random untrusted peers over the net. But maybe data validation can be done in a less intrusive way, something like what Automerge does for JSON merging (maybe in a typed setting?).

That said, hyper-hyper-space is still work in progress, once we have a proper library of container types programming will be a bit more intuitive. Or so I hope, at least.


I think you're laying some very useful foundations and that I look forward to experimenting with it or ideas based on it. I also look forward to what you decide to do or build and will be monitoring your project.

I see you're going for a zero trust model where you have baked identities into the data structure. Which is very cool.

We have a discussion about this problem on Infinity.

https://0oo.li/method/60001/community-managed-software-secur...

One idea is to strictly review all distributed apps, so sign them by a central party.

If you want to run arbitrary source code which is untrusted you need a interpreter like Ethereum/Bitcoin or browser JavaScript.

I like the approach you can exclude nodes or identities you dont like which is the federated approach mastodon and matrix take. But it doesn't really scale if actors can mass produce identities to spam the network.

I liked what Freenet's FMS did (freenet messaging system) which is web of trust. Your visibility of content is based on a transitive trust setting on an identity. So if you knew a good person and trusted them you could still see good content.

Reddit solves the problem by having a 'new' section which is where any users content goes but it's not the default view. HN does the same. Good content gets manually audited.

I am really excited by P2P software and Braid. I think the problem is running information systems over a P2P network is not a solved problem. The automerge, yjs and Hyperhyperspace projects all provide parts of a data storage layers. Either need to hitch a ride on the browser like Hyperhyperspace with webrtc or something dedicated like a DHT like Kademlia.

The day I can distribute host an information system backend and frontend and others can help host it with me that would be success to me.

So we have problems with distributed storage and distributed compute.

You might get replies on Infinity to your comment..if you join the following Telegram groups you will get notifications.

https://t.me/en0oo and https://t.me/oo0oio

This invite link doubles as an explanation of the site. https://0oo.li/accounts/signup/?invite=0x3fFca3853B7eb67A5a8...

And we are planning to make the site peer to peer at some point when the ontology is concrete.


It’s interesting to me that the focus of this analysis is handhelds. But what about all the web apps out there that use SQL databases to rowify my data?

Can/should one argue that web apps should be backed by highly indexable/searchable file systems so that when I want to depart a platform I can just ask the backend for my files?


SQL is at least a meaningful format, so as long as the holder of your data is required to give you a copy you are still good.


Yes. Active Record ORMs (Rails, Django, etc.) are a plague that teaches devs to pretend data engineering isn't important.

IMO, the only valid use for them is storing configuration that somehow can't be kept in memory after reading config files. Actual "data" (content) should be kept in stable, readable formats.


The terrible culture of app-centric (rather than document-centric) computing goes back to the 1980s. iOS (and as a result android) merely accelerated the trend.


Or the terrible culture of accepting the framing that phones are not only computers but the only computers that matter, since it erases the existence of those of us who use computers.


It seems each new release of Android makes it harder for apps to access boring old files. Seems to be on purpose.


To paraphrase an old saying: how do "files" help me go viral on TikTok?


Seems to me the focus on "Files" is more of an excuse to discuss the broader data architecture of mobile devices, in particular how they store and make accessible private data to ML type applications.

Not sure they are (or ever will be) the core issue preventing this future. Linux desktops are file based, in principle one could envisage intelligent open souce PIM applications that integrate all the personal data securely and privately, but nothing of the sort exists...


On the other hand, file format compatibility is hard to get right and results in ugly hacks and subtle breakage when multiple apps want to extend a standard. Steve Wittens has a good blog post about this:

http://acko.net/blog/on-variance-and-extensibility/


If you’re on an iPhone I recommend iA writer which allows you to save .md or .txt files and sync to your other devices with iCloud or Dropbox

https://apps.apple.com/us/app/ia-writer/id775737172


I never would have believed that "filesystems are useful and good for organizing information" was a controversial statement until last year when I worked for a company where the CEO insisted that "files and folders are ruining people's minds" and started a very serious push to avoid anyone using a local filesystem in any way.

In principle, sure, centralize data management. It's important. I get it. Messy shared filesystems aren't a great way of archiving data. But at the same time, they're a good solution for quickly producing and using data.

The context was around specialized desktop application for technical users (desktop GIS). Yes, you can stream data, and yes, database connections are very much a thing, but that doesn't get around the simple fact that the UIs on everything are set up for local file access. Furthermore, the other applications used alongside this also expect (and in many cases, only accept) local files. 90% of the expensive software you're paying for (or the open solutions too) can't talk to that nice fancy internal-only API you set up to "replace filesystems". In practice, folks are going to wind up with local files anyway, which means you wind up with multiple different copies of the data, which was the whole thing you were trying to avoid.

Also, users are _mostly_ creating new data and not just consuming existing data. The whole point is to have a gui toolbox for processing data and creating new datasets. At a deep level, that's best solved by local files for this use case. (i.e. the formats you need to use for interoperability are meant to be updated efficiently as local files. You can stream them, but not easily update them.) Yes, it's possible to have centralized data stores, but they often wind up being a SMB/NFS/etc share because you need to open the data in multiple applications and local files _are_ the interoperability standards. Embrace shared filesystems where they make sense. They're not the enemy.

Next is hierarchy. Folders are a _great_ system for organizing temporary ad-hoc work. They're also a pretty good system for organizing longer-lived structures. Pushing things into a limited database schema or tagging structure actually makes things harder to find than a "working folder" approach, as it's difficult to group unrelated data together. Sure, you can tag things for grouping, but that generally loses hierarchy. You can reproduce hierarchy with nested tags, but then you're back to representing things as folders. That representation is _useful_ in many cases and isn't a bad thing.

In short, filesystems are good for interoperability on the desktop and desktop workflows are still very necessary for many things. Hierarchical organization is useful, and insisting on dropping it comes with a cost. "Shoebox" / no-file solutions are good for some cases, but not for all. Be _very_ careful about trying to force solutions where they don't fit.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: