Hacker News new | past | comments | ask | show | jobs | submit login
Should data structure be part of data?
1 point by naturalcomputer 15 days ago | hide | past | favorite | 2 comments
The word "data" implies portable information. We expect to be able to port data from a piece of paper to a spreadsheet or a text file, or from a persistence platform into memory. But what constitutes data - is the structure behind a piece of data part of it or separate from it ? If the former then shouldn't the structure be portable as well as the data ?

Humans are in the grip of self-inflicted systematic behaviour modification based on data; a social catastrophe. Jaron Lanier's clarion call has been coherent, articulate and prescient. And yet surprisingly people of all experience and intellects seem sceptical and unreceptive, even allowing for the resistance-to-change which he predicted. I live in the UK, have deleted various social media accounts, and have been a catalyst in passing the message on successfully to some others but the adoption rate is low. Might people's attitude towards change be different if it was more in pursuit of a positive hope and less in flight from a negative fear ?

The mediators or "MID"s proposed by Lanier and Weyl sound reasonable and natural in the future, but will surely need to be preceded by the adoption of some new simple, stable and transparent platform which has data dignity baked in and is intelligible and useable profitably by individuals rather than mediators - perhaps something like Sir Tim Berners-Lee's Solid. MIDs are likened for illustration to familiar organisations such as corporations, unions and guilds. My point is that such organisations were all built around the pre-existing commercial activity of individuals or small groups.

For individuals to own their own data requires that they should at least be aware of it; and preferably understand it too. It is reasonable to expect that individuals could understand their data at a high level, but understanding structure is more technically challenging and depends largely on the persistence platform. Persisted data is typically considered to be either loosely structured as in a data lake or a nosql database, or tightly structured subject to some data schema. The latter is easier to understand for inexperienced individuals but the trend of platforms over recent decades has been more towards the former, to avoid making the persistence rigid and brittle. The average data owner has little chance of interacting directly with persisted data structure unless they learn the details of, and become locked into, a specific persistence platform. In short the structure of persisted data is generally not as portable as the data itself.

A universal standard which allowed data structure to be represented alongside data in a simple extensible markup might solve some of these problems, with benefits including ; a) allowing data structure to ported alongside the data it represents b) making a single common data structure accessible to different persistence platforms for import/export c) helping individuals to understand, shape and commercially leverage their data




Think of data like a Lego set; the pieces are the data and the instructions are the structure. Right now, people can take the Lego bricks anywhere, but not the instructions. Making both portable would let everyone build their cool ideas more easily, and maybe help people feel more positive about using their data creatively, rather than running from digital traps.


Thanks for the reply. In between data-values and code-instructions lie data-schemas. We could arrange for data-values and data-schemas to be portable / taken anywhere, but the code-instructions which manipulate them will likely remain tied to specific execution platforms. Building on the pair of elements in your Lego analogy {1:bricks=data-values, 2:structures=code-instructions} perhaps we could add a third element : {3:snap-connect-patterns=data-schemas}. Anyway I like the sound of making it easier for people to build their cool ideas Lego style - a neat analogy which everyone can relate to and many will feel positive about - the toy-loving child buried in all of us tends to be geekier than the adult :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: