This one of the ideas we have been attempting to realise in Eve (eg http://www.lighttable.com/2014/07/18/imperative-thinking/). We separate the data model (expressed as relations) from the physical implementation of that model (one or more physical data structures per table that implement the queryable interface). The compiler is then responsible for deciding how to evaluate a given query, based on the available data-structures, indexes, hereustics and user hints.
It seems radical in a programming context but it is a pretty standard and well proven separation in the database world. My favourite work along this line is https://infosys.uni-saarland.de/publications/DJ11.pdf which defines the data model to be an append-only list of tuples and treats the creation of all other data-structures, indexes, views etc as part of a single global optimisation problem.
Have you read the "Out of the tarpit" paper (http://shaffner.us/cs/papers/tarpit.pdf)? It deals with using relations in programs as opposed to just in your databases.
At Standard Chartered we regularly used relations as a perfectly cromulent data structure in our programs.
I thought LightTable is an IDE, not rethinking programming :) Anyway, great thinking, I've actually come to a lot of the same conclusions. I'd suggest an inspiration: spreadsheets. Can you implement the separation of data model from indexes/optimizing physical model in an ad-hoc spreadsheet-like UI? When thinking "spreadsheets" you should think about not just Excel, but also Lotus Improv (which is a more structured spreadsheet, which is both a strength and a weakness).
Light Table is an IDE that attempted to simplify programming by improving the tooling on top of existing languages. Unfortunately, too many of the problems we are trying to fix are baked in at the language level and there is only so much an editor can do. Eve is our attempt at rethinking how languages should work.
The UI we are currently working on is very spreadsheet-like. We show tables as grids. Rules are built by dragging columns of data into a working space. We deliberately don't support adhoc layout in the program itself but we will probably add some kind of presentation tool that links various tables into a nicely laid out report.
Sounds like Japan's 5th generation computing project. Heavy efforts were made in the 80s/early 90s in the logic/relational programming direction but ultimately failed.
I think these efforts will ultimately succeed only when they are combined with ML, but there is currently very little cross over between PL and ML (I wonder what Peter Norvig will say about this at his Splash keynote).
What is the missing piece that ML brings to the party? I can certainly see lots of neat opportunities for ML tie in with the system we're building, but I can't think of anything that would end up relying on it.
ML brings much better heuristics to the party. Basically, the dream of declarative programming with hand written generic decision and data scheduling procedures hasn't really panned out. All the arrows point to it continuing not to pan out very well, while ML has recently blasted beyond our wildest dreams.
This appears to roughly describe a precursor to SQL, which appeared 7 years after this document. This was definitely a common problem. Imagine building your most recent application, but not having easy access to an SQL-backed ORM. How would you go about implementing it? The data model would definitely heavily confound your attempts to build a clean system.
> The data model would definitely heavily confound your attempts to build a clean system.
Can you expand on that. I build systems that don't use an ORM but still requires interfacing with a SQL database so I'm curious how one ends up writing "unclean" systems if not using an ORM. Sure, if you've got SQL all over your code-base that's bad but if you were to use the repository pattern it would be as if you were using an ORM in your code-base except within the repository itself.
I actually equated using SQL to using ORM in my comment since I imagined most people would be using one, but imagine if you didn't have SQL and you had to store the data on a file system yourself.
So the solution to this problem is common knowledge by now, isn't it? Separation of concerns, different abstraction layers, MVC (at least the M and C parts), even OOP interfaces. These after all should be known by all CS professionals.
The article seems to be just a piece of history. What am I missing?
It seems radical in a programming context but it is a pretty standard and well proven separation in the database world. My favourite work along this line is https://infosys.uni-saarland.de/publications/DJ11.pdf which defines the data model to be an append-only list of tuples and treats the creation of all other data-structures, indexes, views etc as part of a single global optimisation problem.