
Excel Is The World’s Most Used “Database” - jasonlbaptiste
http://jasonlbaptiste.com/startups/microsoft-excel-is-the-worlds-most-used-database/
======
seldo
In defence of Excel, imagine it didn't exist. Then somebody says:

"I've invented a program that allows non-programmers to input loosely-typed
semi-structured data without needing to define a schema. It is automatically
human-readable, and data can be simple code that can act on other data. It
doesn't scale too well, but it can export to CSV and other standard data
formats, allowing you to build real systems with it once the non-technical
people have finished their logical prototyping."

People would be singing its praises and proclaiming the death of SQL for
umpteenth time.

~~~
Daniel_Newby
NoSQL with a GUI!!!1!1!!

~~~
aaronblohowiak
and reactive programming

------
edparcell
Excel is totally free-form, and for small-scale "databases", it's robust
enough (until it isn't). This means the user can do pretty much what they
want, without getting a programmer involved. And that's the killer feature,
that more advanced or more specific solutions miss.

Let's take an example. If the user wants to stick some free-form text in
between the end of sales records for one year and the start of the next, they
are free to do that. In any less free-form application, they need to define a
"comment record" or similar, and they probably need to get a programmer
involved to do that. And although the SaaS web version of their "database" may
have a better interface, in a lot of cases, having to get a tech involved to
make that sort of change is not a compromise people want to make.

And they have a point. It's not a slight against programmers, it's just that
when they need to make that change, you'll be 2 companies and 5 projects down
the line, and it won't be possible. The article mentions a sheet that has been
in use for 15 years - if that had been made as a proper program, at that time,
it likely would have been done as, say, a VB application, with an MDB back-
end, and it probably would have had purple buttons. The source code would now
be lost, and if the business process changed at all, the choices would be a
full re-code, or working around it. I would be surprised if in 15 years time,
we don't look at today's pet technologies in the same way that you just did
when you read VB and MDB.

For me, the direction that Excel, and other spreadsheet, need to take is the
same route that browsers needed to take when IE6 ruled the world. We need
standardisation, and innovation. I've written a couple of blogs on this, and
for me, the way to go is a central repository of extensions
([http://edparcell.posterous.com/how-about-an-app-store-for-
ex...](http://edparcell.posterous.com/how-about-an-app-store-for-excel) for
more on that). In the case of Excel "databases", it might be sensible to
create an extension to standardise the management and creation of such
"databases". It could even allow features like sharing data, backing up etc,
but for that to still happen where users are comfortable: within Excel.

~~~
stcredzero
See my proposal below. What I'm proposing would only require the installation
of a Dropbox-like daemon on workstations and an Excel add-on that could make
REST queries to a repository server. Otherwise, users of Excel just keep on
using it, but their data is backed up, can be rolled back, and data in
spreadsheets is available company-wide and can be integrated.

<http://news.ycombinator.com/item?id=1430661>

------
whyme
This cartoon sums it up best.

[http://www.neopoleon.com/home/blogs/neo/archive/2003/09/29/5...](http://www.neopoleon.com/home/blogs/neo/archive/2003/09/29/5458.aspx)

Seriously though, the problem is that any real solution needs to accommodate
housing data internally that's accessible concurrently with backups. Once you
do that you still need IT involvement, which still makes it 3 months slower
than excel.

\+ Most companies have share-point, and the newer versions allow users to
define spreadsheets with rules that can be edited directly in excel yet still
upload centrally for all users to access with backups.

I think you're trying to find a sweet spot where technology will overcome
organizational bureaucracy. It's a tight spot.

Best of Luck!

~~~
rokhayakebe
Great cartoon. This needs to be submitted as a story. Thanks for the laugh.

------
farez
I have a feeling the UX has a lot to do with it. Excel is just one big grid
and all you need to do is move the cursor around using the arrow keys and key
in data. Done.

All the 'Excel-replacement' solutions I've seen are forms based, so there's a
lot of pointing and and dragging going on. That's so much slower than just
using the keyboard.

Just observe how true Exel ninjas use it... the keyboard calisthenics is mind-
blowing.

Perhaps an Excel replacement will need to have an Excel-like interface, or
find a way to get rid of excessive pointing and clicking.

~~~
patrickk
" _Perhaps an Excel replacement will need to have an Excel-like interface..._
"

Perhaps an Excel replacement will basically _be_ Excel, with a background
script/plugin that scrapes that routinely scrapes the data and feeds it into a
database. (Hey! Startup idea). The first entry in each column becomes the data
field. The program is smart enough to take a sample data from each column, and
use that to auto-pick the best data type. (Also re-checks a random sample
periodically and re-defines data type if necessary). Bonus points for creating
excel templates for consistency ('actions template', 'to-do list template',
'known bugs'....etc. Last one was a joke) and for easier definition of
relationships between different spreadsheets (like for like data comparisons
and relationships as long as the user enters the right info in the right
cell). Now that Office has migrated to the web, this could be entirely web-
based.

People's workflow visually to them remains exactly the same, which would
overcome the biggest obstacle - unfamiliarity and resistance to change. Non-
techies who need to make a list of customers, processes or actions don't know
or think about headaches they are creating for themselves or others down the
line. It's only the technically literate who would appreciate the problems of
consistent, clean data, version control etc. This solution would begin to
answer some of those problems.

------
edw519
Funny, just last month I sat in a meeting with the president of a $1 billion
company to kick off the development of a mission critical app. When I asked
him how to go about collecting requirements, he answered, "You don't have to.
We're already doing this. Just see what everyone is doing in Excel. My people
have to get the job done whether they have support from IT or not."

~~~
jasonlbaptiste
Thats exactly what happened with my senior year project. Just look at the
spreadsheets to see the processes/data they collect. Fairly ridiculous.

------
gaiusparx
I believe it is also the most used project management software.

~~~
jasonlbaptiste
It's also widely used for issue tracking I bet. I almost said bug tracking,
but those tracking bugs are it inclined enough to get away from spreadsheets.

~~~
marcinw
I'd love to say I agree, but I can't count the number of times I've been asked
to deliver a list of bugs/issues/vulnerabilities in an Excel spreadsheet.
--shudders--

------
S_A_P
I am actually amazed at how heavily some fortune 500 companies I have worked
for relied on Excel as both a database, and repository for what is sometimes
the only copy of sensitive financial data...

~~~
stcredzero
How about a combined backup/sync solution for Excel spreadsheets? The sync
portion could work like Dropbox, but the daemon would only pay attention to
Excel files on a given machine, or in a folder, like a user's home directory.
There would be a company account on a website or a locally running web
appliance, where users could share spreadsheets, particular pages in a
spreadsheet, or even particular cells. This would be combined with an Excel
macro that can pull-down such shared data from the account website. Updates to
data would only be sent when the spreadsheet is saved and uploaded by the
local daemon.

This could even be automated by screen-scraping Excel running on virtualized
Windows instances.

~~~
jasonlbaptiste
A automagically simple syncing solution for excel data ala dropbox would be
great and a hell of a start. This might even be a nice app for the dropbox
api.

~~~
stcredzero
It wouldn't only be a great place to start. Once everyone's spreadsheets are
backed up, you automatically have a repository of all of their data, which can
be federated, shared, synced, data-mined, displayed in a web app, etc...

~~~
jasonlbaptiste
Doing this with the new open source dropbox thingy Sparkle would be great. I
like it because it doesn't force people to switch from Excel, but adds another
awesome layer: web access, versioning, syncing,etc.

------
scotty79
I think MS dropped the ball with Excel similar way as it did with IE.

Users of Excel see its great potential and its intuitive interface and use it
for things that builders of this product have not anticipated.

What's wrong here is that builders do not follow users and don't try to
improve their product to better fit what the users are using it for. Excel
should turn into actual database many years ago.

Personally I'd like to see Excel migrate in this direction:
<http://dabbledb.com/demo/#play>

~~~
andybak
I came here to ask if anyone remembered DabbleDB. When I first saw the demo I
thought that thing would change the world.

Isn't that the kind of thing the OP is asking for?

~~~
spinchange
I just headed over to DabbleDB's site to check it out and I see that Twitter,
of all companies, has acquired it (!) They're no longer accepting new sign-
ups. It looked like a totally awesome service. I can't imagine how twitter is
going to use this.

~~~
SingAlong
The same team also run another analytics app called Trendly. And the team will
be joining the analytics team on twitter. So you can see where it's going :)

------
daleharvey
this is exactly the problem we are trying to solve at hypernumbers.com, the
internet has been in programmers hands since its inception, wufoo does a
better job than most at enabling ordinary people to do more and we are looking
to do more of the same.

Ill chuck up a review my startup thread in the next day or so when we get the
website / documentation cleaned up, but would be happy to hear what anyone
thinks.

~~~
megamark16
That's awesome, I'd love to see more when you've got it up.

------
lutorm
I'm pretty sure the world's most used "database" is the notepad and a pen...

------
Tycho
In the last office I worked in (mortgage processing place) they used Excel for
everything and got no help from IT. One time I heard the general manager say
'I don't want to use Access, Joe Bloggs had everything running on Access a few
years ago, and then he left, and then it broke, and nobody knew how to fix
it.' In other words, Excel is a safe option because it's so ingrained and
there's always Excel whizzes around.

Also, I never thought schools, PCs or technical manuals did an adequate job of
explaining _what the difference is between a spreadsheet and a database._
Usually they just say 'database is for large amounts of data' but as we know
businesses have enormous spreadsheets anyway. I'm not even sure how I would
explain it without referring specifically to relational databases and saying
something like 'think of a database as like a 3D spreadsheet...' and mumbling
a lot

------
megamark16
This is the problem I'm trying to solve with AppRabbit.com. DabbleDB is a
sweet online database, and Wufoo has awesome custom forms and reporting, but
for actually creating a database driven web app, well hopefully that's where
AppRabbit comes in. An intuitive model builder, column and record level access
controls based on group membership and customizable filters, relationships
between models in different apps, workflow, etc.

Also, I'm building everything on top of Django with plans for an Export App
feature that will give people portability. Build your app and tweak it with
us, then export and host it on your own servers if you want (although you'll
lose the ability to customize it without getting into the code).

All in all, I think there's room in the market for a lot of different
approaches, since everyone's problem is a little different, and everybody's
data is different.

~~~
Pheter
I'm struggling to understand what AppRabit actually does. It claims to
"give[s] you an easy way to create web based database applications" that is
build on top of Django.

Django already is an easy way to create web based database applications so
what does AppRabit add to this?

------
tjmaxal
We have a very complex SQL database that we use to keep track of patient data
(we are a healthcare company) but to access most of the data for reporting
purposes we still have to use Excel and pivot tables.

I'd love to find a better solution but excel so far is still the
cheapest/easiest to train.

~~~
jasonlbaptiste
What would be the ideal solution? This may be pandoras box considering its the
healthcare industry.

~~~
tjmaxal
I don't know but I'd love to hear any ideas the main problem is it such a
niche market that few people have tried to tackle the problem on a micro
level. Instead most companies come in and say let's change everything and lock
you into our proprietary solution for the next five years.

~~~
stcredzero
I implemented a graphical drill-down on relational data with similar
functionality to pivot tables. You could bring up an aggregate view on a
table, like calls per day, then right-click on a bar in the bar-graph and
choose an AND of a relationship. As an example, let's say it's
"representative." Right-click choose "representative" and all of a sudden,
you're looking at a graph of calls per day per rep.

The cool thing about that program, was that it was based on an OR framework,
so all we had to do was to map it to a particular schema, and it could work
with anything. It was implemented in Smalltalk running in a web plugin. It
would be easy to do as a webapp in any dynamic language with a good OR
framework and chart graphing framework.

~~~
kenjackson
The problem is that Excel already has pivot tables for relational data, and
charting. The charting isn't interactive, so that would be cool. But you're
not going to get people out of Excel just for that one feature. Because there
are 50 other features in Excel that they use daily.

~~~
stcredzero
Maybe I need to build this for Rails & Django, then.

------
nl
I'm pretty surprised no one has mentioned Google Spreadsheets. They are all
over this problem - it _already_ has pretty close to perfect interoperability
with Excel, but also includes really nice web-friendly features (eg, forms for
entry, an Atom interface etc etc)

~~~
jussij
Dose this include VBA interoperability?

From the Excel systems I have seen a lot of the business logic is coded at the
VBA level. So without full VBA support you only have half the a solution.

~~~
dacort
It doesn't support VBA, but it does support Javascript...

------
gruseom
Our take on this: users like spreadsheets. The spreadsheet interface is the
goose that lays the golden eggs; it's what allows so many people to get so
much done. Thus any replacement for Excel has got to preserve the spreadsheet
UI and preserve it in a relatively familiar way. Those criteria narrow down
the class of possible replacements.

We met with a group of spreadsheet support people once at a fairly large
company where, as is typical, IT was trying to force the users off their
spreadsheets. Here is how they said users were actually using the spreadsheet-
replacing-proper-enterprise-apps that they had been mandated to use:

    
    
      1. Open the enterprise app.
      2. Paste the data into Excel.
      3. Do whatever they want.

------
ebiester
The problem isn't that the software to replace Excel and Access isn't out
there. In fact, Alpha Five does a damn good job at it. I started doing market
research for exactly this idea, and saw dozens of people who are trying to
approach this market in slightly different ways.

Unfortunately, I'm away from my notes, but my latest thought was this:

1\. Able to import basic excel data, even if an MVP can't deal with VBA.
Support the basic types of Excel, and be able to interact with the data in a
spreadsheet sort of way. (Much like Access.)

2\. Don't make IT afraid of this app. Imagine the technical support
requirements as people lose data, do something weird. It's a burden to IT. So,
it has to be dead-easy to use (Read: like Excel) with incredible documentation
(video, audio, tutorials, dead-tree), and has to give buy-in to IT that it
will make their lives easier, not harder. Judging by my experience working in
shops with small IT, this is really hard.

Or, it has to be for very-small businesses without a major IT presence. Then,
it has to be cheap, because they already have excel. This looks to be the
biggest barrier! You have to convince the client they have a problem. To me,
this looks like the wrong market, though the one I started to look at first.

It's a sexy market. I'd still love in on it, and the people who finally figure
it out will be a billion dollar business. However, more people than you know
have tried it. It's a deceptively hard problem and I've been thinking about it
off and on for over a year now.

I'm starting to think it should look a little different. Take care of an
enterprise app framework for small businesses that takes care of single sign
on with easy LDAP/AD integration, connections to multiple database,
connections to legacy apps (simple links with option for single sign-on
integration, with options for new windows or frames) and THEN have an
Excel/Wufoo-crossbreed add-on for creating small applications. This would have
IT buy-in, as they could use it themselves for small things, it would have
built in transparent security, and they would be able to delegate to power
users who could define the problem and would be able to work with IT for
solutions.

This would be a big app. I haven't found the MVP to extract, save for the
framework itself. I'm looking for partners on this idea currently.

------
elblanco
And Power Point is the world's most used Vector Art tool I'd wager.

------
jasonlbaptiste
I put it at the bottom of the post and if I decided to do something with it,
I'd do a more focused ask hn post. Here's a video of what my last startup
built, but never released. Ironically I think it solves some of the problems
defined in the post. Would this be useful?
<http://www.youtube.com/watch?v=vNl4cJNdizA>

~~~
megamark16
Wow, that looks a lot (lot lot) like my current project, AppRabbit.com. Even
the layout looks similar, although this is the first time I've seen your demo
or heard about you guys, I swear! I love your approach, and your presentation
and UX looks sweet! It's awesome to see other people dabbling in the same area
I'm working in.

~~~
jasonlbaptiste
Ha we deadpooled in december 2008. This was a demo video from back then, but I
did ressureect the code on my localhost last night for fun. Shoot me an email:
j@jasonlbaptiste.com

~~~
stcredzero
A big barrier to something like your product is the notion of something
replacing Excel spreadsheets. People love their Excel spreadsheets. They are
used because there are a lot of perceived benefits. Such barriers could be
avoided by solving the same problems as an adjunct to spreadsheets. (This
makes sure that there's a backup of all your company's spreadsheets. You can
roll back to any prior version. It also enables easier sharing of data...)

ObjectStudio Smalltalk could do interesting things with Excel spreadsheets
through OLE. You could open a spreadsheet and pull data out of individual
cells, etc... I'd see if there are other technologies in more popular
languages with better licensing terms.

------
dlsspy
I'm pretty sure SQLite's got it beat by far.

~~~
daeken
In much the same way that x86 is not the most used computer architecture, due
to all of the embedded devices out there. Yes, SQLite is probably used by more
people than Excel, but it's not used directly. Excel is used directly by non-
programmers constantly.

Edit: Clarified my first statement.

~~~
pasbesoin
Reminds me of this

[http://blog.gobansaor.com/2009/03/14/sqlite-as-the-mp3-of-
da...](http://blog.gobansaor.com/2009/03/14/sqlite-as-the-mp3-of-data/)

though I never tried it (or similar) -- my days of heavy Excel use are some
years in the past and a few hats away.

------
babyshake
Somebody who works at Kaplan told me the other day that all of their databases
were in Excel. Insanity.

~~~
jasonlbaptiste
The sat prep company,!? How are they getting data in there? What data?

------
joshu
I would love to see this happen. Excel provides so much and yet falls so
short.

~~~
jasonlbaptiste
This, meaning a replacement for using excel as a database?

~~~
joshu
Actually both things:

I'd like to see something that's more databasey as a desktop app. I mean,
hell, just let me rename the columns, or something.

I'd separately like to see a "forms and workflow" webapp that you suggest.

------
mitjak
Well I'm not blind _quite yet_. Command+-.

------
mcdowall
Its my most used Word Processor too

------
andreshb
Very good points. Just for fun I've wanted to create a web app for cap tables.

~~~
adaugelli
This is a great graphical resource of a cap table simulation across three
rounds ...

<http://www.ownyourventure.com/equitySim.html>

~~~
Figs
Reported attack page...

------
paylesworth
This could be an excellent title for an Onion article.

------
shill
DNS is probably the worlds most used database.

~~~
melvin
I find it difficult to use for storing arbitrary information, personally.

------
drivebyacct
I can't even read this without zooming out at least three times.

~~~
jasonlbaptiste
I've been playing around with arc90 styles. Resolution/device? I started
making it so it would read well on the iPad.

~~~
melvin
The text is set to 22px, then that style is overwritten with '100%', meaning
22px. This is huge by desktop standards. 3% of people will be reading your
words with an iPad. A much more common font size would be about 16px.

------
balding_n_tired
Sigh.

