

What database structure powers News.YCombinator efficiency? - CalmQuiet

As database web-dev newbie (or at least elementary level), I'm trying to get up to speed on optimization issues for my growing rdbms.<p>I learn a lot from studying examples, and I'm really impressed with the speed of HN's performance / data freshness.  Has there been a HN post that describes the HN-news database structure behind its burgeoning service? [ I tried some search within site and via searchyc, but general terms like "database structure" yield way too much, and I've not known what terms would helpfully narrow search. ] Thanks!
======
davidw
This has some information about it:

<http://en.wikipedia.org/wiki/DRAM>

~~~
CalmQuiet
Well, that's a nice intro to DRAM, but I'm unclear as to its relevance to
database/file structure [whether on hard disk or RAM].

~~~
davidw
You could always get the source code... But the speed mostly comes from being
in memory (or at least that's my guess). That pretty much trumps any
structure:

<http://news.ycombinator.com/item?id=468611>

Also, it's fairly slow for some operations:

<http://news.ycombinator.com/submitted?id=davidw>

Seems to be quite slow, for instance.

~~~
CalmQuiet
Hmmm... I hadn't realized _submissions_ retrieval would be so slow, since my
17 submissions show almost instantaneously; your hundreds _do_ take a while.

Something about the indexing for submissions doesn't work that fast, even
though all in memory?

~~~
davidw
Maybe, since the submissions stretch back over a longer time (I submit less
than I comment), they're hitting the disk, and consequently slow?

~~~
CalmQuiet
That seems likely. But I thought I saw a thread where pg seemed to say it was
_all_ in RAM ?

[Though in a fairly recent post, I seem to recall he was musing about
upgrading from a 32 to a 64 system - and working with more than 4GB RAM. Can
all HN comments & indexes be happening in 4GB RAM? ]

------
allenbrunson
pg prefers to avoid databases where possible. viaweb stored all its
information in flat files, and so does hacker news.

~~~
eatenbyagrue
Hmm, that seems a little... anachronistic. From my perspective, I figure the
many thousands of programming hours that have gone into making your average
RDBMS fast and reliable might be worth taking advantage of...

~~~
lacker
There have also been many thousands of programming hours invested in making
your average filesystem fast and reliable.

It all depends on what you want to do. If you want relational queries a RDBMS
provides a huge advantage. If you only want key-value lookup an RDBMS might be
overkill.

