On: The genius and folly of MongoDB

[byroot]

> What's wrong with the Viaweb/Arc/HackerNews/Mailinator approach of just using in-memory datastructures

https://news.ycombinator.com/x?fnid=cjVXpi8HxVR5TTze3bqSCa

  Unknown or expired link.

Oh I remember now...

[pg → byroot]

That's caused by using closures to create dynamically generated "callbacks" on the server, not keeping data structures in RAM. If you ask for some old item not in memory, it just gets lazily loaded.

[byroot → pg]

Sure you have full permalink support, but why do you have to rely on closure to do pagination ?

My guess: because by relying on in-memory data-structures you can't do what any half assed php forum do, ad hoc queries.

[nostrademons → byroot]

I suspect he doesn't have to rely on closures to do pagination: they're a programming convenience that means you don't have to do things like think about what state persists between pages.

Anything you can do with SQL you can do with in-memory data structures. If you're interested, I'll be happy to take any SQL query and convert it to some Python list comprehensions on arrays of dicts.

[byroot → nostrademons]

> Anything you can do with SQL you can do with in-memory data structures.

Sure, but unless you also do some indexing manually, you can't really query your whole dataset when it start to become too big.

[swah → nostrademons]

Some statements like self-joins become relatively compact in SQL though...

BTW, do you miss Java's more advanced structures (say MultiSet) when programming in Python/Go?

[nostrademons → swah]

Pretty rarely, at least in Python. I don't miss MultiSet, because Python has that (collections.Counter). Ditto LinkedHashMap (collections.OrderedDict). Those are the two "extended" collections that I most often use. I do miss the absence of balanced binary trees occasionally, since sometimes it's useful to have an associative container with a defined iteration order, but sorted(dict) is usually good enough where performance is critical. And Python's heapq module is a bit harder to use than Java's PriorityQueues, but all the functionality is there.

I think I'd miss these a bit more in Go because the built-in datatypes are privileges in some of the language statements, but I haven't written enough Go code to really feel their absence.

[pg → byroot]

Because it's much cleaner and more powerful if the thing that generates the next page is a closure rather than just an index. Among other things it lets you show each user a different set of items (depending on whether they have showdead turned on for example).