

Meteor Devshop 2: How Spark Works, Meteor 101, 26Plays, and other wisdom - qiqing
http://meteor.com/blog/2013/04/10/meteor-devshop-2-meteor-101-how-spark-works-other-wisdom

======
btipling
This was really fun to be a part of. Meteor.js is pushing a new way of
building real time web applications. They're on fire and pulling me back to
JavaScript even as I want to experiment with other languages like Go and Rust.
They really have a brilliant team, and a growing mind share of really smart
people like @eventedmind (who gave a talk about Spark at this) who are
focusing on learning and building with meteor.

------
bjourne
Correct me if I'm wrong, but dealing with "big data" is still an unresolved
problem in Meteor, isn't it?

The leaderboard example (<http://meteor.com/examples/leaderboard>) works
because it is easy to synchronize with a datastore that contains at most a few
hundred values. It's so tiny the whole dataset can be kept in the web browsers
memory.

Say you have a database with 100k searchable indexed documents. You run the
query "foo" to fetch the 10 best matches and also a count of how many
documents among the total number of documents searched that matched the query.
Now if someone simultaneously adds another document, the whole search query
must be run again because the new document might match "foo" and even if it
doesn't, the total document count still needs to be updated. Multiply that by
a few concurrent clients and performance goes down the drain.

The above scenario or permutations of it is incredibly common in web
applications and I don't think Meteor has a good solution to it. I'd gladly be
proven wrong though.

~~~
btipling
You don't need to run queries, you subscribe to queries. As the data updates
your app will be able to react to it without having to run a query. For really
"big data" you are probably going to want to supplement your application's
architecture with something specifically designed to work with "big data" just
as you would have to with any web framework. The new smart package feature in
meteor.js was intended for adding new things like this into your app.

~~~
bjourne
The difference is academic. There still has to be a database that runs queries
even if Meteor prefer to think of it as a subscription. The reason I used "big
data" in quotes is because 100k rows isn't really that big. Most other web
frameworks has no problem dealing with it, but with Meteor it's almost
impossible.

