

Getting Meteor to 1.0 - yaliceme
http://www.meteor.com/blog/2013/10/01/geoff-schmidt-at-devshop-8-getting-meteor-to-10

======
sonnym
After seeing the first, extremely slick demo of meteor, I was, as I believe a
lot of people were, extremely excited to see where the project would go. Then
came the funding, and some twelve or sixteen months of development, and I
finally decided to dive in.

I really wanted to like it. I was predisposed to do so.

But from the very beginning, I was confronted with something I consider a deal
breaker. While I am wary and generally disinclined toward the increasingly
popular pattern of curl an installation script and pipe into sh, in this case
it is not only skeptical, it is downright ludicrous. Why would I, nay anyone,
want to install an npm package this way? Should not the installation
instructions simply be `npm install -g meteor`? And what if I need to work on
multiple meteor projects with different versions? Surely there is support for
adding it to your package.json file, but why is this not the primary means of
installation and well documented?

Maybe I am being overly sensitive to these issues, but it really baffled me
that the very foundation upon which a meteor project is predicated would be
contradictory to the typical node workflow. I will probably try meteor to
spike out a project at some point in time, but I do not foresee myself using
it extensively in the near future.

~~~
mr_luc
Thank you.

Saying that Node has a package manager is a huge understatement. Node has one
_hell_ of a package manager, but you have to hack[1] around Meteor to use npm
modules, even in the latest version.

My impression of Meteor, and the impression a number of node people seem to
share, and which I've seen reinforced on HN, is:

Meteor is in an almost adversarial relationship with the node.js open-source
community, because while they're good hackers building something cool, they
took millions of dollars in funding[2] and want to keep their options open for
monetizing that codebase.

This makes every decision they make differently than other open-source real-
time frameworks (to build their own package manager, require a contributor
agreement on an MIT licensed project, and use a nonstandard install
process[3,4]) considerably more worrisome.

And it's frustrating, because there's nothing going on in Meteor that
necessitates going outside of the "node.js ecosystem" \-- aka NPM [4,5,6].

That impression might be unfair in some way, and obviously the meteor people
spend a lot of time responding to just these sort of concerns from the node.js
community.

But "Meteor" means "worry" to me.

\---

[1] [http://meteorhacks.com/complete-npm-integration-for-
meteor.h...](http://meteorhacks.com/complete-npm-integration-for-meteor.html)
[2] [http://venturebeat.com/2012/07/25/meteor-
funding/](http://venturebeat.com/2012/07/25/meteor-funding/) [3]
[https://atmosphere.meteor.com/](https://atmosphere.meteor.com/) [4]
[https://github.com/meteor/meteor/pull/516#issuecomment-12919...](https://github.com/meteor/meteor/pull/516#issuecomment-12919473)
[5] [http://derbyjs.com/](http://derbyjs.com/) [6]
[https://github.com/substack/node-
browserify](https://github.com/substack/node-browserify)

~~~
glasser
I'm not sure what your problem with contributor agreements are; basically
every sizable open source project has one. You know, like... Node:
[http://nodejs.org/cla.html](http://nodejs.org/cla.html)

~~~
curiouslearn
Meteor is awesome. It considerably simplifies web-development. I just want to
let the core team know that many many people really appreciate your efforts
and I sincerely hope that negative comments such as those by Mr. Luc don't
discourage your team and do not distract you from the really awesome work you
are doing.

------
akbar501
Congrats to the MDG team on having 1.0 within reach, and thanks for releasing
an amazing platform.

Meteor is one of those platforms that you should try. I could give you a menu
of features, but the productivity boost from Meteor is one of those things you
really should experience. The productivity boost is very real and its
quantifiable.

An experienced developer can get a very good feel for Meteor over a single
weekend: 1\. Buy the book:
[http://www.discovermeteor.com](http://www.discovermeteor.com) (the book is
$39...I did not purchase the other packages) 2\. Spend a weekend doing nothing
but coding

Come Monday morning you'll know what Meteor is about.

PS: I am not part of MDG, and I have nothing to do with the book (other than
having bought a copy).

~~~
sgdesign
Co-author of Discover Meteor here, glad you liked the book!

~~~
Kequc
Curious to learn about meteor but probably will end up needing an up to date
book when I actually get around to programming with it. So if I purchase the
$40 package, will that include updates up to maybe a year from now?

~~~
sgdesign
Yes, we'll update the book for free at least until Meteor reaches 1.0.

------
mark_l_watson
Earlier this year, I did an experiment: I implemented the design for a
reasonably simple rich web app using two different stacks: Clojure on the
server Clojurescript for the client, and the other being Meteor.

I have lots of Clojure experience but little Javascript experience. Still, the
Meteor implementation took me half as long and had more features. That was a
surprising result for me.

------
sgdesign
If you're curious about Meteor, I recently wrote a short introduction to it
from the point of view of front-end JS developers:
[https://www.discovermeteor.com/2013/09/17/meteor-for-
front-e...](https://www.discovermeteor.com/2013/09/17/meteor-for-front-end-
engineers/)

------
joeblau
I made a meteor app and it was one of the best experiences I've ever had
making a web application. Even though what I made was fairly basic, the ease
of creating a real-time application was amazing.

------
rglover
Thanks for posting this. Great overview of what to look forward to. As someone
writing an app in production based on 0.6.5, should I plan on any big
"rewrites" as the framework approaches 1.0? A better question might be, what
will be backwards compatible and what will I have to scrap? It sounds like
packages are the one area I should be the most careful with...

~~~
akbar501
There will be breaking changes as Meteor moves from 0.6.5.1 to 0.7 to 0.8 up
until it reaches 1.0. This is part of the bargain of working with pre 1.0
software (APIs etc. will be stabilized once 1.0 is reached).

I have found these "breaks" are minor and usually require a few minutes to
fix. That said, I also tend to lag behind by a few weeks as other in the
community work through the details of the breaking changes.

Meteor Hacks ([http://meteorhacks.com](http://meteorhacks.com)) and
EventedMind ([https://www.eventedmind.com](https://www.eventedmind.com)) are
two great resources.

------
barkingcat
I'd love to see freebsd support for the install scripts. Last I looked support
was in its infancy. Maybe I should revisit.

------
driverdan
I've read through the documentation twice (once when it was released and again
a week ago) and reviewed the basic demos. Meteor's front end structure seems
like it's missing something. It has a data layer and a view / template layer
but nothing else. No routing, controllers, etc.

Is there a good demo / tutorial that integrates these missing components,
either from meteorite libs or well known projects like Angular or Backbone?

~~~
jacquesc
I agree, it's missing a good routing system. iron-router is a 3rd party
solution but I didn't have good luck with it. Seems this is functionality that
is core to the framework (it's hard to build anything more than a tech demo
without routing).

~~~
jumpman222
Once you understand iron-router, which takes an hour or two of going through
the documentation and trying examples, it really does everything you could ask
a router to do. I think its worth a shot of trying it.

------
parasj
I've been using Meteor since its release and its been great at making apps at
maximum velocity.

You mentioned that some people are already scaling Meteor across multiple
servers. Looking at it, round robin load balancing won't work given requests
to a server seem to be stateful. How is that being done as of now? Do you guys
have any guidance on how to do that?

~~~
geoffschmidt
Yes, if you're using DDP over sockjs instead of just over websockets (which
Meteor currently does out of the box to support older browsers), you'll need
to use a HTTP proxy with sticky session support.

This will be in the official docs at some point, but for now, Arunoda's blog
post about this is a great starting point:

[http://meteorhacks.com/load-balancing-your-meteor-
app.html](http://meteorhacks.com/load-balancing-your-meteor-app.html)

------
aphelion
I'm very impressed by the technology behind Meteor, I just wish it weren't
quite so "opinionated". It relies on standard parts of the node/js web stack,
it would be great to have a stand-alone library that supported DDP for front-
ends and servers of the developer's choosing.

------
pdfcollect
Any hope of writing/supporting meteor with python backend? :)

~~~
geoffschmidt
Sure! The Meteor client and the Meteor server speak a protocol called DDP,
which runs over websockets or HTTP long polling. Meteor clients can connect to
any DDP server and vice versa. So you just need to implement a DDP server for
Python.

DDP is pretty simple (just a few messages) and the spec for the 'pre1' version
of the protocol is in Git:
[https://github.com/meteor/meteor/blob/devel/packages/livedat...](https://github.com/meteor/meteor/blob/devel/packages/livedata/DDP.md)

This is pretty close to version '1' of the protocol that will be locked down
in Meteor 1.0: [https://trello.com/c/kMkw800Z/53-official-ddp-
specification](https://trello.com/c/kMkw800Z/53-official-ddp-specification)

~~~
1qaz2wsx3edc
I love the DDP as a concept. It's akin to REST-on-steroids, over a websocket
with pubsubhubbub sprinkled on top. I really wish more people knew about it, I
hope drivers are adapted quickly (some have been). Interoperability &
extensibility will make it a killer web feature.

~~~
icebraining
I know I sound like a fundamentalist, but how is anything like REST? It's an
RPC protocol! Nothing against RPC, but it doesn't fit any of the REST
constraints, except for the client-server; it's not stateless, it can't have
middle-man caching and it doesn't really follow an uniform interface. It's
nothing like REST (and as such it doesn't get its benefits).

~~~
geoffschmidt
Speaking as the protocol designer, the eventual intention for DDP is to model
reactive data set publishing in a sufficiently semantic way that it can be
cached by proxies. We didn't get there in pre1 and decided to not let that
block Meteor 1.0, but it is the goal for the future when we have some post-1.0
breathing room (and when the Meteor community needs it.)

The idea behind DDP is that HTTP got us three huge benefits: (1) shared
tooling (I can write a caching proxy, and anyone with a website can use it);
(2) interchangeable parts (your client and your server can be built with
totally different languages and frameworks); and (3) easy APIs (I can describe
my site's REST-y API to you in a page or two). These days, a lot of sites are
moving away from REST/HTTP and toward ad-hoc, custom publishing schemes that
run over websockets or a HTTP long-polling transport that is emulating
websockets. Since they use these ad-hoc protocols to move around data rather
than something like REST, they lose (1), (2), and (3). But all of these ad-hoc
protocols are basically isomorphic to each other. DDP is an attempt to nail
this kernel of data publishing functionality down semantically to the point
that you can get those three benefits back.

As an aside, what's the pressure that's pushing these apps away from REST? The
two big ones are:

\- They cache data locally and need to keep the cache fresh (say they are apps
that run in your browser and never reload the page, or they are native mobile
apps), and they don't want to poll for updates. And/or,

\- They need to do joins and they care about latency. (When you are loading
the news feed, you want to fetch the feed stories, the comments, and the
userpic URLs in one round trip, not first request the stories, and then only
once you have them request the comments.)

Also, many modern apps have verbs that don't map well to REST (a RPC like
transferBalance affects multiple objects and doesn't map well to the idea of
updating the representation of an object identified by a URL), and many modern
apps need to perform some form of latency compensation (predicting the outcome
of a RPC and simulating it locally to update the client's display while
waiting for the server's answer, but then reverting to the authoritative
outcome chosen by the server.)

DDP attempts to address these needs and run over a stream transport like
websockets, while recovering some of the benefits of standardization that made
REST so successful.

~~~
icebraining
Thanks; I can perfectly understand the advantages of having a standard
protocol, and if DDP becomes an effective standard for applications outside of
Meteor, it'll be a great achievement. I just wish people were more aware of
what REST really means.

DDP seems like a good technology _for webapps_ , but the issue is that I find
webapps a really uninspiring trend in web development; my vision for the web
is much more data-oriented than the code silos we're all making. And I believe
the lack of understanding of what REST means is preventing people from seeing
a bigger picture.

 _Also, many modern apps have verbs that don 't map well to REST (a RPC like
transferBalance affects multiple objects and doesn't map well to the idea of
updating the representation of an object identified by a URL)_

I've heard that a lot, and I don't doubt that it's true in many cases, but
usually it seems more representative of a difficulty in the modeling process.

For example, a transferBalance call certainly doesn't map well to updating any
object, but creating a new Transfer (and passing the URLs of both Accounts)
would be perfectly RESTful, in my opinion.

~~~
porker
> but the issue is that I find webapps a really uninspiring trend in web
> development; my vision for the web is much more data-oriented than the code
> silos we're all making.

Can you tell me more about your vision?

~~~
icebraining
I'm sorry, I started a reply, but after two paragraphs of rambling I had to
delete it all.

If you want, I can try to form a more cogent reply later, but in a short
summary, it would be keeping the current trend of sending structured data, but
restoring the principles that made the web great - decoupling code from data,
connecting it across services and communities (turning Hypertext into
Hyperdata) and aggressively adopting standard formats.

I see a web where I don't go to Hacker News; messages come to me from
everywhere, to where I want them to. Where I don't open an issue on Github,
but on my personal tracker, which federates with theirs. Where businesses work
for consumers, responding to their demands, instead of pushing their crap on
us.

This view is mostly based on a mix of the concepts such as the Semantic Web /
Linked Data, VRM, the Intention Economy and other ideas.

~~~
geoffschmidt
I'm totally there with you, and my hope with DDP 1 was to make some progress
on that first step: decoupling code from data, so that a standard DDP client
can connect to any DDP server and pull data out of it.

In the future, I'm super excited about investigating ideas like DDP discovery
(asking a DDP server what data and methods it has) and description (having a
DDP server publish metadata in some standard format that explains the
semantics of the other data that it's publishing in terms of microformats). I
think the main place we differ is that I think that these formats should use
something more like a relational model rather than a hypermedia model, or a
formal knowledge representation model like RDF. Isomorphisms exist between
these models, and used properly they are all equally powerful, so really this
just a design decision about what will be easiest to implement and adopt. And
of course I believe that it has to be server push (websockets) rather than
poll (HTTP), and that it has to support server-side joins (doing 1 query
rather than N+1 queries, and taking 1 roundtrip rather than 2, to retrieve
your newsfeed stories plus the comments on them.) I can't tell from your
comment whether you agree with that or not, but like I said, these are two of
the main things that motivate DDP.

All of that said, I think that by far the hardest part of the puzzle is the
product design and federation issues. In your federated Github and HN, the
technical work is the first 10%, the next 90% is finding a user experience for
it that makes sense to and is ergonomic for a large number of real users, and
the final 90% (really should be more like 9000%) is convincing the industry
(read, creating incentives to force the industry) to cooperate with the
federation and to resolve all of the political squabbles about advertising,
control of the data, licensing, and so on. I mean, Google pulled out of XMPP
federation -- all practical evidence suggests that keeping these federation
together is wicked hard.

So that's why we're just trying to build the first 10% of that first 10%,
which has a very clear use case for building rich browser apps and mobile apps
-- and hoping that we will be a stepstool that others such as yourself can
stand on to build the full solution that everyone dreams of.

Anyway, if you are sympathetic to this line of argument, I hope you'll check
out DDP and see what interesting discovery and self-description features could
be built on top of it, or suggest how its semantics could be strengthened!

------
prottmann
Hope you finish soon. I am excited than for an review of meteor and derby, i
think that this are the two most exciting projects at the moment.

------
MrBra
Metheor got about something like 16M $ in funding, if I remember good, and
they can't afford a proper microphone for the speaker? :)

~~~
MrBra
comeon, downvoters, don't you have a sense of humor? ... man...

~~~
geoffschmidt
Doing AV right is hard! You can devote your whole career to learning how to do
AV well, and I have a ton of respect for the skills of the people that choose
this path. It's a lot more than plugging in a microphone.

We do try to improve AV with every Devshop, and we have recently started to
bring in AV professionals to help. But honestly, pre-1.0, I think it's way
more appropriate for us to be spending our time and money on advancing the
framework than on super slick video presentations. In many ways it's more like
the Meteor community's time and money.

