

HireFire - The Heroku Worker Manager - meskyanichi
http://jeffkreeftmeijer.com/2011/managing-herokus-workers-with-hirefire/

======
icefox
As you are most certainly already aware you never start a article with "As you
are most certainly already aware" because it is looking down on your reader.

~~~
nborgo
Just as important: if I am already most certainly aware, why are you wasting
my time by telling me?

The same goes for people who write "obviously". If it's obvious, why write it?
If the person doesn't already know, then you sound pretentious or the person
feels like they missed out on something. It's pretty much lose-lose.

On a more related note, I'll probably check this out sometime. The dashboard
looks nice.

~~~
meskyanichi
Good pointers! Definitely will take that with me to my future posts! Also,
thanks for the compliments! Hope it helps!

------
bengl3rt
How fast do Heroku workers come up and start crunching? I'd consider porting
my worker code to Ruby (right now it's in Clojure) if it meant I could bring
it up in seconds.

And they're billed per minute or per second?

~~~
meskyanichi
Heroku Workers are pro-rated to the second, just like Dyno's. So the second
you shut it down, the timer stops ticking.

The time it takes to boot a worker mostly depends on your ruby application.
The heavier it is, the longer it takes to boot up. Heroku spawns them fast, I
think instantly because they are spawned from a compiled slug which is
generated each time you push your app to Heroku.

The HireFire checkup interval takes at most 60 seconds, and less if more
resources are available on HireFire's servers.

~~~
bengl3rt
And presumably if I ran the HireFire open source library I could check even
more frequently..?

~~~
meskyanichi
The open source library checks immediately when job gets enqueued, but this is
because it does a post-create API request to Heroku every time it does it.
Problem is that doing an API on reqests is terribly slow, which is one of the
reasons that led me to create a hosted service out of it.

See the HireFireApp homepage's text for more information, or check out it's
knowledge base.

------
joshuacc
Very cool. Though I don't see myself needing the hosted version for a while, I
can see how it would be useful for people who need tight control over the
scheduling of their job queue.

~~~
meskyanichi
Definitely. The open source solution is fine in some cases. Though because it
does post-create api calls it tends to be slow. Something I had not thought
about prior to developing it, ironically. :)

------
herval
I might just have skipped something o heroku's docs, but is there any similsr
service/app to allow spinning dynos automagically as needed (in traffic
peaks)?

~~~
meskyanichi
If the demand is high enough I may incorporate such functionality in HireFire.
There used to be another service but they have been down for I think a couple
of months.

If you feel this is an important feature, please submit it to the forums. I'll
definitely keep it in mind and see what the possibilities are for also
handling Dyno's!

------
swampthing
Sweet! Any plans to provide this as an add-on in Heroku?

~~~
meskyanichi
Good question! I have actually considered doing so! I haven't taken an in-
depth look at to how their add-on platform works but it's on my short-term to-
do list.

Would be nice to get an even more "plug-and-play" feel. Though, thankfully
setting it up manually isn't hard at all, but it is indeed a nice touch to
create an add-on for it.

~~~
dotBen
Why would Heroku let you be an add-on provider in order to under-cut their
pricing model? Makes no sense.

~~~
hopeless
You've made this remark twice now. The fact is that Heroku have always been
open this sort of scaling (and there's a fork of delayed_job which does
autoscaling of workers on Heroku and... it's _written_ by a Heroku employee.
It's regularly mentioned on the mailing list too.

<https://github.com/pedro/delayed_job/tree/autoscaling>

------
dotBen
What happens when Heroku bans requests coming in from your IP address?

I actually think it would be reasonable and justifiable for them to do it too,
given that you are preventing a revenue aspect of their business model.

Heroku's pricing has always seemed VERY fair and generous to me - it costs
them money to provide the service even at the free tier but rather than
charging by cost lineally they charge based on value. If you need to start
using Workers then you probably are deriving a larger degree of value and so
it makes sense for them to charge for that.

Their model seems fair to me, and is what gives them the ability to offer the
free tier.

I'm all for disrupting big businesses, and yes, Heroku is now part of
SalesForce. But their pricing is what I call "nice guy" pricing and as a
community we should be supporting that, not undermining it.

~~~
meskyanichi
Really? I don't know if you've ever set up a VPS or dedicated server, but you
should compare the cost of that to what Heroku charges for the same resources.
You'll see a significant difference.

Let me put it this way. 1 Heroku Worker (roughly 80-100mb of ram) $36 a month.
Now, what can you get for $36 a month? Go look at 6sync.com linode.com
webbynode.com prgmr.com. You can easily fit 10 workers in any of them. With
this I come to the conclusion you're paying 10 times more than with a quality
VPS provider. Sure, you don't have to monitor these processes, and it scales
up and down easily, but 10 times more expensive is reasonable? Please. :)

Let me tell you what I find reasonable: If you're charging me 10 times more
for the same thing, I find it reasonable I shut down my resources when I don't
need them so I only pay for what I actually use. Why else would you offer the
ability to set the worker quantity via their API?

Or how about this: You park your car, you pay for a parking ticket that lasts
until the next month. Reasonable? I think not.

You should also consider the fact that this could also potentially encourage
more people to actually host on Heroku, rather than be scared off by
ridiculous pricing at early stages. I'm also not trying to "disrupt" their
business. If this would greatly affect their business then I'll be damned, and
I think I'm not the only one. So don't count on it.

