
Ask HN: What are ways to schedule tasks for the future? (web dev) - ge96
In this particular case it&#x27;s for a tracking system.<p>Namely what I want is a non-performance affecting, for example I may want to gather the ip, time, what a person did (what they clicked on) but instead of doing all of that. You&#x27;d just say &quot;Hey task manager, this person did this&quot; but not do all the computation that goes into your database eg. inline in your server-side scripting or multiple calls from the front end.<p>What technology am I looking for here? I currently use the LAMP stack but am open to other solutions as I have to evolve eventually to keep up.<p>To further elaborate:<p>To do this right now I would have to create several inserts in the database that would happen when the person clicks on something that I&#x27;m tracking. Rather than doing that, I&#x27;d do a short thing like sending a JSON string that has the details I need to insert to a background process that then does this insert outside&#x2F;away from the main part that handles front end use.<p>Thanks for any suggestions.<p>I don&#x27;t mean crontab either, unless that is relevant.
======
tmaly
I built something that does this idea. I wanted to be able to do expensive
computations, but I did not want the person on the website to wait on them. It
is sort of like how the old school craigslist does it not updating a category
till it gets around to batch processing it.

One example I use it for is image processing of an image that is uploaded.
Another is sending an email. And yet another is a ton of database inserts for
metrics.

It works in the same manner as a bank queue processing one thing at a time, so
you can run it on a smaller server and not worry about overloading your
system.

I plan to open source the code in the next week or two if you are interested.
I wrote it in Go, and the job queue part uses beanstalkd

~~~
ge96
Wow, Go. I've been listening to a podcast about that "Go time on Change log" I
hear a lot of great stuff about it "2000 servers to 2" for example haha,
25,000 concurrent requests per second or some crazy thing.

I would have thought this was a common thing. I am interested thanks for
offering however I can't really contribute/possibly even use it as I don't
know how to use Go at this time.

Good luck though with it, I think Go by the sound of it will be even more
popular than it is now. I mean they even talked about hardware-control with
Go. It sounds incredible.

I'll keep it in mind though but yeah outside of my scope at this time, unless
it's not difficult to integrate into what I use right now which is LAMP stack
on a single-core VPS. Pretty pathetic I know but I domain map. Probably not
related but I have a raspberry pi server with Cron (actually working cron) as
well that I look to use for prototyping.

------
andyklock
Sounds like you need a queue of some sort. What I usually see is something
like each web server will dump these details (json is fine) into a file. Then
at some scheduled time these details can be inserted into your database in
batch. (nightly, hourly, etc)

~~~
ge96
Does that suggest Cron? I mean I'm looking for something like a background
running process (CRON) or Daemon haha. But for some reason I'm thinking of
something like Node. I don't know.

~~~
andyklock
It could be cron, but mostly I'd just recommend using whatever you are
comfortable supporting. Cron is great for simple batch jobs (I would imagine
this would fall into that category). Cron isn't ideal when you have to chain a
bunch of jobs together. But, if you have a script that loads files, cron could
do the job easily enough.

~~~
ge96
I actually figured out an alternative for this particular case. I would create
a client-side JavaScript object and store stuff there, then when they leave,
it makes one call to the server with the object passed as JSON. Problem then
becomes cleaning the output. I was just thinking, if you can edit stuff in the
browser console, can't you edit the contents of an array? I guess actually you
wouldn't see that displayed in the code, it's a state maintained in browser
memory I think. But I'm still considering this potential problem and making a
known/limited pattern eg. numbers only separated by a known symbol like a dash
to use for exploding/splitting.... anyway will try it out.

