Presumably after several hours you could discount the votes altogether since they would have such a little influence on the final result. That would stop the system getting bogged down over time.
Out of interest, what is your "and so on". Do you go to 0.5 for the next 2 hours?
Yes, the so on means go to 0.5 for the next 2 hours.
Of course you have to tweak the parameters to get the desired results for your site. It how all depends on how much you value recent posts and votes, versus overall votes. I also subtract a value depending on how old the post is. The site I use this for doesn't have any users, but it seems to work well in my limited testing.
Presumably after several hours you could discount the votes altogether since they would have such a little influence on the final result. That would stop the system getting bogged down over time.
Out of interest, what is your "and so on". Do you go to 0.5 for the next 2 hours?