Hacker News new | past | comments | ask | show | jobs | submit login

One trick that helped when I was doing log crunching (where time parsing was a good 10%) was to cache the parse.

All log lines began with a date+time like "2015-12-10 14:42:54.432" and there's maybe 100 lines per second. You can therefore just take the first 19 characters, parse that to a millisecond unix time and then separately parse the milliseconds to an int and add that. All you need is one cache entry (since logs are mostly in order) and then you can just do a string comparison (i.e. no hashmap lookup) to check the cache - instantly 100x fewer time parsing calls.

The best way to speed up a function is to not call it!




That's called memoization, and it works really well in quite a number of places.

There are some compilers that can do it automatically with some hints, but I think they are mostly experimental not production.


Well, it's half memoization, and half knowing that you can split the work into smaller parts, which allows you to take advantage of the memoization.

Just adding memoization to date and time parsing gets you very little when there's little duplication of the inputs, and without the breaking apart of the data could very likely have yielded worse performance.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: