Hacker News new | comments | show | ask | jobs | submit login

Isn't bytes.Fields what you were looking for with splitSpaces?



http://golang.org/src/pkg/bytes/bytes.go?s=6894:6924#L282

It uses `unicode.IsSpace` which might be slower than necessary for this use case (after all, how likely is the proc filesystem to use \u2000?). If this were the bottleneck and the program wasn't IO bound anyway, I bet someone could hand-code something clever that skipped multiple space and/or tab characters at a time.


bytes.IndexByte is clever in that way, using SSE instructions: http://golang.org/src/pkg/bytes/asm_amd64.s

I tried using it (https://github.com/bpowers/psm/commit/55bdd3f51c9c61a9247fec...), but it wasn't very helpful, I think because the lines in /proc/$PID/smaps are relatively short.


IndexByte is overkill. I meant something like reading four bytes into a register, comparing the value of the register to 0x20202020 and skipping four bytes.

I see that you're using ReadLine(), this has to read the input and look for "\n". As the person who wrote gnu grep said, avoid splitting the input into lines.

After looking at: http://lxr.free-electrons.com/source/fs/proc/task_mmu.c#L549

I suggest the following: For future proofing, first read the lines of the first mapping, and verify that: 1. The "Pss", "Private_clean" and "Swap" lines come in this same order. 2. The numeric values do not begin earlier than byte 17 in each line (i.e. "KernelPageSize: " is still there).

If these assumptions hold, go to the fast-path code. If not, use your existing code as the safe code path (and output a warning that says that since /proc/*/smaps format changed you program's code needs maintenance for performance but probably not correctness).

In the fast-path, do not split into lines. Instead, use Boyer–Moore to look for "\nPss:", "\nPrivate_Clean:" and "\nSwap:" in the input (after pss, lookup private_clean, after private_clean lookup swap, after swap lookup pss). In each of those, skip to byte 17, and fast-skip spaces. Then read digits until "\n" and perform the next string search.

If you verify that not only did the order of the lines not change but no new lines were added between them, you can hard-code the offsets of "\nPrivate_Clean:" and "\nSwap:" from "\nPss:" and not lookup those. Then you only need to lookup the next "\nPss:" (because the file path is variable length).


excellent suggestion, I will definitely try something like that soon.

There are in fact only 2 variable sized lines - the first VMA info line and the last VmFlags line. In the fast path the middle hunk of map info can be accessed as a single []byte of 392 bytes, with constant offsets for the Pss, Private_* and Swap values.


haha, yes that does seem to do what I wanted. Not sure how I missed it. I will test it, but I imagine that since it uses unicode.IsSpace it will be slightly slower.


I've updated the article to mention this, thanks.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: