Have you traced what system calls xzgrep is making in one shell vs the other? I ask because there are a few odd behaviors in each shell that can slow down some operations. A funny one I ran into was not setting the TZ variable and thus slowing down some system calls. This was also on CentOS which I have not used in a while.
I did try a much larger dataset, and the speed advantage does decrease.
Test files - number and total size:
$ ls *.xz | wc -l
59908
$ du -b *.xz | awk '{s += $1}; END {print s}'
444456636
$ echo $((444456636 /1024 /1024))
423
Dash trial:
# head -1 /usr/bin/xzgrep
#!/bin/dash
# time xzgrep 1234567 *.xz 2> /dev/null
...
real 16m41.275s
user 8m50.386s
sys 11m0.066s
# time xzgrep 1234567 *.xz 2> /dev/null
...
real 16m1.004s
user 8m41.904s
sys 10m45.139s
Bash trial:
# vi /usr/bin/xzgrep
# head -1 /usr/bin/xzgrep
#!/bin/sh
# time xzgrep 1234567 *.xz 2> /dev/null
...
real 21m53.975s
user 7m23.522s
sys 22m10.284s
p.s. The wrapper script fails for filenames that have spaces. It could definitely use some developer attention.
