On: When setting an environment variable gives you a 4...

[kilianc]

Good point! But sometimes a large-scale parallel filesystem can't beat a local SSD on certain patterns.

Also, we didn't see any timings for that 1,000x claim, so let's say it's an estimate. :)

[emmelaich → kilianc]

It's a great article, thanks.

But .. do people actually use `ls` to list so many files? I mean, it'd scroll off the terminal anyway.

[CaliforniaKarl → emmelaich]

So, I need to call out one thing. For the people mentioning `ls | grep` and the like, in auto mode `ls` disables colors when it sees it is not running in a terminal. So using it in a shell script (or similar construction) doesn’t have this issue. Otherwise your loop variable would be full of weird ANSI escape color sequences!

[CaliforniaKarl → emmelaich]

Yup, some people do. Maybe they don’t expect so many files. Maybe they are gonna scroll around. But they do.

[chris_wot → emmelaich]

I admit to being guilty of the odd

  for i in $(ls)
  do
    Something dumb
  done

[xorcist → chris_wot]

It clearly depends on what good that subshell running ls does for you, but the trivial form would be:

  for i in *txt
  do
    ...
  done

The reason to stuff every file name through ls (keep in mind that it's always your shell doing glob expansions, ls is not involved) would be to sort them by creation date or some other processing that ls can do. But as soon as there are many files involved, or those sorting arguments gets non-trivial, it will fall apart completely.

Use find | while read instead, or xargs. That's going to be easier on the eyes and actually work.

[wahern → chris_wot]

  for i in *; do
    ...;
  done

works even better as it can handle files with whitespace. I usually disable file globbing but enable it specifically for code like this.

EDIT: Actually, one issue with the above is that if there aren't any matching files then you get a literal "*". But I'd rather deal with that then break when filenames have embedded whitespace.

[zaphirplane → chris_wot]

Out it interest why not xarg or find’s exec

[freedomben → zaphirplane]

Not GP, but I often find myself writing making statements (which is awkward with find), or using syntax that find struggles with.

[myrandomcomment → chris_wot]

It is like you read my script.sh over my shoulder as I was writing it. :)

[mirimir → emmelaich]

I sometimes ls | grep.