They scale perfectly fine, as long as you filter locally before aggregating. Lo ...

They scale perfectly fine, as long as you filter locally before aggregating. Lo and behold:

  mkdir -p /tmp/ssh_output
  while read ssh_host; do
      ssh "$ssh_host" grep 'keyword' /var/my/app.log > "/tmp/ssh_output/${ssh_host}.log" &
  done < ssh_hosts.txt
  wait
  cat /tmp/ssh_output/*.log
  rm -rf /tmp/ssh_output

Tweak as needed. Truncation of results and real-time tailing are left as an exercise to the reader.