One of the purposes of X was to run the work on a super computer and display it back to your desktop. Generally it was expected that time on the super computer was too expensive for interactive use, but you might display back a running simulation to see status.
Ah that seems so obvious now that you mention it - a sort of "thin client" model - but I was genuinely confused finding an X11 binary on a company AS400 the other day. Granted, that machine is older than I am. At least that binary makes sense now.
