Luckily that type of work has been pretty much replaced by robots today.
Assembly line work is exhausting - very exhausting. However you can continue to do it while your brain is shot. This contrasts with programming where you simply usefully can't work on many problems unless you're in good enough shape to keep the necessary state in your head.
And an incidental note about your dad. Driving involves a lot of complex visual recognition and situational awareness of a constantly changing environment. We do not think of this as a complex cognitive task because we're wired to do it without even being aware of doing so. But when you look on a brain scan, huge parts of your brain light up and are working. This is real mental effort being made, and it is no surprise that you get tired after constantly doing it for long enough.
To give you an example. Try writing code in Java these days. With Eclipse very soon(And trust me you will feel this in days) you will realize that you are hardly doing anything that is intellectually heavy lifting. Unless of course you are doing something that is totally new to your area of work. In that case its totally different. Which is the same case with assembly workers too(A totally new product unknown to you will be difficult to assemble).
But imagine doing applications which have nothing more than a DB interaction layer and some business logic. You can do this sort of a job for first 6 months of your programming career, and then you can have your muscle memory built to work with nearly auto completion method out there. Soon you will see that you will be doing nothing other than assembling already well known blocks of code without putting any sort of hard intellectual effort.
Rinse-Repeat this with a few domains/programming languages/architectures. Very soon you will find that you are doing nothing more assembling reusable blocks of logical pieces.
The only difference between hard labor and something like programming is the physical tiredness. This I completely agree with you that physical labor is more demanding to your body.
But other wise there is no reason for us to believe that the nature of our work is any different than anybody else.