Great post, I used to work in manual labor. There are issues with mental fatigue in software development that affect work/life balance more so than physical fatigue, but in general I agree with you.
I had 2-3 summers jobs in manual labor and couldn't believe the amount of physical effort (and long hours) that went into it.
I think even more significant is the number of guys I've known in their 50's and 60's who have to walk with a cane, or have bad backs, and other injuries that have severely impacted their lives.