I'd still consider hunting/gathering a form of work. And who knows, maybe the disposition for individuals to hyperfocus on that form of work led to more genetic success and desirability for reproduction, since it meant there would be more security of resources. I have no idea if these tendencies have a genetic component, totally spitballing here.
