I've heard that action precedes motivation - you force yourself to start, and then you become engaged. I've found this is true for me.
Many (not all) professional novelists have a schedule, to ensure they write each day. They force themselves to type X words, or sit there for X hours (they get bored and start writing). But that's work, not leisure.
I think a distinction here is relaxation verses intrinsically meaningful activity. Many people find the latter the most rewarding... but goofing off and aimlessly following our whims, just for fun, has a place too. (Actually this worked well for Feynman).