It's not really the same photon, though, is it? It's just some measure of energy.
Imagine you shoot a puff of air across a room (presumably using an Air-zooka(R)) and score a direct hit on me. If I turn around and say, "I can't believe that puff made it all the way across the room," you could object to my comment by saying "It's not really the same puff of air; most or all of the molecules in the original puff are still over here." But this just obscures the fact that a puff of air, as people use the term, really refers to the wave pattern of cause and effect that travelled across the room. The gas molecules that are inside such a puff at any given moment are incidental and ultimately not what I'm talking about.