My opinion is that this is probably what is happening when people claim to have many clear memories from a very young age, though I dare not suggest it since they tend to take offense at having their reality challenged and I don't have data to back up my suspicion. The correct term for this is "confabulation" (http://en.wikipedia.org/wiki/Confabulation) and is related to "false memory" and "source amnesia".
I'm sure most people have a few legitimate clear memories (the single one I have is indeed of a traumatic, though amusing, event) of the 4-6 age range, and of course you have your savants; it's the unusual number of people who claim to vividly recall their youth pre-3 to 4 years of age who I'm skeptical of.
To use technology apologies, the thing is that memory is not a video recording. It is constantly actively maintain reformed in your brain every time you recall it. Some research has shown that a long term memory can be destroyed by inducing minor amnesia at the moment of recall, while many diseases can catastrophically damage brain and memory without harming long term memories. It is very similar in some sense to maintaining hot backups of database data, which can be corrupted by writing bad data over the backups, or messing up during restore.
A 20 year old real memory is not a recording that you have carefully preserved. It is more like a a clarisworks document on a floppy disk that you copied to a wordperfect doc on a hard drive, then to a word doc on a CD, then to Google docs, and each time you copy it you have any to reconstruct some lost details.
Rehearing and retelling the memory is critical. As such, the actual different in constructing a real memory vs false memory diminishes to kill over time.