Maybe it's the case that YMMV. I didn't touch-type QWERTY, so I learned Colemak just fine. On the other hand, my friend who touch-typed QWERTY at 90 WPM also had no problem switching to Colemak. In addition, the official Colemak website states:
"Easy to learn – Allows easy transition from QWERTY. Only 2 keys move between hands. Many common shortcuts (including Ctrl+Z/X/C/V) remain the same. Typing lessons available."