That said: if you are acclimated to the shock (and possibly have a genetic propensity to survive colder water), it's definitely survivable.
Capsizing a kayak is something you need to know how to deal with to execute properly anyhow, and yes, cold water makes appropriate response and motor control much more challenging.
I've done extended swims (30+ minutes) in ~54F water, and shorter dips in colder (40s, high 30s). That last was more a dash, plunge, and run back to the hot tub, truth be told. Even in the 50s, the cold shock and numbing are quite noticeable, though once you get your head in the right space, it's very tolerable.