Most of the conversation around flat has been removing all the superfluous ornamentation (shadows, reflections, 3D icons) from when UI designers were taking skeuomorphism too far. So if skeutomorphism was about how unnessarily photo-realistic we could make UI, flat design is an effort to simplfily... to see how minimal a UI can be.
Just like any pendulum swing, some UIs way overcorrected and lost a lot of the small UI affordances that helped make interfaces understandable. Aesthetic preferences aside, it IS harder to know what's a button when buttons are styled to have fewer visual differences from normal, unclickable text.
You'll see that the most recent interpretations likes Material or Windows 10th Annivesary seem to actually be bringing back some of the depth, layering, shadows and other visual cues.
The answer, like so many things, is "just enough and not too much".