The openness and mystery of this subject reminds me of how on a few different occasions, E.O. Wilson has said or written [1] that if he were starting out as a scientist now, he would study microbe ecosystems (versus e.g. island biogeography [2]).
Is it so hard to test this? Just give people various types of bacteria, track if they receive antibiotics, and watch outcomes.
It's slightly harder in that you have to give combinations, and not just one, but I think it's doable.
I think it should be routine to give people probiotics after every single antibiotic treatment. That would go a long way toward getting rid of C. difficile.
The majority of microbial species in the world cannot be cultured due to a lack of understanding in the nutritional requirements. This means microbiologists have a very difficult time studying them, one way of getting past this is studying the genetic makeup of a population.
However the best way to understand microbes is to grow them, a lot of bacteria are easy to grow (beef broth) others impossible.
Then there are ethical considerations in 'giving' people bacteria that are not well characterized.
It is a pretty well supported theory that exposure to bacteria is important for the development of the immune system, it is also pretty well accepted that the normal flora of humans is important in preventing infections.
How antibiotics, probiotics etc. affect humans are however much harder to characterize and understand.
[1] http://www.theatlantic.com/unbound/bookauth/ba980318.htm
[2] http://en.wikipedia.org/wiki/Island_biogeography