Feynman's strength (well, one of them) was that he insisted on a simple example to follow during an experiment. This is an incredibly powerful technique for understanding complex calculations, and no doubt many hackers here use it when doing a code walk-through.
The problem was that the mathematicians fell into the trap Feynman laid and gave overly simple descriptions. Feynman seduced them into trying to give physical descriptions when it was, in truth, important to think about the details.
Banach-Tarski is a specific example of that, as many of you will know. I think Feynman knew full well what he was doing - just one of the games he played.
The technique has its limitations too. Restricting yourself to consider only the physical loses the power of abstraction. Many of you know that too, although perhaps you don't think of it that way. When you extract a method you're using abstraction - the method may not represent something physical. Feynman's technique can limit you there.
The story comes from "Surely You're Joking, Mr Feynam", in the chapter called "A Different Box of Tools."
Feynman makes a big deal about having unusual tools in the box - they let you solve problems that others find intractable. That's one real lesson. Acquire a wide variety tools, and know how to use them. Functional, Object Oriented, Imperative, Logic, Database, etc. They all have their place, and knowing them, really knowing them, gives you enormous flexibility.