If you're really only interested in Xfce vs. LXDE, I'd go with Xfce -- but only because it should be less of a memory hog & I doubt you'll miss much from its heavier cousin. Honestly though, either would probably work fine.
Or hey, give OpenBox a crack. :)
(Be warned I'm a keyboard junkie, so my love of OpenBox might be tainted by that :) Between the customizable shortcuts + menu & some of my own stuff [http://tomlee.co/code/ginvoke] I rarely have a reason to reach for the mouse outside of the browser.)
Xfce is nicer than LXDE, and OpenBox on it's own is better than both once you pick the apps you want to go with it.