Why not use something nice if you have the chance? Using your arguments about fundamentals, it's not even necessary to treat him computer science, he could just learn maths and he has all the foundations he needs.
Only if all your fundamentals are restricted to what Java implements. A student that knows only Java may have trouble using multiple inheritance, first-class functions or prototype-based inheritance. I've seen a lot of Java programmers who cannot grasp these ideas and get utterly lost. It's not fun to watch.
