Fuel costs would obviously be greater, but not all that much
Fuel is just a small fraction of the cost anyhow. Also, you could build "cycler" craft using ion or plasma thrusters that could ferry crew modules from LEO to the moon and back.
The convenience of being on a giant rock versus a sitting duck in space can't be understated, either. It makes a lot of things a lot less complicated, especially supplying the base. The deceleration and coupling that ships to the ISS have to go through would be completely negated on the moon - all you have to do is land something relatively close (within a few hundred yards, maybe?) to the base and you're set. It's like threading a needle versus hitting the broad side of a barn.
There's a lot of valuable payloads you wouldn't have to actually land. You could ship frozen water up there in a projectile designed to keep it in a super-cold, super-hard state using liquid nitrogen as a boil-off coolant, and also clad it in armor and partially shield it from impact. Then a rover could go and retrieve the frozen water capsules. Even better, have a few shipments consist of frozen, dampened potting soil, which can be supplemented with lunar soil (made into a slurry with water and sloshed around to wear down the super-jagged edges) and start growing your own food up there with redirected sunlight. (There are a number of plants that are workably tolerant of 2 week sunlight cycles, like peas, provided you keep them cold between sunny periods.)