I would argue that if one needs to know where to look and how to manage them, then CL _is_ lacking in terms of libraries. One first has to know where to look and find a CL implementation then to know where to look and find batteries for it.
You don't complain about not knowing where to look to get a Linux distribution do you? Freedom leads to fragmentation, but the fact that there are a million Linux distributions doesn't make Linux any less powerful.
If you want an Ubuntu, don't use Lisp. More power to you. Some people just want to get the job done quickly with at little thinking as possible. I know I feel that way sometimes. But if you want an Arch, where you built the system from the ground up, then you want Lisp. And at the end of the day when something breaks, you just might know how to fix it. ;)