Although I typically hate discussions of "programming languages are like human languages" I think it's the key here. All the team members need to share and be comfortable in not just a spoken language, but the teams local jargon, and the programming language in use (including your team-specific conventions, which form a kind of jargon too). So teams in these esoteric languages (and I say this as a Lisp hacker myself) tend to be by definition small, have a lot of cultural overlap, and can be hyper-effective. And small teams can be hyper productive/effective because of the lack of coordination overhead and because it's easier to have a high 1st derivative from a small starting point (in other words: no large team can beat a small team in efficacy/person, but small teams can't do everything).
That only answers half the problem; the other is: well if small team X can beat small team Y in significant part because of the choice of tools, why don't bigger teams use the same tools?
There's no easy answer. One is power: Lisp and C++ have too many degrees of freedom so it's easier to get in to trouble with languages with fewer "guard rails". This was an explicit design criterion for Go BTW: new grads are coming here and screwing up, so let's make it hard for them to make the most common/dangerous mistakes.
Second in the case of Lisp: it wasn't considered a deployment language as many of the things that made is powerful (e.g. GC, lambda, dispatch, dynamic polymorphism) were considered too expensive at a time when C and smaller machines were taking off. Now many of those features are considered mainstream in other languages, while the Lisp community itself therefore attracted people who were more power users.
C++ had a driver (Windows) and an impetus (like Go's: C gets you into a lot of trouble) but even then people looked for easier / safer languages (e.g. Java, an explicit reaction to ++)