Good, well developed internal tools which speed up company work in a way that something off the shelf cannot, can be a good thing. Even the best internal tools, however, will require some ramp-up time for someone joining the company to understand and use them to their full potential. They require developing in the first place. The gains must be able to justify the costs, which even a good tool cannot always do due to the limited consumer base of an internal tool.
In less ideal circumstances, they can be unholy buggy unsustainable messes that actively sap resources for maintenance and are worse in many if not all respects than cheap off-the-shelf alternatives, even for the company's own specific needs. They may be kept alive by the difficulty of transitioning, other internal tools depending upon it, or misplaced attachments to the system ("we've invested too much into it to switch now!" etc.) or misguided approach to tools in general ("licenses are expensive, let's just build our own.")
But a large company probably has at least one unessential tool meeting a need better filled by something off the shelf, just by a matter of statistics. There can be all sorts of legitimate reasons this happened other than NIH Syndrome (it predated the off-the-shelf solution, it couldn't be found despite research, regulatory requirements, etc.) but NIH Syndrome is one to watch out for.
Two of the last three companies I've worked at had their own internal version control systems. I'm sick of having to learn how to check code in all over again every time I start a new job.
On the other hand, if you ever find a company that wrote their own VCS 3 years ago that has half the features of Git, then run far and run fast!
I think it's usually fairly easy to tell which person is which. (Person is either a bullshitter you can quickly catch out, or is smart and inspiring to talk to).
But he does have a point in that the majority of rewrites are not significant improvements; you have to look more closely to evaluate this.