I agree that IntelliJ is OK comparing with other Java apps. It is never near performant. Visual Studio being 32 bit app is still much faster than 64 bit IntelliJ, given that IntelliJ can use much more memory. Every time, people complain about IntelliJ, they are told that they will need better hardware. The reason Enterprises chose Java, is because they don't care about performance that much.
Visual Studio being 32 bit is an embarrassment for Microsoft in this day and age. That's exactly the kind of technical hole that JetBrains avoided when they bet on Java to write their IDEs in. The transition for them from 32 bit to 64 bit was extremely smooth despite having a huge plugin ecosystem. I've read about the bizarre hacks people with big Visual Studio projects have to use to ensure they can actually load them, that's just not sustainable.
I haven't compared the speed of a recent VS vs IntelliJ as I use a Mac now (another benefit JB got from going with Java). But of my complaints about IntelliJ none are really related to speed.
BTW if it's typing responsiveness you're thinking of, I recall that JB had some inefficient drawing algorithms which they've optimised in very recent builds (or it might even be an experimental feature still). It could cause issues on slow graphics cards. That isn't a Java issue though, it was just IJ using inefficient graphics code.
I have used latest edition of IntelliJ on OsX. IntelliJ is acceptable but not smooth at all considering that I have 16g ram with 1tb Ssd. And font rendering due to the jvm swing bug is hurting eyes. It is far fetched to say the experience is any pleasant. To fix the issue of Java, jetbrain bundles their custom built jdk
I use IntelliJ on the same setup and don't see any font issues. I haven't noticed any lack of smoothness either, but perhaps we have different standards for that.
Another important performance issue is related to Java's handling about generic type erasure and auto boxing. This is one of the biggest design issue comparing with C#'s handling. So manipulating bytes is a pain. And there is even no unsigned byte. Benchmark does not represent real world use case. I don't feel Java is much faster than Python or Ruby. But there is too much boilerplate code comparing with C#. The pure Java language is clean. But with Spring framework and various Apache common, endless different logging framework, it makes the language such a hassle to deal with. Just my 2 cents.