I am not talking about a simple app developed using Java, I am talking about making a JVM based framework based on a fork of JRuby and JVM. Is this against some license?
You can fork the JVM, but the patent license will only cover you if you are pass the compatibility kit, which Oracle may withhold whenever they feel like it.
So from a copyright perspective you are fine (that's the point of the GPL) but from a trademark perspective you may not call it Java, and from a patent perspective there's nothing stopping Oracle from attempting to crush you in jurisdictions like the US with bad patent laws.
Hmm, probably not (if your code is GPL), but you really don't want to fork the JVM (unless your name is Cliff Click). It's one of the most complex pieces of software ever written, and it's the most performant managed runtime out there. It's used in anything from tiny electronic devices to huge data crunching applications, from hard real-time systems to distributed batch processing. And, it has to be maintained for a lot of different CPUs and OSs (all work you'll have to do if you fork it).
On the off chance you could make a very specific use case slightly faster on a non-compatible JVM (which, BTW, you'll not be allowed to call Java), it is certainly more trouble than it's worth. Just use it. The entire world does.