Recently, I started creating a throwaway physics/game engine for fun and it is an awesome way to learn some really cool algorithms. Also, I did a small image manipulation app that exposed me to some cool stuff as well.
As a new grad it definitely pushed me more towards game development- would you mind sharing your reasons for leaving?
It's also tough for some engineers because there's usually a very clear product vs engineering divide -- you're implementing stuff other people come up with, and when they change their minds you throw it out and do it again a different way.
That said, making games is also a ton of fun and is a great way to be exposed to a lot of interesting engineering challenges. If you like games and you can put up with the issues above, you should definitely consider it.