Could you work on an open-source game engine in your spare time, then use that to pivot across? And/or get a web-dev job in a games company and then try to get an internal transfer?
My wife has suggested the latter a few times and it definitely seems like the most realistic approach. I have a small game engine I've worked on off and on again over the last few years, but it never feels like I have enough time to make real progress
