Instead join any related graphics domains: scientific visualisation, graphics drivers, jobs that involve writing game frameworks  for example. These jobs will give you both the fun of working in a hard domain combined with the stability that does not come with the intrinsic fickle nature of game industry. FWIW, I have followed this path of learning to write games on the side but not joining any game industry.
My 2$, I would say don't go to the extreme decision of leaving tech (that is also not a bad idea if you work it out) but before exploring all the other interesting options in tech and giving that a try, may make you think 'but if' in the future. Best!
Another idea is to join a small team, so that you have to become more of a jack-of-all-trades.
“A jack of all trades is a master of none, but oftentimes better than a master of one.”