This. I've been in a similar situation working as a data scientist for 2.5 years and went back to software engineering a couple of years ago.
I'm not trying to say that data science and analytics are necessarily bad environments; I just came to realize that I had different expectations for my work than my organization.
While the skills of a software engineer — e.g. quickly comprehending technical references, operating APIs, ability to type more than three lines of code straight — are highly valuable for a data analyst to be productive, I had to realize that my organization did not appreciate the craftsmanship of producing code as much as I still do.
Over time, I had to witness my analyses end up on slides or in Excel workbooks, only to be looked at once. They'd done their job and weren't needed anymore. I was effectively "prompt completing" analytic requests from management to understand the organization/business/whatever better — always with the same result for "my work".
Providing an organization with the intelligence to understand their business is for sure not a bad motivation and can be fulfilling. It's just not a good fit if — instead of the analytics — you consider the software you create for the analytics as your work.
I agree with this. At some point you need to work on a hard project, with minimal support, for an extended period of time. The skills you'll learn will be indispensable. Open source can help, but it's hard to replicate being immersed in a difficult role for 40 hours per week.
The more important thing is that you have a genuine interest in the field, which should take you where you need to go. But without at least a small amount of hard software experience you'll be stuck at an intermediate level.
It depends. Plenty of developers have burned out due to support and maintenance work they don't appreciate (understandably). There's no silver bullet solution but delegating certain tasks to the community (if there's one) is one way to overcome it. Picking the right license also helps.
No other reliable and meaningful way.
Nothing beats actual job in improving a complex skill. There is no shortcut to imitate that.