I once had a problem we had thought / talked about for maybe a year, that gradually increased in urgency. We were almost down to the wire on coming up with a solution, we had 3 detailed plans written up, and they all had huge trade-offs, none of which were really acceptable to the majority of use-cases. I once spent an entire day thinking it through non-stop and just couldn't see past what we had already planned. Eventually decided to call it a night.
I woke up at 5am and knew exactly how to solve all of it. Took me 3 weeks to code what I knew the moment I woke up and it worked flawlessly from day 1 of testing. Now if I'm stuck I take off early and rest.
do you have an established routine for embarking on major efforts like that? how much value is there in writing out the plan, and at what resolution? I'm figuring out how to "sketch" large problems as I move towards being a more senior developer.
I have a pretty low bar for just starting a Google Doc, and it's part of my early thought process. It's important to very early on (even if you're not sure you're going to spend time on the problem - even if I just spend 30 minutes doing research on it), put the problem statement, exit criteria / success metrics, non-goals and stake holders on paper to clarify your own thinking about what you're even trying to do.
As different approaches present themselves I just write a quick summary with the pros / cons and gotchas that come to mind. At first I don't put in a ton of time - it's just notes organized like it might be a report one day. I'll probably throw a lot of what I've written away as my thinking on it improves, but I never throw away ideas. If it was plausible enough that I spent some time researching, it's worth recording for everyone else's benefit why you decided to abandon it.
Eventually when the problem comes to the surface more I can say, "I have a report started on this!", quickly revise it and share it with my team and stake holders, and we iterate from there. The design doc mentioned in my original post had actually been around for years, and I had shared it 6 months earlier and had a lot of detail and critical thinking applied to my early doc. It was just a progression from something I opened quickly when I had 1 idea.
You should take notes or records for early research somehow anyway, I just try and do it in a way that's quickly shareable as a design / plan. It might take months (or forever) until it's worth sharing, but it's just a place to record my ideas and new things I learn as the problem's on my mind.
I woke up at 5am and knew exactly how to solve all of it. Took me 3 weeks to code what I knew the moment I woke up and it worked flawlessly from day 1 of testing. Now if I'm stuck I take off early and rest.