In the last two months I started deliberately increasing the amount of coding time that is classical algorithm time. Not by working through books and problem sets, but by redefining day-to-day engineering problems that have an easy way out - bash away at it with "industry best practices" and whatever comes to mind, until it doesn't break - into formalized versions that become first data-driven, and then a DSL, so that each "100 lines" solution becomes an effective 1-liner, in the fashion suggested by VPRI's work on new languages. This motivates studying at the algorithmic level and researching PL theory to synthesize that ideal solution.
As a result of this, I am writing very, very few lines of code, in a start-and-stop fashion. I believe it is over 70% algorithmic at this point. The reliance on deep insight is leading me to actively avoid the typical coder's flow state and spend a lot more time on "sit and think" instead. If I hit flow for a long stretch something is wrong.
And, certainly, I would have more "shipping code" at the moment without going this route. But it's still ultimately guided by problems I'm familiar with from doing them the "best practices" way - and each time I make progress, I think to myself "why did I not do it this way before?" (It helps that I have a lot of freedom, working alone.)
Also, it's quite scary doing things this way. The high uncertainty about what the solution is, even knowing the problems well going in, is absolutely terrifying.
I'm embarrassed to show it since I only have parts of a system right now, not a finished system. But as I implied in the OP, the code I'm writing is mostly conventional algorithms and data structures; the twist is in some subtle detail that ripples out towards the top level. I'll give some examples which will hopefully be more instructive than a code dump.
One of my targets was the entity system used to describe a scene in video games. Conventionally this involves an OO hierarchy of some kind, either compositional or inheritance based. The problem is that you don't know up front all of the sorts of properties an entity will need and what algorithms it will work with; the integration of the various subsystems acting upon an entity is a high-maintenance area and tends to involve messy boilerplate data tracking. I pursued the problem further, and came up with a way to move the integration process out of the subsystems code.
This was done by creating a customized set of collections(array, key-value, bag) that associate their data against an entity structure, can handle multiple instances of data associated against the same entity, can efficiently access data of a specific entity, and can remove all of the data related to that entity upon a despawn request. Once these constraints are satisfied, the only thing each subsystem has to do is work with those collections.
Also of interest, and still in progress, is an implementation of J. Paul Morrison's "Flow Based Programming" concepts to control overall program structure. I was looking around at ways to minimize coupling, and this particular method added a degree of structure and contractual behavior.
The first thing that happened after taking on this concept was to work through the ramifications for interactive programs: I had to start with a big data structure containing all the state, and then split it out and merge it together to attain synchronized behaviors. This in turn motivated the entity system idea. It's a very spatial model, so I am now doing some work on a graphical tool.
NoFlo is an example of a more finished FBP implementation than mine: http://noflojs.org/
I have some other ideas too, but they're in a very early stage so there's little to say. Having prior experience in the domain is a huge help, and I wouldn't be taking this approach to conquer arbitrary new problems. I'd try to make at least one mess first ;)
As a result of this, I am writing very, very few lines of code, in a start-and-stop fashion. I believe it is over 70% algorithmic at this point. The reliance on deep insight is leading me to actively avoid the typical coder's flow state and spend a lot more time on "sit and think" instead. If I hit flow for a long stretch something is wrong.
And, certainly, I would have more "shipping code" at the moment without going this route. But it's still ultimately guided by problems I'm familiar with from doing them the "best practices" way - and each time I make progress, I think to myself "why did I not do it this way before?" (It helps that I have a lot of freedom, working alone.)
Also, it's quite scary doing things this way. The high uncertainty about what the solution is, even knowing the problems well going in, is absolutely terrifying.