Hacker News new | past | comments | ask | show | jobs | submit login
Discovery Coding (jimmyhmiller.github.io)
87 points by surprisetalk 4 hours ago | hide | past | favorite | 28 comments





Never thought of it this way, but it makes sense. My default response to probing/planning type questions from business is "uhhh no clue, I have to dive into the code first and find out" precisely because of this.

Interesting piece. As a coder and writer (hardcore outliner here), I’ve thought about this too.

I wonder about writing user stories for fiction. If software is something people use to realize an outcome, fiction is also something readers consume to realize an outcome. What might a “reader story” look like for some of our favorite novels? What kind of impression or change does a writer seek to produce in a reader’s mind? Such documentation could be valuable, similar to requirements documentation.

Aside to the author: King’s first name is spelled with a “ph.” Sorry, long time fan here :)


Someone said on HN a while back that this sort of approach is the most sensible, because any design doc is just an incomplete abstraction until you get in the weeds to see where all the tricky parts are. Programming is literally writing plans for a computer to do a thing, so writing a document about everything you need to write the plan is an exercise in frustration. And if you DO spend a lot of time making a perfect document, you've put in far more work than simply hacking around a bit to explore the problem space. Because only by fully exploring a problem can you confidently estimate how long it will take to write the real code. Often times after the first draft I just need to clean things up a bit or only half complete a bad solution before improving it. Yes, there are times you need to sit and have a think first, but less often than you might imagine.

Of course, at a certain scale and team size a tedious document IS the fastest way to do things... but god help you if you work on a shared code base like that.

I've always thought the rigid TDD approach and similar anti-coding styles really lend themselves to people that would rather not be programming. Or at least have a touch of the OCD and can't stand to not have a unit test for every line of code. Because it really is a lot more work both up front and in maintenance to live that way.

Cyber-paper is cheap, so don't be afraid to write some extra lines on it.


I really appreciate this essay.

I've never been a [traditional] artist, but I reckon that those working in the arts, and even areas of the programming world where experimentation is more fundamental (indie game development, perhaps?), would intuit the importance of discovery coding.

Even when you're writing code for hairy business problems with huge numbers of constraints and edge cases, it's entirely possible to support programmers that prefer discovery coding. The key is fast iteration loops. The ability to run the entire application, and all of its dependencies, locally on your own machine. In my opinion, that's the biggest line in the sand. Once your program has to be deployed to a testing environment in order to be tested, it becomes an order of magnitude harder to use a debugger, or intercept network traffic, or inspect profilers, or do test driven development. It's like sketching someone with a pencil and eraser, but there are 5-10 second delays between when you remove your pencil and when the line appears.

Unfortunately, it seems like many big tech companies, even that would seem to use very modern development tooling otherwise, still tend to make local development a second class citizen. And so, discovery coders are second class citizens as well.


Yea, TIL, I'm a discovery coder. Always found planning early in Greenfield projects kind a pointless. Planning is almost step 3 or 4. I almost always prototype the most difficult/opaque parts, build operations around testing and revising (how do you something is good enough?), and then plan out the rest.

One trick I find helpful is to start by coding a subset of the problem with the goal of understanding the structure better. A brute-force solution, a simulation, a visualization of data, etc. And then use the discoveries of that process to do the real planning.

I've heard this sort of activity called "pathfinding".

I usually "play around with" with a problem or data, but "pathfinding" sounds much better.

I think it's a dangerous philosophy for professional work. You know what happens to code that works? It ships.

If you code out a solution to the problem in order to discover the problem space, I think the idea here is that you then can go back and write a better solution that accounts for all of the stuff you discovered via refactoring and whatnot. But you're not going to do that refactoring. You're going to ship it. Because it works and you don't know of any problems with it. Are there scaling problems? Probably, you haven't run into any yet. Does your solution fit whatever requirements other interested parties might have? Who knows! We didn't do any designing or thinking about the problem yet.


>>You know what happens to code that works? It ships.

there is nothing wrong with that, it is up to the developer and a team to enforce and gatekeep quality

or up to the business user to accept working code


Any person who would not go back and do that necessary redesign is unfit to call themselves a craftsperson.

Any organization that would not demand you do that necessary redesign is a organization unfit for producing critical systems.

Any organization that would go even further and prevent/de-prioritize you from doing that necessary redesign is unfit to call whatever it is that they do “engineering”.

Why would you want to work at such a dystopian hellscape if you have the choice?

Note that this is only about shipping the known incomplete design to customers because it “works”.

And to get ahead of it, yes, startups selling systems held together by spit and twine are perfect examples of “unfit for critical systems”; you should not bet critical things on them until they mature.


Discovery code doesn't have to be bad code. My rule of abstractions is that I defer them until they are needed, even if I know that they will be needed in the next sprint or whatever. I'll do a much better job making a precise abstraction when faced with an existing solution and a new problem.

Even if discovery code were bad code; I'll take "bad" code 8 out 10 times if the competition is abstractions for abstractions' sake (which "good" code often is). Bad code is often simple and direct, and therefore simple to comprehend and fix. Fancy code is a game of jenga.

Also, requirements often change by the next sprint - so my well-laid plains would be moot either way.


> You know what happens to code that works? It ships.

That's a process problem. Even professional writers have "drafts", you shouldn't be shipping first-draft code regardless. I guess maybe you are the equivalent of a 2 cent rag, and then product sucks, but we won't get into all the ways you can suck as a software org...

Programming is neither writing an article nor designing an engine, its somewhere in the middle. You can apply "discovery" or "drafting" to the professional process as much as you can apply engineering design paradigms. The formal act of writing (unit) testing provides this opportunity, I find, to take the "discovery" implementations, harness them in specific "design" requirements, and produce a polished product.

TDD had this backwards, design then develop, as if that would somehow produce fantastic designs (it doesn't, just a lot of test code). BDD (Behavior Driven Development) is better here, you aren't driven by your tests as much as your behaviors. You may discover them, but once you do you test that they continue to work correctly.


But not taking this philosophy is dangerous if you're trying to do something better than professionals.

It’s very simple.

As soon as the code works, tell absolutely no one.

Then rewrite it….


So better to do all discovery in a system/tool that has no risk of being shipped like excel, whiteboard, design document, conversation, human brain, etc?

Am I going to ship it and not address scaling, refactoring, etc? What about discovery outside a code environment forces our hand on this?

Feels a bit to me like the real issue is not where discovery takes place but that crucial discovery steps are skipped for whatever reason.

I'm stuck somewhere in or around 1 & 2 below.

1) those other systems for discovery suck at capturing behaviors of complex systems compared to working code in motion. Happy to discuss, but I'd love if this were an obvious conclusion somewhere in the neighborhood of The Map is Not The Kingdom trope

2) skipping important steps in discovery sucks no matter where it occur. I've seen this institutionalized/practiced at both ends of the philosophical spectrum. I wasn't being sarcastic or facetious (much) about the questions above.

I'd love it if someone could help me move somewhere past the above. I think I might sleep better at night.


This is basically waterfall and agile in a nutshell.

The extreme end of the other side of the spectrum is research and documentation to the extreme, coming up with a rigid, long development process that doesn't readily allow for any sort of iteration.

The only real question is what is the difference between a working prototype that gets thrown away and an actual MVP. You need buy in from the business up front, otherwise you won't be given the option to refactor or redo parts, you'll be told it is good enough and there are features that need shipping.


If this were the prominent mentality, some of the greatest pieces of software there are would never have existed because they would run into too many brick walls to even try.

Do you never go back and redesign things later? Can we not ship unless we have all ends neatly tied up?


This reminds me of a tongue-in-cheek phrase we used to use in college.

"Hours of coding can save minutes of planning."

"Discovery Coding" sounds fun, but be careful with your time!


Common Lisp, with its excellent REPL, is great for this type of exploratory work. The ability to build up a function from the inside out, and to build systems and classes from functions, again from the inside out, trivially creating mocks and ad-hoc tests as you go, is fantastic DX.

I feel I finally can put a name to my coding style! Incidentally, I also like to pretend classes and functions exist even if they don’t. Of course my discovery code won’t work, but I can go very far and discover a lot of useful information this way.

In my experience, some projects are very suited to discovery coding.

For my Machine Learning projects, I usually run a bunch of experiments either using pytest or directly on jupyter. I tend to plot out stuff and try out different types of feature engineering. Once I have nailed the approach, I then port a subset of this to a different jupyter notebook or python script, which is cleaner and more readable. This is because ML experiments are compute bottlenecked. So I want to ensure I spend enough time to pick the best features and model.

At work (which is not ML-related), I tend to do much less discovery coding because most of the unknowns are business related - does this API (owned by another team) scale, what is the correct place for this business logic etc. And, doing a viable Proof of Concept is time consuming, so I'd rather spend the time sweating out the nitty-gritties with product. The Discovery here must happen in the discussion with Product or other stakeholders because that is the expensive part. This is also why Product changing the Spec once the project is underway is infuriating. Sometimes a good chunk of the discovery is nullified.


Casey Muratori talks about this in his Handmade Hero videos. I believe he calls it exploration-based programming.

Discovery coding may be fine, but discovery architecture is a disaster.

Nah. I'd almost argue that discovery coding is what helps define the architecture. I've seen way too many cases of over complexity designing a system for scale that will never be hit which results in an architecture that is too rigid to add new features to. If you do discovery coding you would realize the real bottlenecks and functionality that's required and can build an architecture that addresses those concerns instead of just designing for design's sake.

Yes and no. A lot depends on who the architect is.

This is how I've operated for most of my career and I partly attribute my success and productivity with it.

There is no reason that a discovery programmer cannot create a highly structured, rigorous end-artifact.

They often stop before they do.

Designing software is a human process and not all humans are the same.

...nor is the quality of their results.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: