Agree with almost all of this, but one section bothered me:
> This problem is, by the way, the thing that takes most of my time, 60-70% of my time at a guess. I once spent over a week trying to get a broken LDAP server to work - my boss had forbidden me to implement my own LDAP server - but after a week of struggling with a broken LDAP sever written in C and badly documented I had a lapse of memory and forget that my boss said and accidentally implemented a server from scratch in Erlang during my lunch break.
> To be honest it wasn't a full LDAP sever, but I didn't want a full LDAP server. I only wanted a couple of commands to work, and that was pretty easy to fix
Knowing when to flip the table and reimplement is probably more challenging than anything else outlined in the article.
Joe probably had a good sense for that. Most programmers, however, do not seem to (requires tremendous breath of experience!).
It takes a certain amount of arrogance to assume that you understand (or can predict) the entire problem space along its warts. And if you're wrong, it's often another poor programmer down the line that pays for it—not you.
I had a similar experience with a silly little content assist widget of all things. The company I was with had a very particular set of standards that made a lot of sense for the sort of work they did, but none of the features we wanted were standard in the common drop down libraries. After about a day and a half of trying to wrap my head around some of the hilariously complex content assist widgets, I had a conversation with a colleague that prompted me to write my own. Two hours later, I had it. And it was compact and simple enough that other developers were able to easily make modifications as our problem space evolved.
This is an eternal problem of software design, when do you borrow, and when do you implement your own. I don't know the answer but I think there are some factors that can lead you in the right direction.
Did you do your homework? Implementing your own tools should be the last resort not the first resort. It should be driven by frustration at the inadequacy of current tools, and not based on a whim to do something cool.
Of the examples tools you've found, how much customization is required to get them to do what you want? If a 3rd party library requires a lot of hacking to do what you want, it's a hint that those tools don't actually solve your problem and you're going to end up hand rolling what you really want anyways.
How much overhead do the other tools bring in? Sometimes you need to do one simple thing, but all the tools available have a lot of overhead. That's a hint that maybe you should do your own implementation
Have you talked to other people about this problem? A conversation with a capable colleague is usually the fastest way to get useful perspective on what exactly you're trying to do.
How well do you understand the problem space? No matter how much it may suck, if you don't know what you're doing, take the time to figure out how to use the tools instead of creating your own.
All of these factors must be balanced against each other. One of the most important things though is that if you do decide to hand roll a specific tool, don't keep it to yourself. Such pieces of code can be a little hairy sometimes and you need the feedback from others to ensure the code is clean, and to make sure they know what's going on.
> Did you do your homework? Implementing your own tools should be the last resort not the first resort. It should be driven by frustration at the inadequacy of current tools, and not based on a whim to do something cool.
IME having to learn a fancy framework to edit a couple of things on an open source app that was mostly what we needed hurt a lot more than it would have to make something very KISS with something like Flask as a proof of concept.
Whether choosing to implement your own thing from scratch should be a last resort, or that it necessarily requires tremendous breath of experience, really depends on the circumstance, as you described. In many circumstances you don't have to be a genius to know NIH is probably the sanest option.
Back around 2000, I installed tftpd so that I could boot a VxWorks board. It came locked down for security to the point that it wouldn't even work, and the documentation was no help in making the tftpd operable. After much frustration, I realized that writing tftpd with plain old C and sockets would be easier. I did that.
A decade later, I again needed TFTP. Surely the passage of a decade would have sorted out the issues. Nope! Having long ago lost my reimplementation due to job change, I very nearly wrote TFTP yet again. This time I was saved by the discovery of hpa-tftpd, which actually works.
I wonder why hpa-tftpd is not the default or only TFTP server packaged with Linux. The regular package is, as far as I could tell, non-functional.
> I don't want to have to read a 391 page manual first
Actually, I have no problem reading the 391 page manual first because I've learned, the hard way, that that's the best (only) way to make sure you're not overlooking something - it's my boss who has a problem with it. Thy fingers shalt be on thy keyboard at all times and thou shalt be closing Jira tickets as fast as possible.
Well, I guess that depends on what you mean by an ideal setting: in an _ideal_ setting, everything would be so intuitive that there'd be no need to read any documentation at all; you'd just have a goal, and you'd sit down and immediately start working toward that goal. You could consult an all-knowing "oracle" if you had a specific question that you needed the answer to and that all-knowing oracle would always provide a correct and specific answer. I don't know if this is actually achievable, but I do think most software tasks are estimated under the assumption that it's not only achievable but the present state of things. If the manual exists, though, then whoever wrote it thought it was important to present topics in a specific order rather than just organizing it in an "if you need this result, then do this" FAQ form.
"Using the Google casino for bug fixing is terribly frustrating. I Google a bit and after a while find a posting where some poor unfortunate soul has encountered exactly the same problem that I have. My heart leaps for joy. My trembling fingers enter the magic spell that will remove the curse, and ... nothing. The problem remains." LOL
I really like this quote. It succinctly summarizes what being a programmer is for me. However I would like to add to it that you have to know what to ask when you get stuck.
When I try and learn something and then get to the point where I know what questions to ask, that is when I know that I have learned something.
If you’re staring at a problem and you don’t know what it is or how to describe it then you can not progress, but that is how you progress. You try and fail and ask yourself why you failed. Building on failures is a slow way to progress, but gradually those failures becomes knowledge and it gets easier to progress.
When I learn a new language, tool, library or feature my knowledge gets me going quicker. It doesn’t necessarily bring me the whole way home each time, sometimes you really have to learn something new, but looking at that thing that I know that I don’t know is what gives me confidence in feeling that I will learn it.
> This problem is, by the way, the thing that takes most of my time, 60-70% of my time at a guess. I once spent over a week trying to get a broken LDAP server to work - my boss had forbidden me to implement my own LDAP server - but after a week of struggling with a broken LDAP sever written in C and badly documented I had a lapse of memory and forget that my boss said and accidentally implemented a server from scratch in Erlang during my lunch break.
> To be honest it wasn't a full LDAP sever, but I didn't want a full LDAP server. I only wanted a couple of commands to work, and that was pretty easy to fix
Knowing when to flip the table and reimplement is probably more challenging than anything else outlined in the article.
Joe probably had a good sense for that. Most programmers, however, do not seem to (requires tremendous breath of experience!).
It takes a certain amount of arrogance to assume that you understand (or can predict) the entire problem space along its warts. And if you're wrong, it's often another poor programmer down the line that pays for it—not you.