That book (Ed Mastery, https://mwl.io/nonfiction/tools#ed) is an April fool's joke, but it's a real book and I thought it was actually really good. I've still basically never used ed but it was fun to learn about and it helped me understand vim's history a little bit.
I did use ed, ever so briefly. I was introduced to it when my employer brought-out a new machine that ran SVID. We talked to it through glass teletypes.
The commands in vi are based around the commands in ed. They belong to the same family. sed is also a family-member, and an amazing tool. But I won't be sorry if I never run into ed again. Essentially, it's a rudimentary editor with zero dependencies; it doesn't need a curses library, in particular. And it would work on a terminal with only a printer for output, which was useful, back then.
I still use ed - Ed formatted patches are great for retro computing, I would very rarely use it interactively, but scripting with it is surprisingly useful.
I’ve used ed scripting in serious production work across thousands of hosts. It was nice to have that programmability but be able to “reason” with the entire file.
Like I said in the blog post, I'd love suggestions for other things I've missed that happen when you run "hello world", _especially_ if there's a way to use a Linux spy tool to trace what exactly is happening.
Considering adding a few things that are happening in the kernel and trying to use `bpftrace` to trace them, but I need to figure out how to use bpftrace/kprobes and hunt down the relevant kernel functions.
Hello! I wrote this post and I have a couple of things to say about this "DNS is not actually hard" take. It took me many years to feel totally comfortable debugging DNS problems, and I wrote this post to explain why I think it was hard for me.
I also used to think that "no, actually, it's easy!" was an encouraging response to "this is hard to learn". And I kind of get it! I love DNS! I think it is surprisingly simple in many ways, and I've written about that a lot, for example in https://implement-dns.wizardzines.com which shows you how to implement a toy DNS resolver from scratch in some pretty simple Python code.
But over the years I've learned that "no, it's easy to learn!", instead of coming off as an encouraging comment ("you can do it!"), often gets received as "no, it's not hard, actually the problem is that you're dumb". Like, I've been confused about this for years, and you're telling me that, no, actually it's easy? Not that helpful!
So I've stopped telling people that, and instead I put a huge amount of work into trying to understand _why_ people find certain things hard and work to help remove some of those barriers.
Just my 2 cents: I worked on the DNS in Windows Server back in the 200x's (validating the UI and back-end functionality, not dev work), so I did have to learn about it at one point and I recall that there was a vicious level of complexity to it compared to what we ask it to do. The 99% scenario is turning a domain into an IP via an A record. Next after that is doing the same with an MX record. Beyond that is this deep well of other things people thought someone might want to do with DNS, but that almost nobody takes advantage of in real life. So DNS is probably easy to understand if you're just concerned about the one or two first class scenarios, but if you had to implement an RFC-correct instance of a DNS server, it's a bit of a brain melter.
This is the comment I wanted to make. DNS is one of those technologies that has a simple explanation - "it's indirection" - with consequential ramifications that turn it into someone's job. Most people will not interact with it often enough to know how to perform the job, so they remain hesitant and try not to do anything at all with it, because it burned them once before.
Git is analogous - most uses of Git are formulaic, and the underlying concepts are simple enough - but actually accessing the right lever to pull when disaster strikes is unclear and hard to experiment with. I'm in this latter category with DNS: I get it at a high level, but it's like a student who has only done the simple example project.
I'm of the opinion that we're at a good moment to redo the things DNS does with better separation of concerns by going towards "put those records on a blockchain, streamline it for known applications, reframe the hard problem around bootstrapping access to on-chain data". It's already been explored in varying degrees(e.g. Ethereum Name Service, Symbol namespaces) but it's not really something that has to be the monopoly of any specific chain.
ANYTHING that has the word "blockchain" is automatically disqualified from pretty much any real use.
In particular, pretty much nobody wants to trust DNS names that can cost hundreds of millions to blockchain. Because if you lose (or compromise) your private key, then that's it. The domain is gone.
With regular DNS this would be solved by a couple of irate phone calls to registrars. At worst, via a court case.
> Git is analogous - most uses of Git are formulaic, and the underlying concepts are simple enough - but actually accessing the right lever to pull when disaster strikes is unclear and hard to experiment with.
Would you mind sharing some examples of hard-to-recover git related disaster? I ask because I previously worked with applying hundreds of patches on top of chromium (yes, Opera). This resulted in __multiple__ merge‐conflicts. However not anything close to what I would call a git related disaster. Anything could be undone nicely.
What is important is that git is very verbose by default. It guides through not only a repo current state but also options user has.
I've met many developers not caring to read any of a git output and then claiming git is too complex and git broke their repo and it's "too complex to use". I'm sure it's not the case for you though.
So I'm just curious what kind of disaster was mentioned above.
> I've met many developers not caring to read any of a git output and then claiming git is too complex and git broke their repo and it's "too complex to use".
Not OP, but I've encountered a few situations where the solution was modifying .git/ manually. The one that comes to mind was having two branches with names separated only by case on a case sensitive system, and then checking out both on a case insensitive system.
It's been awhile so my memory is fuzzy, but this was at a windows-only shop back before WSL was a thing and getting a VM involved an annoying months-long requisition process. I think the repo state was actually corrupted too, so pull may not have been working.
Just on the topic of git: git makes it extremely easy to experiment! But far too many people, including those writing tutorials and blog posts, don't understand this. Whenever you're in doubt, just commit your work and make a temporary branch and switch to it. Then you can mess about with merging and rebasing and dealing with the conflicts to your heart's content.
This should be among the very first things we teach to new git users.
Maybe everyone sees this differently? Or maybe is more than one viewpoint to it.
For example, if someone tells me that something is "actually really simple", and I did not get it yet, I tend to think that I likely (not certainly, but likely) have not found the right approach to that something yet, and once I found that way to look at it, things will resolve itself.
But if people assure me that something is "hard to learn" or "hard to get", I'm rather suspecting that that thing is full of unwieldy complexity and/or unintuitive aspects, and unless I'm either up for the challenge (definitely the case sometimes), or learning it is crucial for achieving a certain goal (in which case it may turn out to be fun after all, even if still hard), I might think twice if I want to put the time and energy into it.
In the case of DNS, which I've learned in the nineties, I found the principle simple and elegant, the tools and software unwieldy and sometimes arcane, and a large multitude of little details that are not strictly necessary for understanding and using DNS in basic ways, but that can get important when either debugging or working at scale.
So, in the case of DNS, I'd say the answer is "depends on how deep you want to go", but don't shy away from the simple and elegant principle, even if the arcane software (it has gotten better, though) stands in the way.
I think that matches the content of your article overall?
> Maybe everyone sees this differently? Or maybe is more than one viewpoint to it.
> For example, if someone tells me that something is "actually really simple", and I did not get it yet, I tend to think that I likely (not certainly, but likely) have not found the right approach to that something yet, and once I found that way to look at it, things will resolve itself.
At least in my experience, people asserting that something lots of people have trouble understanding is "not hard" are doing so as a flex at least as often as they are trying to be helpful, so I'm surprised that a lot of people don't make the same hypothesis you do. When someone does so while also touting their status among their peers due to their knowledge and talking about how anyone could do it by just putting in the work, it's not surprising that people might interpret that as implying that people who don't have the same knowledge are either too lazy to put in the effort or not capable of it, regardless of whether that was the intended message. If the goal is to try to help people, it's more effective to communicate in a way that conveys understanding and not judgment; if someone doesn't care to improve their methodology of helping people, that's fine, but it does raise more doubts about whether they're being honest about their intentions.
> anyone could do it by just putting in the work, it's not surprising that people might interpret that as implying that people who don't have the same knowledge are either too lazy to put in the effort or not capable of it
Not spending time on something doesn't mean you're lazy or stupid, that seems like your own leap. If you choose to interpret in a charitable way it just says: "if you haven't looked here don't worry, it's not that hard, you'll just need to spend some time".
I haven't looked under many rocks I've walked by, talked to everyone, read every book I encountered, doesn't mean I'm stupid or lazy, I just haven't done it.
My point is that if your goal is to teach people, you should care about framing things in a way that's conducive to the people who you want to teach. Dismissing fairly common insecurities because you don't happen to have them will significantly reduce your effectiveness at teaching, and if you don't find it worth the time to take into account feedback about how to craft your messaging more effectively, I'd argue that your priority never really was teaching in the first place.
The nature of what's being learnt. Some things require a continuity - to understand B, prior A is needed (or helps, to understand faster).
The method of learning. Book/theory-based, or practical? For either, what's the nature of scaffolding (self, or via resources) to help leap the chasm? If testing one's self, what's the complexity and can that complexity be broken down into simpler (or more discrete) parts, (perhaps testing working better in smaller parts)? Perhaps A isn't fully (or at all) required to 'know' B, depending on how it's learnt. Which goes on to -
The nature of the learner (at that point for that task). Someone that's looking to solve a task, somewhat surface, or someone that's interested and will go deeper into edge cases or approach with greater curiosity?
[I'm skipping the nature of the learning/knowledge, since 'resolving DNS' is a pretty externally verifiable result. However it might be fruitful to consider the nature of the learning is not only 'resolving DNS', and even if 'resolving DNS' fails, learning always happens (intended/unintended, positive/negative, a can of worms there).]
You point out that 'easy' and 'hard' are motivators that might have unexpected, or the reverse, effects vs. intended, depending on the reader. When putting it into those 3 parts, perhaps this shows the usefulness of framing.
My own hassles with DNS, some of which are mentioned.
1. windows and apache httpd and others cache things
for a long time, seemingly forever.
2. The 'A' record is used in case of missing MX records. Who knew that?
3. (in the olden days) confusion with winbios and other naming schemes
4. The defn of 'authoritative'
5. the misleading semantics of 'domain' in resolv.conf
6. (minor) the casual use of CNAME to mean alias, whereas it fact
means the opposite and is used to refer to the record, not the name.
7. That DNS can use tcp/53. The number of misconfigured firewalls I've
dealt with argh.
8. Split vs forwarding DNS. e.g. impact on the .dev
domain.
9. The urge of some to put underscores in their names.
I believe that some (early?) implementations allowed
them.
10. Someone putting in a lower 'serial' in named.conf and
the struggle to restore sanity.
DNS's caching behaviour in the wild is incredibly infuriating. We have one team member at my company who is stuck with a stale DNS record for one of our internal services in our application, but their browser works just fine. I've absolutely no idea how to actually fix it, so for that one persons benefit we bought a whole new domain and ported everything to that.
Split DNS has let us do some cool things, and simultaneously caused no end of headaches. See above for the sorts of headaches.
i thought to post about how i thought "dns looks fairly straightforward" (which i guess it is on paper) then learned all the nuances over the years of maintaining a few services across a few domains and debugging network issues as a networking fw dev. i paused and the difference is perfectly between the the parent and grandparent post really.
some things seem deceptively easy and you can get by for years on basic features. then it happens, you try something new you learned you can do with $THING , add to it and things start to break and you dig in and realize you never fundamentally understood how the basic features that got you here worked. and thats ok
My own response to "X is actually hard to learn" is ... compared to what?
1. compared to your first impressions?
2. compared to general opinions that you're aware of?
3. are you talking about general principles or deep down stuff?
4. are you talking about the learning curve? Is it irregular? Steep? Shallow?
"X is not hard to learn" is ultimately a pretty empty statement.
jvns, your articles and conversational writing style are great.
I hear you saying "'X is hard' is a way to not lose a reader by insulting their intelligence". My immediate reaction is "'X is easy' is a reminder that things that look hard upon first examination can be quite easy".
I think in general this level of "kid-gloves" is overall unhealthy for society. It encourages mental weakness.
At the same time, I understand that your aim is to reach as large an audience as possible so your educational material can have max impact. For this reason, I can justify you having this stance. That said, I don't think it applies globally.
Julia, I agree with your take here, thank you for bringing this thought process into the mainstream. There are many other concepts this happens with and in other ways: people sing praises of the pedagogic excellence in Skiena’s & Sedgewick’s algorithm textbooks (and the awful EPI book by Aziz et al) but I hate reading them, they’re actually written horribly to explain things, IMO. I think this form of difficult writing on top of already difficult concepts is a weird type of gate keeping (perhaps sometimes unintentional?) and it should be tackled all over Computer Science by better writers who have an explanation style like yours.
Simple is not always easy. Too simple can be difficult to grasp and sometimes adding a little complexity can actually make something more intuitive and easier.
DNS is simple, but it is not necessarily easy to understand.
an aside your content is top notch in quality and I always point to your site(s) when I want to share how this stuff works with an easy explainer. You make it seem easy :)
I debated whether I should respond to you at all, however I feel that I must. You and I don't know each other, and based on the last sentence of your reply I suspect we agree with one another more than we disagree. Nonetheless, your reply seems to imply that my comment and my earlier disagreement were done out of malice or arrogance. A different commenter used the term "gatekeeping".
Fundamentally most of the building blocks of the Internet that people interact with regularly, including DNS, are well-defined and relatively easy to understand /relative to other technical things/. Protocols like DNS are effectively shoving text into packet. All of the hard stuff isn't in the bare protocol, it's in understanding the complete stack of abstractions the protocol relies on, the abstractions that allow the protocol to be simple but can sometimes not work in unexpected ways. If there is anything I've learned in my life, it's that integrations and scaling are what make technical things hard, the basic protocols are generally very simple and they are robust primarily because they are simple. They are simple primarily because they get to rely on all the underlying abstractions to be there. Technology now exists in a world of abstractions on top of abstractions, and understanding the basic building blocks is now rare since many technologists focus on mastering a particular layer of abstraction, yet if you understand the basic building blocks, even in an incomplete sense, it can greatly empower you to be more capable and comfortable with the abstractions.
"Hard" and "Easy" are relative terms, and I do believe that saying things are hard can lead people to avoid them altogether. I have spent a considerable portion of my life working diligently to mentor people in their technical careers, foster technical understanding, and bring people along with me on knowledge journeys. I have done this through extensive written documentation (internal to companies), teaching classes (internal to companies), public conference talks, guest lecturing at universities, and mentorship of people who are interested in learning technology but come from non-traditional backgrounds. I myself come from a non-traditional background. I've also worked hard to improve the UX and accessibility of every piece of software I touch, because I think that technology as a tool and the information about how it works should be accessible.
I really appreciated your article and I think you have a great writing style to explain things in an understandable way. Your Implement DNS project is really great, and I will be linking it to others in the future as they go through the process of learning DNS. In no way was my disagreement before or this reply meant to throw shade on you, your article, or otherwise. It's because I believe saying things are hard actually discourages people from learning them, leads to avoidance, and in particular I believe DNS is one of the more simple of the many possible things someone can learn about how the Internet works. I am /very much/ not trying to in any way disparage the intelligence of anyone who is struggling with learning any technology. I believe that learning technology is much like learning anything else, it requires interest and time. Saying that some of the simpler things are hard can discourage people from having the interest, and if people do not have the interest they will not invest the time.
Very specifically, I think for the audience here on HN, they've had to learn and use technologies that are far more difficult than DNS in their lives and careers with an almost certainty. In a relative sense, I do not believe DNS is difficult to learn. DNS has a /lot/ of edge cases though, which can be hard to troubleshoot, understand, and resolve, so from this perspective it is hard to master and I wouldn't claim to have fully mastered it myself. As an example of what I mean, another commenter mentioned Git. I have a pretty deep understanding of Git as well, yet I consider it significantly more difficult to learn and understand than DNS, largely because DNS is better documented, easier to inspect, and doesn't require the technologist to understand complex algorithms like merkle trees. Yet almost every tech worker in the world uses Git every day, often with many frustrations and weird edge cases, as in regular usage it fails far more often than simpler technologies like DNS. The entire strength of DNS from a robustness, adoption, and resilience perspective is largely its simplicity, which is why I think relative to the many other things that exist that it's not hard to learn.
Our opinions are always colored by our experiences, as is mine, which I stand by. One of those experiences I'll recount here, which is that I have known a woman for a little over 6 years now who is currently working in IT and finishing a bachelor's in IT through WGU. She's one of the top performer's on her team and has a gift with understanding and explaining abstract concepts to users, which has made her well-liked by all at her company. Not long after I met her she shared with me that she'd always wanted to go into IT, but felt like she couldn't do it because everyone she'd met along the way had told her it was too hard for her because she suffers from a learning disorder called dyscalculia which interferes in her ability to do math. For reasons I don't fully understand, math ability is used by many technical people as a determinant of your ability to learn and work with technology. I was one of the first people she'd met in her life that encouraged her to pursue her interest in technology and told her she could do it. I did so by pointing out that many of the things people were saying were hard were concepts she already understood in the abstract and offering to go on the journey with her. She had been so dejected by others telling her things were too hard for her she never pursued a college education. I invested significantly in learning with her and 3 years ago she finished an associates and multiple certifications and started her first IT job, and she's been excelling in it the entire way.
I've continued to mentor her, along with many others, because I legitimately do believe that much of the technology around us is much easier to learn and understand than people commonly believe, and that understanding how it works is a key to understanding how our world works now and is incredibly empowering even for people who don't work with technology every day. I am adamantly opposed to any sort of gatekeeping of technology knowledge, and in no way am advocating that gatekeeping or trying to disparage those who struggle to learn. If anything, one of my existential fears is that as a society we fall into a situation where so few people understand the technology we rely on every day that it creates a new and more dire social gap beyond the wealth gap, a knowledge gap that fundamentally separates those who can effectively participate in the levers of power (including democratically) from those who cannot. I am heavily invested in ensuring that knowledge is free, accessible, and that there is an open pathway for people to learn and it's one of my core optimisms about the Internet.
Actually, DNS the protocol is a horribly complex beast with various compression mechanism, tricky encoding rules, workarounds for MTU limits, etc. Most people "learning DNS" never even attempt to learn the protocol.
What is reasonably easy is a high-level understanding of e.g. "given this question, this is what one expects an authoritative DNS server to respond" in the human-friendly form, covering the basic kinds of questions (A/AAAA/MX/TXT/CNAME).
As always, thanks for the article (and the debate).
DNS itself, as an internet protocol, is "easy" in principle.
I did implement and maintain DNS services at the beginning of my career, for more than 200 customers, without any title and without help, except "the internet".
Everybody understands "you ask for a name, you get an address to connect as response", or "you can have more than one MX entry and play with the weight", "what is a TTL", etc.
But as every internet protocol... as soon as you add to the recipe: historical changes and more than one RFC, upgrades to the protocol, cryptography, different client and server implementations, vendors, operative systems and libraries (and versions), levels of compliance, software applications, misconfiguration, bugs, layers and more layers (inside client, reaching intermediate server(s), final server issues), cache!, enterprise interests (from client to server side) with a wild global (or local) positioning, government and agencies interests, ageing, innovation, all issues related to the long running opensource projects related to the protocol, all issues related to the "new is better" opensource projects (client and server) related to the protocol, user/developers/customers/managers asking (or assuming) impossible things or that there are no limitations, a long history (and a future one) of known attacks and known defenses or mitigations, response ordering (/etc/gai.conf), clients that implement from the last century /etc/hosts to mdns and avahi to nsswitch and ncsd, clients that act as servers (developed by enterprise bug creators)... you know... not easy anymore... things gonna be wild.
If there are complementary protocols, let's multiply the complexity: for DNS issues, there is also domain registration issues, whois, DHCP, VPNs, intermediate firewalls, rate limits, anycast, replication, IP, ARP...
With any internet protocol, or anything related to computers:
1) While it works, it's fine (and everybody is happy)
2) While we get an issue previously seen, understood and fixed, it's still calm
3) While we get an issue that we can quickly search by symptoms and fix, maybe ok
The "not easy" comes after that.
When we get a new and unknown problem, a weird issue, unexpected things, and there are 20 or 30 possibly failing actors around the protocol, or XX involved parts.
Recently I did a written breakdown of all things related to DNS in the "strace -y -yy -fq -v nc -vz google.com 80" in a linux laptop, and many people at work was surprised of all the things that happen there, inside the "give me the IP" step locally.
We could say exactly the same against any issue related to any internet protocol:
HTTP? it's easy or hard
Depends. It's easy to explain in a sentence what it does.
But you don't want to be the guy fixing issues against a float of load balancers used by all the customers, with a legacy (non compliant) device from a high priority customer at 4 A.M. oncall.
A DNS resolver is both a client and a server -- for example Google's 8.8.8.8 (which this is a toy version of) is a server (you can query it with `dig @8.8.8.8 example.com`), but also a client of the various authoritative DNS servers that it fetches and caches records from.
I implemented this as a command line tool because that's much easier to do in a Jupyter notebook environment, but you can also pretty easily transform it into a UDP server running on localhost and query it with dig in the same way that you would with 8.8.8.8. That's one of the bonus exercises at the end (Exercise 7).
I might end up bringing "convert it into a server" into the main content though because it's pretty easy to do and I think it makes the whole thing seem more "real".
Nope, Google’s quad eights is “recursive DNS server”.
Resolver is usually a part of operating system (sometimes) implementing DNS client functionality and serving as a link between a userland library providing, say, getaddrinfo(), and DNS implementation.
This one is even less than a resolver, it does not implement a library link. It’s a toy DNS client, implementing minimal functionality.
There's this by Andreas Zeller: https://www.debuggingbook.org/, it's an interactive free book with implementations of a bunch of different debugging tools in Python
