I feel like walkie talkies are something all network engineers need to play with. Furthermore, remember that real world telephone and radio networks were in fact... Networks. (Called 'nets' in the radio world)
Ethernet, and 802.11, are both based on the concept of a singular channel shared by many individuals. When Alice talks, she says 'I'm Alice calling Bob, can you hear me Over'.
Alice is the source address. Bob is the destination address. Over is the stop code/stop bit signifying the end of frame. The payload is 'can you hear me'.
-------
This is everything up to layer 3. But if you wanted to send a radio/telegram across the country, you need one more layer, IP4 or the ARRL protocol (American Radio Relay League meets on the air at predestinated times to pass messages to each other on Ham radio frequencies. It's a human network)
I'm Alice, calling the Eastern Operators. I'd like you to relay a message to Bob for me. The message is 'Can you Hear me', Over.
This is a radio frame like all others, but an additional layer has been added. The radio operator has its own protocol built on top of the Frames.
------
Dave may overhear all of this chatter, but Dave knows to stay off the channel (otherwise he will cause interference). Only if Dave is addressed specifically would Dave respond (Ex: This is Alice, calling Dave. How are you doing? Over)
We can see that the operator is itself just another person on the network. The operator may have a message from the rest of the world.
This is the Eastern Operator calling Alice. I'm relaying a message from Bob. The message is 'I got your message Alice!'. Over.
This is how a router may take a message from the greater internet and pass it to your local network.
full duplex ethernet circuits (eg: a basic cheap 10Gbps 1310/LR carried over two $30 transceivers and a six foot fiber patch cable between two pieces of equipment) and a half-duplex TDD/CSMA regime like 802.11b/g/n/ac/ax are not the same thing.
yes they both enable ethernet connectivity between devices on fabrics of things that can see and arp each other.
the ethernet 10GbE might be shared by many individuals. but it's full speed both directions. one is half duplex, the other is not. one has contention on the shared media airtime, the other does not.
they are both possibly shared in the sense that many peoples' traffic may be carried across the 10GbE full duplex circuit, the the real world implementation is very different.
the real world implementations of FDD point to point (FCC part 101) or similar radio systems are closer to wired ethernet in concept, though with less capacity.
however I do agree with your general point that more people should familiarize themselves with the limitations and usage methodologies for half duplex IP data mediums and the best real world implementations.
I probably should have clarified that when I said "Ethernet", I meant classic 10 Mbps Ethernet that today's kids probably don't know about. The Ethernet hub acted like a bus: anything one connection said was broadcast to _ALL_ other connections.
Of course, today we have far faster point-to-point switches. But it helps to remember that the Ethernet-protocol was built in the days of old: half-duplex shared communication broadcast to everyone on your hub.
-------
Today's wired 1Gbps or 10Gbps ethernet are way way faster than half-duplex channels that 802.11 / Wireless technologies are forced to use (after all: the physics behind 802.11 and walkie talkies remains fundamentally the same)
There probably isn't a need for CSMA/CD systems in Ethernet anymore (since today's networks are all dedicated wires). But still, the CSMA/CD system exists for a reason, a legacy of the old "shared ether network communications" that gives rise to the name "Ethernet". (One Ether that everyone on the network can hear simultaneously, like a walkie talkie)
Raises hand. I remember discovering promiscuous mode, and a stern talking to from our network engineers. It was obvious I didn't know what the hell I was doing, so I got a great tutorial about the differences between hubs and switches.
I'd argue that the legacy of CSMA/CD still alive in ethernet is what makes it harder to implement efficient networks, as the switched network kinda tries to emulate the shared bus behaviour.
I admit that I'm not actually a network engineer at this point. I took a networking course back in college and what I've discussed so far is largely just the basics and how I've internalized my lessons from back then.
Looking at CSMA/CD more specifically, as well as "full-duplex Ethernet" (which are 100Mbit, 1Gbit, and 10Gbit modern Ethernet), it seems like CSMA/CD is no longer used in any full-duplex Ethernet Protocol.
I'm not 100% sure about that, since its outside the scope of my old studies. But it makes sense to me. It seems like "full duplex Ethernet" can only work with point-to-point communications. If you have point-to-point connections between exactly two stations, there's no need for carrier-sense (CS), multiple-access (MA), or collision-detection (CD) protocols at all.
CS: Wait for the current person on the walkie-talkie to say "over" before you yourself start talking.
MA: Walkie talkies: many people can access the same channel.
CD: If two people accidentally start talking at exactly the same time, they need to independently try to restart-talking. A random-number generator with exponential backoff is common for this sort of thing IIRC, though I'm not keen on the exact details.
The problem is that the switched network has to emulate behaviours that were free on CSMA/CD system. So we get MAC learning, Spanning-Tree Protocol, and ultimately if you want performant large scale ethernet switching (and not just terminate ethernet at point2point) you end up with complex software directly managing forwarding tables (all sorts of SDNs) or essentially turn ethernet into routed network (using IS-IS routing, two standards exists for implementing it, one of them - TRILL - is reportedly backbone of Cisco Nexus Fabric system).
There's a reason why another name of Layer 2 ethernet network is "broadcast area".
The book “CODE” uses exactly this pattern. It starts with two people talking to each other via cans connected with string, and works its way up to complex networks and processor architecture.
I recently picked up CODE after seeing it recommended on HN a few months ago and it's really filled in the gaps in my knowledge of how computing technology evolved.
There is still so much of the tech stack that hasn't changed since this book was published in 2000 - TCP, IP, DNS, binary, logic gates, etc. But there's also much that has changed that could be written about - virtualisation, containers, wifi, fibre, cloud infrastructure, USB, GPUs.
I've gone ahead and bought the first edition, and then when the other is out, I'll add that to my collection too. Looking forward to reading it. No matter how much I learn, I find going over the basics again, reading introductions again, really helps to keep everything active up there. Let's face it, we learn so much but we also seem to lose so much as well.
Well, there is already a term for that: "Bottom up". I used to think learning abstract design "Top down" is a better way to get good at design, but the "Bottom Up" way, by walking me through the implementation and design, sticks with me in the long run.
Many of us are amateur radio operators, which is basically walkie talkies. And furthermore, when we're all having a group discussing, it very much follows your last point: https://en.wikipedia.org/wiki/Amateur_radio_net
There is no true radio analogue here in the sense of how OP was explaining it.
At least in a wireless context, CDMA and TDMA are flawed examples because, CDMA uses fancy math to split communications over multiple channels and TDMA cuts the channels up into timeslots. You don't get full bandwidth allotment and utilization.
To make a wireless switch, you'd need to have PTP optical/microwave/radio links which have highly directional antennas. This would give you full-bandwidth per user.
Even in a wired context, frame-based protocols inherently utilize TDMA (over multiple channels), just not on timescales you normally think about.
If I'm not mistaken, the radio example uses both frequency division and time division. You're on a particular channel (typically frequency, although AM exists too), and only one person can talk at a given instant, or else you get a collision which results in probably garbled output. (There's also space division at play, since radios are not infinitely powerful.)
If the relay hop is also done over radio, then it's most likely achieved by hopping to dedicated frequencies for backbone comms, ideally with a separate antenna per channel so we can listen on all of them simultaneously. (You could constantly sweep the frequency space, but it's simpler to just have multiple antennas / radios.)
A (modern) real-world example is that I can use my phone as a wifi hotspot. It relays packets to/from its upstream link (4G or even wifi again). (It also presumably behaves like a NAT, but that's an implementation detail.)
If you wanted to add a learning switch to the example, you'd also need to add the concept of multiple channels. Once a switch learns which channel the recipient is on, it would only send to that channel instead of broadcasting to everyone. It still starts to break down unless you torture the analogy a bit more.
I was about to comment the exact same thing. This book's bottom upwards approach is something that gave amazing insight on how well designed the Internet and computer networks are. I will forever be grateful to these guys for making free so a broke student like I was could afford that knowledge.
I recently moved from a professional developer that didn't know how networks worked to one that gave an intro talk on HTTPS to a student group. I wish I had had a book like this to make that change faster.
One callout: a table of contents is needed for the online version. One that links directly to the chapters. And if one exists, it needs to be easier to find because I couldn't locate it.
>I recently moved from a professional developer that didn't know how networks worked to one that gave an intro talk on HTTPS to a student group.
That's awesome!
>I wish I had had a book like this to make that change faster.
That's the goal.
I definitely need a table of contents, but I was planning on adding it once I got a few more chapters done. Based on the comments though, I should probably add one quicker than that.
> The answers I get have taught me that most programmers don't have a basic understanding of how the internet works outside of their narrow specialty
Just yesterday I couldn't fall asleep because my stupid brain thought something like "What information would I need to fully grasp the topic 'programming'?".
Since I am a visual thinker, I quickly started to draw a mind map in my head that basically started with the history (starting with Alan Turing), going through electrical engineering, went through different layers and finished in different programming languages. The languages alone branched out in a pattern so large that my head startet to hurt. It included different languages, concepts, architecture patterns and compilers.
Each one of those steps is important and you will find people specialized in any of those which will tell you that their part is the most important.
I am a web developer myself and I try to learn and improve every day.
But I also think to be really good you have to focus on a few specific parts of the process and not try to be an expert in everything there. That grew way too complex quite a while ago and trying to understand everything usually only makes you mediocre in everything but good at nothing in particular.
If a web developer does not know the difference of a GET and a POST I would seriously suggest to evaluate their choice of profession.
But for example the last time I've had to care about a subnet mask was 11 years ago and I would not ask anyone I recruit as a web dev to know anything about this topic. You have to draw a line somewhere.
My bottom line in all this is that you should have at least a general understanding on the whole process, but you only need to be an expert in a few of them to be really good.
> The answers I get have taught me that most programmers don't have a basic understanding of how the internet works outside of their narrow specialty. This lack of a rudimentary mental model is one of the largest contributors to wasted time and effort in web application development.
I'm curious what specific mistakes you're thinking of that application developers make when they don't understand computer networks. In my limited experience, the rudimentary abstraction "messages are sent, they're not instant and they might not make it" has generally been sufficient, aside from DNS issues. I learned some networking in college (CS241 at UIUC) and it was really interesting, but I feel I've forgotten almost everything from lack of use. Maybe I've done terrible things as a result. I think I and many others could really benefit from a list of common pitfalls by networking newbs.
Can’t speak for OP, but there are a lot of things people do out there, not just web apps.
People building games, robots, trading systems, distributed databases, CDNs, etc could be really limited in their capability if they didn’t grok networking concepts on a deeper level. It also tends to tie in heavily to other systems concepts (things like zero-copy, or even basic elements like ring buffers).
I think you might also underestimate just how little knowledge a lot of people have. You went to a legit uni and took a legit course.
The interview question OP asks is an old favourite I used to ask too. A lot of people answered something like “clicky-linky-page-rendery!” So a bit of education on the many layers below that doesn’t hurt.
A lot of engineers have only ever written or interacted with server side code using heavy abstractions offered by web frameworks. Stuff like different request methods, response codes, headers and more should be basic knowledge but is not.
I once blew the minds of a room full of senior engineers by setting up a server which responds with the headers and different chunks of the body at different times, and so the content appears sequentially in the browser with a delay. They literally did not know you could do that.
Off the top of my head, not understanding the implications of the 2 Generals Problem is a really big one. Specifically, time and again, I've seen engineers struggle with distributed sagas (or any kind of distributed coordination really), because they don't know enough about the underlying model to reason about it.
There are also simpler issues. Not knowing the difference between GET and POST, not understanding what a header is, how parameters are passed etc...
Then there are the problems in the middle like not knowing how to correctly configure an HTTP client library.
The purpose of writing the book though is to help people develop a good mental model of the system, so that they don't need to memorize a list of all the things that can go wrong.
Adding a few examples to the intro wouldn't be a bad idea though!
I had multi-day bugs caused by people not knowing what "0.0.0.0" IP address is, or network address in IPv4 in general, and using that expecting to be connected to localhost (turns out, when you tell system to connect to any address, there's no guarantee you connect to 127.0.0.1)
Learning to use zap proxy, curl or burpsuite community has a profound effect on my understanding of L5/6/7 (ish) communications and how http works at a high-ish level.
"Alice send the following marbles: Red, Green, Green, Red, Green, Red, Red–No, Yes, Yes, No, Yes, No, No. This translates to: nothing there, rock ahead, turn right, nothing there, rock ahead, turn left, nothing there."
Fantastic. Thanks for the effort and for making it accessible to many.
A friend and I are writing a book on Low-level programming. We are still working on Part 1. The plan is to make it available (along with programming exercises) for free so anyone interested can benefit.
I would love to hear about your plans/experiences on how to spread the word that such material exists for free. Ideas for ensuring the material is found by maximum number of people and benefits them.
This probably isn't what you wanted to hear, but the only thing I've done so far in the way of marketing is post here, on Twitter, and Facebook. I have an email signup at the bottom of the page that has been pretty successful, but I think nearly all of it is from here.
Potentially, once I cover the more basic topics. Part 1 will cover essentially a simplified Ethernet network (plus a few additions). Part 2 will cover the internet as it existed about 20 years ago. And part 3 will cover modern additions.
I’ve done some IOT work so it is something that interests me.
I did things like Nyquist/Shannon, things like Manchester encoding and stop bits, etc 20 years ago in an electronics module at university. Barely remember anything about them, and haven't used them at all. Some colleagues might as they deal with satellite clearance and uplinks, but I'm just nice simple IP, and it's not going to help me with my (presumably) BFD issue which is causing my BGP routes to one site to fail over in 30 seconds rather than a few milliseconds.
It's insanely rare in my industry to encounter anything that isn't ethernet, hell even ethernet with MTUs below 1500 are fairly rare outside of VPN provisions. Sure there are some, especially in the high end areas, but it's so rare and tends to eventually present as ethernet.
Far more use for almost everyone is an understanding of Ethernet, IP(4+6), TCP/UDP/ICMP and how routing works (the concepts more than the specifics - things like how traffic can route asymetrically, how packets can be sent on different routes, etc). You could include other protocols like LLDP and arp (maybe LACP as part of the different routes), mention about how a traceroute will hide information when routers either don't decrement MTUs, or just stick the entire packet in an MPLS packet and pass it through the network, mention how just because your traceroute shows 20% loss at hop 7 it doesn't mean hop 7 is bad, and again how the ttl expired response may be dealt with with a low priority, or travel on a completely separate network, etc.
I would think all of that is far more useful than knowing that you need Manchester to keep your clock or how you use PSK to increase the symbols through your medium
Wow really neat, nice work! I also love the "read on the web for free, buy a print/pdf copy" model. I rarely buy books now without being able to sample them first. I like your writing style, and I think it's well on the way to being a great book.
It might be a little early to start spreading the word though, with only 3 chapters ready so far. Hopefully when it's closer to being done you'll be able to get some attention from HN
Well done Seth. I hope you see it through. Writing in this area goes
stale suprisingly quickly so fresh takes are always welcome. It looks
like this could be pitched at 400 undergrad ""basic coms and nets"
(what's missing is a great text at 300 foundation level that doesn't
need too much math). You can still sell a lot of print books with the
pdf free online, so I hope that works out well for you.
Thank you! It's been fairly burnout resistant so far because I've been able to switch back and forth between writing, drawing, and building a custom static generator.
>It looks like this could be pitched at 400 undergrad ""basic coms and nets"
400 level undergrad is about the level of detail I'm aiming for (eventually). I've been using one of my old undergrad networking textbooks to do sanity checks.
>You can still sell a lot of print books with the pdf free online, so I hope that works out well for you.
Early in my career, i had come across the classic Internet System Handbook by Marshall Rose and Daniel Lynch which gives an overview of the Internet and its Protocols circa early 1990s. It is still relevant for the basics.
I wish somebody would write something like the above for the "Modern" Internet.
What would set this apart from something like Networking a Top-Down Approach? I read most of it and thought it went into great detail explaining the basics to the general audience. It also doesn't require any programming, well mostly!
It essentially takes the opposite approach to Networking a Top-Down Approach. I start out from a state where computers, and computer networks don't exist and build up from that point. Whereas A Top-Down Approach works backwards from the application layer.
>I read most of it and thought it went into great detail explaining the basics to the general audience.
Thank you!
>It also doesn't require any programming, well mostly!
Yep, I did have to throw in a bit of pseudocode for some of the microcontroller section.
Thanks for creating this content, this looks pretty good and it was a good time for me to refresh my basics about computer networks.
I tried checking the content with my kindle (a Kindle Touch paperwhite 6 inches) and the first chapter rendered pretty good, with images and text aligning as intended. However the second chapter did not render the first figure and beyond. Chapter 3 rendered OK, similar to Ch 1.
A few more feedback if that helps for enhancing the format even more:
- Colors, and references to those do not work on a kindle. I only saw one minor use of colors (red, green) in Chapter 1 so that's not a blocker.
- For some reason, the kindle browser doesn't put left and right margins on the page, so the text uses the full length of the device.
- Links are rendered with a pale gray, sometimes hard to distinguish.
- Figuring out why the Chapter 2 did not render would definitely help.
If you're just starting in programming you may thing that Computer Networks falls into a nice to know category. At least I did as a freshman in CS. At a time I was more interested in gaining practical knowledge: popular programming languages, frameworks, OOP. In fact you can become a successful programmer without knowing much how networks work.
The reason why I recommend learning about Computer Networks from scratch is how fascinating the subject is. The way layers are implemented, their interactions, how the overall complexity is reduced by having layers of responsibility. This pattern can be seen in other areas as well: hardware -> operating system -> SDK -> programming language -> framework -> DSL. Ultimately from a Turing complete machine to seeing 200 FPS 3D FPS game with 150 other players on your screen.
I really like the idea of this. I look forward to reading what's finished now, as well as what will be released in the future!
I have a huge gap in knowledge for so many things that I interact with on a regular basis, but that I don't actually need to understand well in order to meet minimum job expectations.
This seems like a good book to address network basics. Sadly, I'm a huge procrastinator, but this seems especially important for me to address.
Books that have been in my backlog for years: Database Design for Mere Mortals, Designing Data-Intensive Applications, Building Secure and Reliable Systems, Writing a Compiler in Go, Crafting Interpreters.
All books that I know I need to read, but I just can't get around to it because there's no immediate reward for me, as someone who doesn't have to worry about any of these days in my day-to-day duties.
Thank you! Your backlog sounds like a fantastic list as well. I can wholeheartedly recommend Designing Data-Intensive Applications. I’m reading it a 2nd time for a book club at work, and I think it’s one of the best overviews of distributed systems out there.
When sailing through the computer network topic, my great companion was Tanenbaum's and Richard Stevens's book. Although, I have to be honest that if I only read that book without exposure to the real-world problem, I might quit computer networks many years ago.
My two cents for the rest of the chapters would be: to make sure to provide a real-world problem and example in which anyone could have access to them. In my case, I had the privilege to build a real network (10k users) with FreeBSD as the router, proxy, DNS, mail, etc. With all the tooling available today, one can easily acquire these swiss army knives for networking. Sure, tooling comes and goes, but at least it can provide better abstractions to the students. At least it worked for me.
What a strange world. 621 points on Hacker News but the thing doesnt actually exist. I'm thinking about submitting a new post soon 'Show HN: I am writing An effective micro transaction platform to eliminate ads from the internet.'.
The idea and few samples exist. If you can communicate it well enough, people actually have the ability to anticipate - at least partly - how it'll look like when it's done.
Not sure what your prior experience is in this matter, but here are my general thoughts on book writing.
I have written, or been involved in the writing of, a few books. My e pertain every was that it is easier to start a book from the TOC.
I am assuming you will be using LaTex or something similar? Writing a book in anything other than a structured writing environment can be hell. Certainly avoid ms word like a cockroach should avoid a hammer.
If you're looking for ideas on what to write about I'd love to learn more about networking on the ISP side of things. I know about routers, TCP/IP, sockets, and so on, but my knowledge of the complete topology of the Internet is quite limited. I feel like if you were to fit that all together it would be an incredibly valuable resource.
Nice work - so far this looks like a fun way to learn about how computers communicate with one another.
If I'm not mistaken, it looks like there's a typo under the section "Binary is better?": it reads "...then R is also number 5" when I think it should read "...then F is also number 5".
Awesome! Thanks for reporting the first typo! I'm currently deciding between building a report errata feature, or just making it public on GitHub and accepting PRs.
That's not a bad idea. Its already on GitHub (I showed my wife how to use git because she's doing the proofreading), so it would just require me making it public.
Thanks for what you put together! Do you mind sharing who did the doodles? I'm working on my own free web book and I've been hoping to include illustrations like these.
Thanks! I started Feb 1st when I took paternity leave. My wife was already back at work by then, so I mostly worked on it while the baby was napping. I've been working on it in my spare time in the evening since going back to work.
Question : Why would people (these days) be motivated to spend time on reading about a subject slowly as opposed to trying to read short targeted content (blogs/tutorials) quickly and jump into the practical bits?
May be it's just me and the kind of people I have been meeting who do not seem to have as much patience and time and would prefer getting hands-on experience quickly.
I can see how I might sound complaining about going through ~30 pages of information whereas full books with several hundred pages take much longer to read and this is relatively much shorter.
People read books because grabbing bits and pieces from blogs and tutorials often means that you end up with holes in your understanding that you don't know are there. Books are also a nice place to start when you don't even know what blog posts to read.
Tutorials are great if you want to learn how to use library X. I prefer books if I want to understand the problem that library X solves. Of course that's a sweeping generalization.
I'm not sure that I'm qualified to write that book. But I do have an interest in game engines, so something with networking might be the perfect tangent for me one day.
Wild guess: Due to overlap with previously flagged activity (the specific IPs currently pointed to by Cloudflare, or their system overly broadly marked gcp-us-west1-1.origin.onrender.com.cdn.cloudflare.net)
I really like how in depth you are going on foundational Layer 1 material so far. Example, I was so happy to see you discussing transceivers. I feel like mosts texts just give a cursory treatment to layer 1 and its hardware. I hope the next few chapters continue on with this layer 1 and hardware before moving up to Layer 2. I think it makes your book quite unique so far. Thanks for sharing.
That's interesting. I've given a talk various times over the years on "how the internet works", and when I later read Code I also noted the similarity in style. So no, not directly, but we apparently just have similar styles (not that I would place my book anywhere near Code).
This book looks really promising. Just a small recommendation, you should give an option to the reader to download the book, if they give there email address. This would increase your newsletter size and hence, it can go long way.
It takes a surprising amount of insight to remove all the extraneous information from a drawing, bringing it to its essence. That's what makes your drawings, although appearing simple, so good.
P.S. Scott Adams has talked about using the same technique for the Dilbert comics.
That's high praise, thank you! To some extent the drawings have started guiding the writing when I originally thought it would work the other way around.
I see you are going for the Thing Explainer style a la XKCD. That only works for a subset of learners.
A major hurdle is that you jump right in to “imaginary devices”. And these imaginary devices are rolling marbles of all things.
Perhaps a better way would be a historical look at how communication evolved. Smoke Signals, Flags on Ships, Telegraph, Radio, Switching Telephone, Packet Telephones
These are concrete things people can see and grok.
See “Don’t make me think” book. The essence is that “imaginary devices” is super effortful for the reader and hampers the gradual learning for beginners to acquire the knowledge
Ethernet, and 802.11, are both based on the concept of a singular channel shared by many individuals. When Alice talks, she says 'I'm Alice calling Bob, can you hear me Over'.
Alice is the source address. Bob is the destination address. Over is the stop code/stop bit signifying the end of frame. The payload is 'can you hear me'.
-------
This is everything up to layer 3. But if you wanted to send a radio/telegram across the country, you need one more layer, IP4 or the ARRL protocol (American Radio Relay League meets on the air at predestinated times to pass messages to each other on Ham radio frequencies. It's a human network)
I'm Alice, calling the Eastern Operators. I'd like you to relay a message to Bob for me. The message is 'Can you Hear me', Over.
This is a radio frame like all others, but an additional layer has been added. The radio operator has its own protocol built on top of the Frames.
------
Dave may overhear all of this chatter, but Dave knows to stay off the channel (otherwise he will cause interference). Only if Dave is addressed specifically would Dave respond (Ex: This is Alice, calling Dave. How are you doing? Over)
We can see that the operator is itself just another person on the network. The operator may have a message from the rest of the world.
This is the Eastern Operator calling Alice. I'm relaying a message from Bob. The message is 'I got your message Alice!'. Over.
This is how a router may take a message from the greater internet and pass it to your local network.