
Let's code a TCP/IP stack, 1: Ethernet & ARP (2016) - maastaar
http://www.saminiir.com/lets-code-tcp-ip-stack-1-ethernet-arp/
======
jacob019
I use SLiRP over SSH as a poor man's VPN. It's ancient. The bulk of the code
is a userspace TCP stack that performs NAT. I can use any SSH host as full VPN
with only basic user privileges. I spent quite a bit of time poking around the
code to try to improve performance. Increasing the receive window was
sufficient to achieve >1Mb/s over a high latency link, but I can't go much
higher without <100ms latency. It turns out that SLiRP never implemented
window scaling, it was unnecessary for the links back then. The code has been
reused for virtualization applications like VirtualBox, etc; but window
scaling was never needed for that since client-host latency is basically zero
there. Digging through old network code gives an appreciation for how far
we've come.

~~~
plasticchris
Sounds like sshuttle! Low bandwidth as you said but very useful, I like to use
it to vpn to my home where the gateway router is an ssh server.

~~~
jacob019
sshuttle is great too! Great for road-warrior applications where a basic SOCKS
proxy won't do, but last time I tried it I found it unstable under heavy load
and the iptables routing to a user process can be somewhat restrictive, still
super cool though. I don't know why but network tunneling is just fun.

------
i_don_t_know
Operating System Design: Internet Working With Xinu by Douglas Comer (1987) is
a nice explanation with lots of C code of how to add a networking stack to an
operating system, in this case his educational unix-inspired XINU.

~~~
bogomipz
I actually can't find the book you are referencing. I do see:

"Operating System Design: The Xinu Approach" and also the books
"Internetworking with TCP /IPvolumes1-3." Might you have a link to the title
you are referencing here? I have read the Internetworking series which is
excellent.

~~~
bblough
This seems to be the book - [https://www.amazon.com/Operating-System-Design-
Vol-Internetw...](https://www.amazon.com/Operating-System-Design-Vol-
Internetworking/dp/013637414X)

It looks the the second volume that goes with "Operating System Design".

I can't be 100% certain, as I didn't find a table of content for either of the
1st edition books, but the 2nd edition of "Operating System Design" includes a
section in implementing ethernet, so perhaps the two volumes got combined into
one for the second edition.

~~~
i_don_t_know
That's indeed the book I was referring to. Thank you for clarifying.

------
pjmlp
For those that rather learn how to implement it in a safer language, Fuchsia's
TPC/IP stack is written in Go.

[https://fuchsia.googlesource.com/third_party/netstack/](https://fuchsia.googlesource.com/third_party/netstack/)

~~~
legulere
Isn’t garbage collection going to introduce jitter?

~~~
pjmlp
Not necessarily, there are several OSes written in GC enabled systems
programming languages.

Just because a language has a GC it doesn't mean it is the only means to
allocate memory.

Go also allows for global statics, stack and plain old C style manual
allocations. It is a matter to learn how to use them.

And make use of profilers as well.

For example, on performance critical paths always use a standard for loop,
never a for range one.

~~~
TickleSteve
GC will always introduce jitter.

You can choose not to use it and allocate statically etc, but that does not
mean that use of a GC does not introduce jitter.

Also don't confuse high performance with latency-sensitive. Highly
performance-optimised code will still have issues with GC jitter if you or
someone else is causing it.

There is a reason why the GC-enabled OSs aren't used for anything in reality.

BTW: There should be no difference between for and for-range loops given an
optimiser thats working.

~~~
pjmlp
> GC will always introduce jitter.

> You can choose not to use it and allocate statically etc, but that does not
> mean that use of a GC does not introduce jitter.

It only introduces jitter if it runs at all.

If one really wants to be drastic in performance critical code, in most GC
runtimes it is possible to just turn it off.

On Go's case a _runtime.SetGCPercent(-1)_ will take care of that.

> There is a reason why the GC-enabled OSs aren't used for anything in
> reality.

And me thinking I had two running on my phones, go figure.

~~~
dullgiulio
And the default is 100, which means a double in newly allocated memory space.
I have written several servers that after starting up (and running uselessly
the GC once), never need to run it again.

------
algorithmsRcool
Not directly related to the article, but there is an experimental effort to
develop a userland TCP stack in .NET right now.

It's interesting to see how this looks in a higher level language.

[https://github.com/ProjectMagma/Magma](https://github.com/ProjectMagma/Magma)

~~~
pjmlp
In addition to that, here are some other examples.

Go in Fuchsia,

[https://fuchsia.googlesource.com/third_party/netstack/](https://fuchsia.googlesource.com/third_party/netstack/)

Oberon in Oberon (network stack but not TCP/IP though)

[https://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources...](https://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources/SCC.Mod.txt)

[http://www.projectoberon.com/](http://www.projectoberon.com/)

Active Oberon in A2 - BlueBottle OS (TCP/IP stack)

[https://github.com/btreut/a2/blob/48dcfc1f1a6ed2bec110ca3af3...](https://github.com/btreut/a2/blob/48dcfc1f1a6ed2bec110ca3af3674657d4229d16/source/TCP.Mod)

[https://github.com/btreut/a2](https://github.com/btreut/a2)

Mesa at Xerox PARC (Courier RPC, XNX)

[http://www.bitsavers.org/pdf/xerox/mesa/3.0_1977/listing/](http://www.bitsavers.org/pdf/xerox/mesa/3.0_1977/listing/)

Sing# on Singularity

[https://archive.codeplex.com/?p=singularity](https://archive.codeplex.com/?p=singularity)

Start at sourceCode\sourceCode\base\Libraries\System.Net\Sockets

~~~
dannas
Mirage OS written in ocaml has a tcp/ip stack as well. The marshal and
unmarshal code is very succint compared to C implementations.

~~~
pjmlp
Thanks, forgot about that one, it is even part of Docker for macOS.

[https://blog.docker.com/2016/05/docker-unikernels-open-
sourc...](https://blog.docker.com/2016/05/docker-unikernels-open-source/)

------
0x00000000
Does anyone know why you would use an unsigned char array to store the MAC
address instead of uint8_t? Char is one byte but a but a byte is not
guaranteed to be 8 bits yet MAC is defined as 48 bits

~~~
jws
Unless you are adding TCP to your CDC6600 or PDP-8, bytes are 8 bits. Any
suggestion to the contrary in standards is fantasy.

That said, since the next field is a uint16_t, you may as well stay in Rome
and call them uint8_t. _short_ really is variable on some live architectures.

(Ok, or some DSPs only address words larger than 8 bits, but there's no reason
for your compiler not to pick up the slack.)

Also, TIL that __attribute__((packed)) assumes the struct can be malevolently
aligned and for some architectures generates very large and slow code to
handle that. If you must pack, also add the ",aligned(2)" or whatever you can
get away with to mitigate this.

~~~
xxpor
I don't see any reason in modern C to use int/short/long over explicit
int32_t/int8_t/int64_t other than they're shorter to type.

~~~
karolist
Where can I read tips like that about modern C, would you have a book to
recommend or a blog to follow? Thank you

~~~
macintux
I can't speak to the utility of either, but a couple of resources I've found
recently:

[http://icube-icps.unistra.fr/img_auth.php/d/db/ModernC.pdf](http://icube-
icps.unistra.fr/img_auth.php/d/db/ModernC.pdf)

[http://shop.oreilly.com/product/0636920033677.do](http://shop.oreilly.com/product/0636920033677.do)

------
panic
In case you didn't realize it at first (I didn't until I happened to click on
"/home"), this is the first of a 5-part series:

1\. Ethernet & ARP (this post)

2\. IPv4 & ICMPv4: [http://www.saminiir.com/lets-code-tcp-ip-
stack-2-ipv4-icmpv4...](http://www.saminiir.com/lets-code-tcp-ip-
stack-2-ipv4-icmpv4/)

3\. TCP Basics & Handshake: [http://www.saminiir.com/lets-code-tcp-ip-
stack-3-tcp-handsha...](http://www.saminiir.com/lets-code-tcp-ip-stack-3-tcp-
handshake/)

4\. TCP Data Flow & Socket API: [http://www.saminiir.com/lets-code-tcp-ip-
stack-4-tcp-data-fl...](http://www.saminiir.com/lets-code-tcp-ip-stack-4-tcp-
data-flow-socket-api/)

5\. TCP Retransmission: [http://www.saminiir.com/lets-code-tcp-ip-stack-5-tcp-
retrans...](http://www.saminiir.com/lets-code-tcp-ip-stack-5-tcp-
retransmission/)

------
jacksmith21006
Written three TCP/IP stacks and the first was pretty bad as wrote off of RFCs
before the Comer books.

But then purchased the Comer books and made it so much easier. Highly
recommend buying the Comer books if really interested in writing a TCP/IP
stack.

Also if you really want to learning something you write an implementation. To
this day makes it so much easier to deal with IP problems, configuration,
buying products, etc.

[https://www.amazon.com/Internetworking-TCP-IP-
One-6th/dp/013...](https://www.amazon.com/Internetworking-TCP-IP-
One-6th/dp/013608530X)

[https://www.amazon.com/Internetworking-TCP-Vol-
Implementatio...](https://www.amazon.com/Internetworking-TCP-Vol-
Implementation-Internals/dp/0139738436)

[https://www.amazon.com/Internetworking-TCP-Vol-III-Client-
Se...](https://www.amazon.com/Internetworking-TCP-Vol-III-Client-
Server/dp/0130320714)

