The way the NATs (network address translators) are sharing the scarce public IPv4 addresses is by multiplexing on the transport level fields (ports in case of TCP/UDP and IDs or inner packet transport level fields in case of ICMP).
Since they are unaware of your protocol, they get into a “special case mode”, which on a naive translator might consume a whole IP address (so you would really make a network admin with a few of those, because you exhaust all their available addresses :-) ; but on the carrier grade NAT there are safeguards against it and the packets are just dropped.
At some point it seems like we just need to start passing laws requiring (major) internet services to provide all services on IPv6. Routers could then intercept ipv4 only devices and their dns locally and translate it to ipv6.
Still lame that in 2024 major services like Steam and Quest basically require IPv4.
I want to be able to use the internet without silly things like exhausting a network admins ipv4s.
At least cell phone networks through their standards processes have pushed most consumer networks and consumer hardware to be IPv6 by default ("by only option" in many cases with DNS64 and NAT64 filling in the gaps).
The real pressure we need are to corporate networks. Too many of them think they can use 10.0.0.0/8 forever. Too many of them own giant chunks of public IPv4 space and think they are immune to the address exhaustion. At least the prices for IPv4 addresses are going up at major clouds like AWS and Hetzner. But it still seems too slow of a price rise to hit enough bottom lines that major corporations are feeling the pressure yet.
This is outside my area of expertise.. so naiive question.. but ports aren't tied to the protocol .. right? If you open a raw socket, it's still on some associated port number. NAT traversal multiplexes ports.. so why would that preclude using any arbitrary protocol?
Ports are very much a concept of the transport layer. They are a very useful concept, so they are used in all major transport-layer protocols, but they are not necessary in a theoretical sense (though a transport layer protocol without them would only allow a single stream of traffic between any two machines, or at least IPs). But TCP port 22 is a completely different thing than UDP port 22, and they are both completely different from SCTP port 22. To prevent confusion, IANA typically assigns the same port number for a protocol on both TCP and UDP (e.g. DNS over TCP uses TCP port 53, just like DNS over UDP uses UDP port 53; and QUIC uses UDP port 443, just like HTTPS uses TCP port 443).
When a machine receives a packet, after the Ethernet and IP layers have made sure the packet is addressed to this machine, the very next thing that happens is checking what transport layer implementation should receive the packet - and this is done based on the "transport" bits in the IP header. If the packet is, say, TCP, then the TCP layer starts reading its own header and finds out the port number, and proceeds from there.
You are partially right, though. The OS network stack does expose and handle ports if you use a protocol that has them.
Networks are built in layers. There's a physical layer underneath IP, then there's IP, and then there's TCP and UDP on top of IP.
The OS network stack has components that handle all of these layers. That's why it's called a stack.
Port numbers are part of the individual protocol (TCP or UDP) because there are a lot of things you can do with networking, and port numbers don't necessarily make sense with all of them.
For example, when you ping another computer, that uses ICMP, and there is no need for ports with ICMP. You're pinging the whole computer, not trying to connect with one of several applications running on it. So ports are not really needed.
TCP/IP is a protocol called TCP encapsulated in protocol called IP. Raw IP packets have "protocol number" for use by payload, but not port number. Port number is technically part of custom data that IP layer should not care about.
If you mean socket as in `my_socket = socket(AF_INET, SOCK_STREAM, 0);` that's TCP/IP, not raw IP, so it will have port numbers in the TCP part of the packet. `SOCK_STREAM` and `SOCK_DGRAM` respectively correspond to TCP and UDP. Raw IP sockets on Linux can be created by `socket(AF_INET, SOCK_RAW, protocol);` and that will have no TCP/UDP header attached by the Kernel after the IP header in the packet.
IP has no ports, it has a protocol field, TCP, UDP, IPsec and other layer 4 protocols have their own protocol number. NAT (PAT really) uses TCP and UDP ports to connect what can be considered separate connections, ie a connection on the inside and a connection on the outside, these connections don't even have the same source/destination ports as connection state tracking is what connects these separate connections.
The socket API has different types of sockets, typically you use stream (TCP) or dgram (UDP), but you can also get a completely raw socket where you need to construct your own ethernet and IP headers, or create your own replacement of IP protocol. So socket does not mean only TCP/UDP or something with ports, unix sockets are another example of this.
When using a packet socket you are sending bytes directly to the device driver. The only ports at this level are the physical ports on your machine which will have names like "eth1" etc. Assuming an Ethernet driver, the bytes must be a valid Ethernet frame[0], but that's all. The payload is just bytes.
I've been trying for a few minutes to figure out what you mean by "it's in the name"... What about Internet Protocol implies that ports are not an inherent property of it?
All I can think of is that an "IP address" does not have a port component and that's all that IP deals with.
IPv6 alone won't help if there are some firewalls sitting between the endpoints that will drop anything they don't know. Having two linux hosts without firewalls talk to each other over IPv6 without consumer grade routers (which tend to come with firewalls by default) between them might work.
This gives you a raw Ethernet socket. For a raw IP socket, use AF_INET,SOCK_RAW,0.
A raw Ethernet socket sends packets at the Ethernet level while a raw IP socket sends packets at the IP level. If you want to play on your local network you want Ethernet (maybe). If you want to send weird packets across the Internet you probably want IP, so that you don't have to waste effort doing route lookups and MAC address lookups and your code will still work on non-Ethernet networks, including VPNs.
There are some protocols that run on top of Ethernet but Internet-compatible protocols all run on IP by definition.
This comment was delayed several hours due to HN rate limiting.
Yeah, this was intended to eliminate the problems with protocol=0, but yeah you need to implement IP and maybe ARP to do a similar thing at this level. Thankfully IP is pretty simple to implement.
If you want to play with packet sockets you might find my notebook useful where I was going to implement TCP/IP from scratch (well, from layer 2 up): https://github.com/georgek/notebooks/blob/master/internet.ip... I did ICMP over IP but got bored before I got to TCP, though (it's way more complicated than IP). You could drop in your custom protocol in place of ICMP.
