K8s itself doesn’t introduce any real additional network complexity, at least not vanilla k8s.
At the end of the day, K8s only takes care of scheduling containers, and provides a super basic networking proxy layer for convenience. But there’s absolutely nothing in k8s that requires you use that proxy layer, or any other network overlay.
You can easily setup pods that directly expose their ports on the node they’re running on, and have k8s services just provide the IPs of nodes running associated pods as a list. Then rely on either on clients to handle multiple addresses themselves (by picking an address at random, and failing over to another random address if needed), configure k8s DNS to provide DNS round robin, or put an NLB or something in front of it all.
Everyone uses network overlays with k8s because it makes it easy for services in k8s to talk to other services in k8s. But there’s no requirement to force all your external inbound traffic through that layer. You can just use k8s to handle nodes, and collect needed meta-data for upstream clients to connect directly to services running on nodes with nothing but the container layer between the client and the running service.
| Networking Layers | Direct connection to EC2 instance (optional load balancer). | Service VIP → kube-proxy → CNI → pod (plus optional external load balancer). |
| Load Balancing | Optional, handled by ELB/ALB or application. | Built-in via kube-proxy (iptables/IPVS) and Service. |
| IP Addressing | Static or dynamic EC2 instance IP. | Pod IPs are dynamic, abstracted by Service VIP. |
| Connection Persistence | Depends on application and OS TCP stack. | Depends on session affinity, graceful termination, and application reconnection logic. |
| Overhead | Minimal (direct TCP). | Additional latency from kube-proxy, CNI, and load balancer. |
| Resilience | Connection drops if instance fails. | Connection may drop if pod is rescheduled, but Kubernetes can reroute to new pods. |
If you read my reply again, you’ll notice that I explicitly highlight that K8s does not require the use of a CNI. There’s a reason CNIs are plugins, and not core parts of k8s.
How do you think external network traffic gets routed into a CNIs front proxy? It’s not via kube-proxy, kube-proxy isn’t designed for use in proper production systems, it’s only a stop gap to provide a functioning cluster to enable bootstrapping of a proper network management layer.
There is absolutely nothing preventing a network layer directly routing external traffic to pods, with the only translation being a basic iptable rule to enable routing of data sent to a nodes network interface with a pod IP to be accepted by the node and routed to the pod. Given it’s just basic Linux network interface bridging, happening entirely in the kernel with zero copies, the impact of this layer is practically zero.
Indeed the k8s services setup with external load balancers basically handle all of this setup for you.
There are plenty of reasons not to use k8s, but arguing that a k8s cluster must inherently introduce multiple additional network components and complexity is simply incorrect.
While Kubernetes theoretically allows for simple iptables-based routing, in practice, very few production environments stop there. Most clusters do use CNIs, kube-proxy, Service abstraction, and often external load balancers or Ingress controllers, which together form a nontrivial networking stack.
The claim that kube-proxy is “not designed for use in proper production systems” is simply incorrect. It is widely used in production and supported by Kubernetes core. While it has limitations—especially with high-connection-load environments—it is still the default in most distros, and even advanced CNIs like Calico and Cilium often interact with or replace kube-proxy functionality rather than ignore it.
If kube-proxy is just a stopgap, why do so many managed Kubernetes platforms like GKE, EKS, and AKS still ship with it?
While it’s true CNIs are plugins and not core to Kubernetes, this outsources complexity rather than eliminates it. Different CNIs (e.g., Calico, Cilium, Flannel, etc.) use different overlay models, BPF implementations, or routing approaches.
Even in a CNI with kernel-fast path routing, pod churn, rolling updates, or horizontal scaling still introduce issues. Service IPs are stable, but pod IPs are ephemeral, which means Long-lived TCP clients pinned to pod IPs.
You can design a lean Kubernetes network path with minimal abstractions, but: You lose things like dynamic service discovery, load balancing, and readiness-based traffic shifting. You must manage more infra manually (e.g., configure iptables or direct routing rules yourself). You’re fighting against the grain of what Kubernetes is designed to do.
Not entirely sure what your point is. You state that Kubernetes must come with a complex networking stack, and I point out that simply isn’t true, which apparently you agree with.
But because other people use complex networking stacks in Kubernetes, that means for a bunch of good reasons, that apparently means it can’t be used without that stack? If you don’t need that functionality, and you have different requirements, then why would you implement a k8s stack that includes all that functionality? That would be foolish.
Everyone ships with kube-proxy because it’s the simplest way to provide an unopinionated network stack that will give you a fully functional k8s cluster with expected bells and whistles like simple service discovery. But most people quickly replace it with a CNI for various reasons.
My understanding is that your application is all about managing high throughput, direct to application TCP connections. So I have no idea why you’re talking about the need for fully fledged service discovery, or long lived TCP connections pinned to a single pod (I mean, that just the nature of TCP, got nothing to do with k8s or any of its network stacks).
> You can design a lean Kubernetes network path with minimal abstractions, but: You lose things like dynamic service discovery, load balancing, and readiness-based traffic shifting. You must manage more infra manually (e.g., configure iptables or direct routing rules yourself). You’re fighting against the grain of what Kubernetes is designed to do.
If you’re building a stack without k8s, then you don’t get any of those feature either, so not sure what the fuss is about. Doesn’t mean that’s k8s core competency as a workload schedular, and its general framework for managing stateful and stateless resources doesn’t provide a lot of value. You just need to learn how to take advantage of it, rather than following some cookie cutter setup for a stack that doesn’t meet your needs.
Having worked at multiple IoT companies with many millions of connections. This is the way.
People tend to overcomplicate things with K8S. I have never once seen a massively distributed IoT system run without a TON of headache and outages with k8s. Sure, it can be done, but it requires spending 4-8x the amount of of development time and has many more outages due to random things.
It's not just the network, its also the amount of config you have to do to get a deterministic system. For IoT, you dont need as much bursting (for most workloads). Its a bunch of devices that are connected 24/7 with fairly deterministic workloads, that are usually using some type of TCP connection that is not HTTP, and trying to shove it into an HTTP paradigm costs more money and more complexity and is not reliable.
> So clearly it must have some kind of financial impact as well, if an entire city's public transport system works slower because of it.
Unlikely, most cities transport systems will run into issues with capacity long before they run into issues with ticket gate latency. No point getting people through the gates faster if they’re just gonna pile up on the platform and cause a crush hazard.
At peak hours in London, the inbound gates are often closed periodically to prevent crowding issues in major stations. If you look at normal TfL stations you’ll notice there’s normally a 2:1 ratio of infrastructure for people leaving a station vs entering. Because crowding is by far the biggest most dangerous risk in a major metro system, and also the biggest bottleneck.
When it comes to ticket gate line in a public transport system, latency and throughput are basically the same thing.
There is no practical way to increase the throughput of ticket line, without reducing latency. It’s not like you can install more gates in most stations, the station isn’t big enough, and you can’t make the gates smaller because the people aren’t small enough.
Every transit system measures and has throughput requirements that their gates need to meet. That inherently means latency requirements, because one gate can’t process multiple people in parallel!
Also western systems aren’t that slow. The videos in the article are a decade out of date and show people in London using paper tickets! And almost nobody using a contactless card or Oyster card.
In London the Oyster cards are very fast. Fast enough that a brisk walking pace the gates will be completely open by the time you reach them, assuming you put your card on the reader the moment it’s within comfortable reach.
Contactless payments on the other hand are slower. But there’s not much TfL can do there. The slower speed is entirely the result of the contactless cards taking longer to process the transaction. TfL track contactless cards latencies by bank and card manufacture. There’s a non-trivial difference in latency between different card manufactures and bank configurations. But that’s not something TfL can control themselves.
> I hope it lasts, but I'm seeing gates which have QR code readers, IC card readers,and contactless payment readers, which is obviously unsustainable.
Why is this obvious unsustainable? The IC readers and contactless payment readers are normally built on exactly the same tech, or very similar tech. And they’re pretty much always just a single reader capable handling IC cards and contactless payments locally, with back office processing to manage bookkeeping, and any needed external transaction processing.
TfL in London has been operating paper tickets, contactless and IC card for something like two decades now. The IC system is starting to show its age, but that’s only because the current stored value cards don’t have enough on-board storage to handle the continued growth of TfLs systems, and all the new regions they now operate in. But even if the IC system they have plans to migrate and merge their IC and contactless system into one system that can handle both payment types and provide proper feature parity between them.
> If you push that conditional out to the caller, now every caller of your function has to individually make sure they call it in the right way to get a guarantee of idempotency
In this situation your function is no longer idempotent, so you obviously can’t provide the guarantee. But quite frankly, if you’re having to resort to making individual functions implement state management to provide idempotency, then I suspect you’re doing something very odd, and have way too much logic happening inside a single function.
Idempotent code tends to fall into two distinct camps:
1. Code that’s inherently idempotent because the data model and operations being performed are inherently idempotent. I.e. your either performing stateless operations, or you’re performing “PUT” style operations where in the input data contains all the state the needs to be written.
2. Code that’s performing more complex business operations where you’re creating an idempotent abstraction by either performing rollbacks, or providing some kind of atomic apply abstraction that ensures partial failures don’t result in corrupted state.
For point 1, you shouldn’t be checking for order of operations, because it doesn’t matter. Everything is inherently idempotent, just perform the operations again.
For point 2, there is no simple abstraction you can apply. You need to have something record the desired operation, then ensure it either completes or fails. And once that happens, ensures that completion or failure is persistent permanently. But that kind of logic is not the kind of thing you put into a function and compose with other operations.
Consider a simple example where you're checking if a file exists, or a database object exists, and creating it if not. Imagine your filesystem or database library either doesn't have an upsert function to do this for you, or else you can't use it because you want some special behaviour for new records (like writing the current timestamp or a running total, or adding an entry to a log file, or something). I think this is a simple, common example where you would want to combine a conditional with an action. I don't think it's very "odd" or indicative of "way too much logic".
> a database object exists, and creating it if not. Imagine your filesystem or database library either doesn't have an upsert function to do this for you, or else you can't use it because you want some special behaviour for new records (like writing the current timestamp or a running total, or adding an entry to a log file, or something).
This is why databases have transactions.
> simple example where you're checking if a file exists
Personally I avoid interacting directly with the filesystem like the plague due to issues exactly like this. Working with a filesystem correctly is way harder than people think it is, and handling all the edge-cases is unbelievably difficult. If I'm building a production system where correctness is important, then I use abstractions like databases to make sure I don't have to deal with filesystem nuances myself.
Sure, I agree that a transaction should be used here (in the database example at least). But that's orthogonal to my point, or maybe even in favour of it: doesn't a transaction necessitate keeping the conditional close to the effect? It's a perfect example of what I'm trying to say, how do you make sure the conditional happens in the same transaction as the effect, while simultaneously trying to push the conditional towards the root of the code and away from the effect? Transaction boundaries are exactly the kind of thing that makes pushing up the conditionals difficult.
By pushing up the transaction boundary. The only reason why the conditional is important is because it part of a larger sequence of operations that you want to complete in an atomic fashion.
Your transaction needs to encompass all of those operations, not just parts of it.
Probably implicit in your #2, but there are two types of people in the world: people who know why you shouldn't try to write a production-grade database from scratch, and people who don't know why you shouldn't try to write a production-grade database from scratch. Neither group should try to write a production-grade database from scratch.
This is why we invented type systems. No need to examine call chains, just examine input types. The types will not only tell you what assumptions you can make, but the compiler will even tell you if you make an invalid assumption!
If instead of validating that someone has sent you a phone number in one spot and then passing along a string, you can as easily have a function construct an UncheckedPhoneNumber. You can choose to only construct VerifiedPhoneNumbers if the user has gone through a code check. Both would allow you to pluck a PhoneNumber out of them for where you need to have generic calling code.
You can use this sort of pattern to encode anything into the type system. Takes a little more upfront typing than all of those being strings, but your program will be sure of what it actually has at every point. It's pretty nice.
Yep! I have seen so much pushed into a type system that in the end there was hardly any code needed to do validation or scaffolding… to the point where it felt magical
You can enforce them (statically) by other means if you’re determined enough, eg by using lint rules which enforce type-like semantics which the type system itself doesn’t express.
This does rely on the language having a sophisticated-enough type system to be able to extract enough type information for the rules to work in the first place.
True, but there are still documented interface contracts you can program against. The compiler won’t catch violations of the non-type parts, but the requirements are still well-defined with a proper interface contract. It is a trade-off, but so is repeating the same case distinction in multiple parts of the program, or having to pass around the context needed to make the case distinction.
Same, right up until I zoomed in and waited for Safari to produce a higher resolution render.
Partially zoomed in was fine, but zooming to maximum fidelity resulted in the tab crashing (it was completely responsive until the crash). Looks like Safari does some pretty smart progressive rendering, but forcing it to render the image at full resolution (by zooming in) causes the render to get OOMed or similar.
The core of an LLM is completely deterministic. The randomness seen in LLM output is purely the result of post processing the output of the pure neural net part of the LLM, which exists explicitly to inject randomness into the generation process.
This what the “temperature” parameter of an LLM controls. Setting the temperature of an LLM to 0 effectively disables that randomness, but the result is a very boring output that’s likely to end up caught in a never ending loop of useless output.
> It is just a reference that Apple Laptop speakers have been waaay above anything the competition uses
More like the opposite. The MacBook speakers are absolutely rubbish, just like all laptop speakers (there's only so much you can do when constrained to a laptop body). The reason why MacBooks sound good is entirely god-tier signal processing which manages to extract extraordinary performance out of some decidedly very ordinary speakers.
Not sure what you are saying (or just ranting?) - MBP speaker are the opposite as in the rest of non-apple Laptops have way better sounding speakers? That is definetely not my experience at all.
If they are all rubish together, well, they are laptop speakers - and as such you have to treat them. Still there is nothing preventing some set of laptop speakers being objectively better than others.
They're saying that the physical speakers inside the MacBooks body are not what makes them sound good (and that the physical speakers are on par with other manufacturers) — it's the fancy, custom post-processing that does.
2Quote from their own link: "In the case of Apple Silicon machines, Apple has taken things one step further by including actually good speakers on most modern Macs"
Climate change doesn’t impact the physics of weather, but our understanding of the inputs to a weather system is still very incomplete. Climate change does impact the relative importance of different physical inputs into a weather system, and inherently pushes weather systems away from the most studied and well understood aspects of weather physics. That in turn degrades the accuracy of our models, as it takes time to research and understand whats changed in a weather system, and what previously unimportant inputs have become much more important.
Let's say I live in the hypothetical state of Midwestia. It gets up to 100 degrees F in the summer and down to -40 in the winter months.
Over the course of ten years, the average daily temperature goes up by 1 degree F (which is many times faster than reality).
That means that there's basically one day in the entire year that doesn't fit within the existing model, or maybe a handful if some other part of the year is cooler than normal. We see more variation between normal years than we expect to see average increase in both the short and medium terms.
Our weather models predict no more than two or three weeks into the future.
There's literally nothing about climate change that will change how weather systems interact that will occur faster than our weather models will adapt to. Our weather patterns may change drastically, but that's a fundamentally different problem.
How quickly do you think our weather models adapt? Our current models are built on top of decades of detailed weather research by tens of thousands of researchers around the world, and has involved the launch of some the largest satellites in space. All to help us try and understand how weather works.
Those models don't just "adapt", they're carefully and manually refined over time, slowly incorporating better physical models as they're developed, and better methods of collecting data as it become available. Climate change absolutely changes how weather systems interact faster than our models can adapt, primarily because we can only adapt models to changes after their accuracy degrades, and it becomes possible to start identifying potential weaknesses in the existing approaches.
You're example of seasonal temperature changes vs global average temperature increases, vastly underestimates how complicated weather models are, and how much impact changes in global climate systems (like the North Atlantic Current) can drastically impact local weather conditions. Weather models, like all models, will make assumptions about the behaviours of extremely large systems which aren't expected to change very much, or are very hard to measure in real-time. As those large systems change due to climate change, the assumptions made by weather models will grow increasingly incorrect. But that hard part is figuring our which assumption is now incorrect, not always easy to identity exactly what assumptions have been made, or accurately measure the difference between the assumption and reality.
At the end of the day, K8s only takes care of scheduling containers, and provides a super basic networking proxy layer for convenience. But there’s absolutely nothing in k8s that requires you use that proxy layer, or any other network overlay.
You can easily setup pods that directly expose their ports on the node they’re running on, and have k8s services just provide the IPs of nodes running associated pods as a list. Then rely on either on clients to handle multiple addresses themselves (by picking an address at random, and failing over to another random address if needed), configure k8s DNS to provide DNS round robin, or put an NLB or something in front of it all.
Everyone uses network overlays with k8s because it makes it easy for services in k8s to talk to other services in k8s. But there’s no requirement to force all your external inbound traffic through that layer. You can just use k8s to handle nodes, and collect needed meta-data for upstream clients to connect directly to services running on nodes with nothing but the container layer between the client and the running service.
reply