Hacker News new | past | comments | ask | show | jobs | submit login

So there are actually three tech revolutions going on in parallel:

1) Kuberentes 2) Microservices. 3) AI

All of them are based on open source, and all of them are permissionless (I.e. the startup does not need permission from a big company, e.g. an app store owner).

They also erode the build-in advantages of big tech:

1) Kubernetes erode cloud lock-in on compute/storage. 2) AI is best serve on the edge.

I would say that everything today is up for grab.




Kubernetes is a solution worse than the problem.

Microservices are unix pipes but slower.

AI is linear algebra that costs x1000 because of the brand name.

There is no revolution in software. We are just building crud as high and as fast as we can because the quality of developers tends to zero as the number of developers increases.


> Kubernetes is a solution worse than the problem.

Not if you're a 1,000+ engineer org with an SOA.

> Microservices are unix pipes but slower.

Humans are way more expensive than the machines running our code. You can always spend more on spinning up additional instances. In reality the overhead isn't that bad. The principles of immutable stateless deploys free up so much mental energy when it comes to thinking about how to get your code out and not break things that it more than pays for itself.

> AI is linear algebra that costs x1000 because of the brand name.

Tell that to my real time voice conversion system I just wrote. There's a lot of new magic happening right now.

> There is no revolution in software.

I'm on a fucking phone responding from hundreds of miles away. We're launching satellites that beam internet.

> We are just building crud

You can't speak for everyone here.

Are you in adtech or a social spyware company? Maybe don't work like that then.

Find meaningful work that speaks to you.

> quality of developers tends to zero as the number of developers increases.

I work with brilliant people.


>Not if you're a 1,000+ engineer org with an SOA.

That's the problem. With a sane simple architecture instead of 40 years of crap on top of crap you wouldn't need thousands of engineers to spin up instances.

>In reality the overhead isn't that bad.

Over 1e2 times if the solution is in software, over 1e5 times if it's an fpga. Around 1e8 if you do it in silicon. You make the savings from not hiring the 1000s of engineers to deploy your bloated horror.

>The principles of immutable stateless deploys

Has nothing to do with k8s. Nixos and Guix are two systems that are as close to functional as possible and they sure as hell aren't 'stateless', even though they come closer than any of the sexy trending tech.

>Tell that to my real time voice conversion system I just wrote.

Thank you project librevox for proving a near infinite training corpus, thank you nvidia for gpus to do linear algebra 1e5 more efficiently than you can on a cpu.

>I'm on a fucking phone responding from hundreds of miles away. We're launching satellites that beam internet.

Like everything else you listed there this is a hardware improvement. A Casio watch from the 90s had the same order of magnitude computational capability as the first generation IMPs. Good luck getting it connected to the internet using any software you could write for it.


> 40 years of crap on top of crap

It's all new.

> wouldn't need thousands of engineers to spin up instances

If thousands of engineers had to worry about deploys, that'd be a problem. The deploy team is small. K8S makes this dead simple.

> Over 1e2 times if the solution is in software

That's ridiculous. Where did you get those figures?

> 1e5 times if it's an fpga. Around 1e8 if you do it in silicon

If you're writing code for FGPAs or burning it to silicon, you're far removed from application server development. That would move at a glacial pace.

> bloated horror

Our containers are thin.

> Has nothing to do with k8s.

You're just hating on k8s at this point.

> Like everything else you listed there this is a hardware improvement.

Kubernetes. :P

Honestly there's been a metric ton of software improvements. Literally everywhere.

Git, modern kernels, Rust, Paxos (popularized in the 90s), LLVM, modern game engines, tmux, Wikis, bittorrent, blockchain, protobufs, gRPC, Bazel, Redis, modern PL idioms, futures, TOTP, U2F, Markdown, RSS/Atom, so many algorithms, seam carving...


> > 40 years of crap on top of crap

> It's all new.

Now I’m tempted to reply, “No wonder that 1000+ engineers can pile up crap that fast” :)


>> Kubernetes is a solution worse than the problem.

> Not if you're a 1,000+ engineer org with an SOA.

The fact that most FAANGs (Amazon especially) developed their own systems that are not Kubernetes, and often much simpler, should make you think. Many don't even use containers - by design.

(Source: direct experience)


Amazon didn't have the kernel expertise to develop kernel features required for containers like Google did, and had massive sunk costs in their bespoke hacky solution (Apollo environments).


They didn't adopt Kubernetes because it didn't exist at the time. The original paper came out, what, four years ago?

These companies had deployment needs that weren't at the time met by off the shelf open source software.


> They didn't adopt Kubernetes because it didn't exist at the time. The original paper came out, what, four years ago?

That's not what I wrote.

I wrote that the systems they came up with are much, much less bloated and provide very comparable features - if not better.


> Kubernetes is a solution worse than the problem.

I disagree. Maybe it's because we haven't gone "full Kubernetes" where every cronjob is now a Kubernetes cronjob, or what have you, but for all the complexity we've had to overcome, we've seen substantial benefits.

For example, autoscaling instances of my data pipeline apps based on Kafka lag.

Was it complex? Yes. We had to expose Kafka topic lag as a metric to Prometheus, then configure the k8s metrics server that HPAs (Horizontal Pod Autoscalers) use to scale to pull that in from Prometheus using k8s-prometheus-adapter as a custom metric, then set reasonable scaling limits in the data pipeline apps HPAs.

Was it worth it? Fuck yes. We no longer have to worry about data arriving out of time at our data warehouse, because the scaling we've configured (in YAML, god rest its dirty soul) ensures that our data will always arrive at the data warehouse within 5 minutes, which is ideal for a near real time reporting product that greatly enables our end users.

Is Kubernetes worth it? For us, definitely. For anyone else? Really depends on your use case, don't cargo cult this shit.


How much data do your "data pipeline apps" process with Kafka, Kubernetes and probably other K-named things?


Several terabytes a day, but it varies, hence why I love the auto-scaling.


That's 10tb = 10000gb = 10000/24/60/60 = .11gb/s. My 2015 desktop could handle that.

Where do you work? I'd like to pitch my radical idea of edge consolidated cloud computing.


"Big data" frameworks are very good at throwing a lot of hardware at problems. See eg the classic big data vs laptop treatment: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...


The inevitable comment lol. I'm sure it could. To clarify, when the app scales, it's pods (container instances) that are scaled, not necessarily machines,that is managed separately by k8s. Depending on how much of the cluster is currently being used.

Yep, we could handle it easily on one machine, if we gave it unfettered access to all the resource, but our throughput varies during the time of day, so dedicating a machine to it to handle the peak throughput wouldn't make financial sense at 3am at night. So it's far simpler to scale pods with known resource usage as needed.

It also helps prevent issues when throughput suddenly doubles because of a business decision that you're left out of the loop on.

So autoscaling pods is ideal for our use case.


Were talking about $5k of hardware to meet your needs 20 times over with the new threadrippers.

Is your aws bill really under $500 a month?


On AWS you would be paying for many other things besides elastic CPU power. CPU and bandwidth is infamously expensive there. (But I don't think AWS was mentioned anywhere in this thread...)


You don’t need kubernetes to do it, any cloud provider has api that will make it possible. Kube main point is declarative model, which comes with a big price (like sql had). Yaml - is error prone way to achieve anything.

Will see if history repeats itself and we will see be imperative systems.


Well, arguably that's true in the webcrap space. The HTML->CSS->JS->Webasm line of development really is crap on top of crap. Trying to fix the design problems of one layer by putting another layer on top of it only sort of works. It gets worse over time. "The mold seeps through the wallpaper" (a comment I usually make about C++ collection classes trying, but only partially succeeding, to hide the problems of C arrays).

I was doing "microservices" on QNX using MsgSend/MsgReceive for hard real time over 15 years ago. It doesn't have to suck; the mainstream is just using the wrong tools for doing it fast. The QNX people managed to shoot themselves in the foot by going closed-source -> open source -> closed source -> open source -> closed source, after which their developer community was fed up.

Machine learning is not linear algebra. It's nonlinear. That's what makes it useful.


> Well, arguably that's true in the webcrap space. (...) crap on top of crap

Not that you're wrong per say, but personally, I think you should take a look at the HN guidelines. Like don't be snarky, and don't post shallow dismissals.

Just categorizing the arguably most popular way of building software as "crap" doesn't seem like thoughtful or curious conversation.


> The HTML->CSS->JS->Webasm line of development really is crap on top of crap.

I completely get the hate for modern web, but in this case I think it's unrelated. The whole Kubernetes/microservice world exists basically to serve JSON and similar protocols. Maybe rarely some static HTML.

For HTML->CSS->JS->Webasm stuff it is more fashionable to compile at build time and service as static assets. That's the polar opposite of Kubernetes/microservices.


“ AI is linear algebra that costs x1000 because of the brand name.”

This doesn’t even make sense because it can be said about anything involving numerical work on a computer and it’s still roughly true. Also because all of the libraries are free anyway.


>AI is linear algebra that costs x1000 because of the brand name.

Sure, but we've learnt to apply linear algebra to achieving things we could only dream of 10 (or even 5) years ago.


Current AI is still mostly trash of the high school level project quality, linear algebra taken to the absolute possible level of insanity.


I'm pretty sure that no high school project can beat the world's best Go player.


^the account name that posted this comment is the single most honest thing i have seen on the internet today. well done!


I like that username... :D




Applications are open for YC Summer 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: