MS is going where developers are, and outside MS there is no community around SF. Hired one of K8S founders, built new products around k8s, advocates docker left and right at conferences, these are clear signals that the tech doesn’t have a future
SF can also manage docker containers. They are doing lots of work with k8s, but this system has a huge amount of internal usage and has provably been used to build global scale services. Its strengths are different than k8s and the distributed system space has lots of options.
You (and others) seem to be enjoying some schadenfruede about SF not "winning" some competition. I don't think the world works that way. Lots of people will use it for big applications. More people will probably use something else. That's about it.
The type of comment you made doesn't get marked by the moderators for some reason even though it doesn't actually provide any insight. The article had detailed information about the internals of how it worked. That's the interesting thing here. The Ziff Davis style dev-adoption-horse race is pretty pointless.
I have to say, I'm evaluating some of the .net technologies for building distributed systems, namely Akka.NET, Orleans and SF, and using any of them requires quite a lot of investment and therefore time to learn to use them right.
The last thing I want to experience is that after a year or two, the tech I pick is going to be abandoned or at least put into maintenance mode.
Therefore I'd appreciate all insights (if possible data-driven) on the future plans about SF and how does it stack against k8s investments by MS. It would be great if MS themselves made an official announcement.
Seeing that SF doesn build on Windows for several months, makes me doubt whether the momentum behind open sourcing SF is real.
Theres a good presentation from the Build conference about the future.
In summary they will add the hosted “mesh” variant. And move the SDK towards polyglot libs for use in docker containers instead of a C# framework. The reliable collections part of the SDK will be just a lib providing storage for abitrarily shaped data needing replication.
They also mean to hav it all work similarity for your own clusters on-perm or on azure.
(And some one asked, yes managed service principles and secrets is coming up in 6.3 probably)
P.s There is also a regular q/a session over Skype they record and put on YouTube if you want more details or ask questions directly
(I'm Matt, from SF@MSFT, the same guy from up here [1])
So about those presentations and announcements! This[2] is the main presentation where we gave a bunch of updates and roadmap, including the upcoming changes to the SDK and the announcement of Service Fabric Mesh. Here's a short video that shows a demo of what that's really like[3].
So about the new programming frameworks and the changes/history here: You have a lot of choices for frameworks. Roll your own, Spring/Steeltoe, Orleans, Akka.NET, and the ones that SF provides, etc. For most of those though, they are just frameworks. They help you develop services, but don't have as much to do with how or where you run them.
...Except for the ones that came with SF, which were of course special in an unexpected and not entirely desirable way.
Previously the SF provided frameworks (Reliable Services and Reliable Actors) were different. They were coupled to the Service Fabric runtime environment. You had to run them in a SF cluster, you couldn't run them anywhere else.
And managing a cluster is hard work. You had to manage all the infrastructure, figure out the networking, figure out how you wanted to configure the drives and storage, manage certificates, all that stuff. Managing a cluster was real work and most of that work contributed basically 0 to how well the service actually worked. You had to patch and secure the OS. Some of this stuff was easier in Azure than doing it all on your own for sure. But easier still isn't easy in a lot of situations, and it's all boilerplate.
Just as an example, since you managed the hardware, at some level, you had to deal with scaling decisions yourself. SF could help you scale within the cluster, but because SF tries really hard not to get too wedded to a particular infrastructure, when you ran out of hardware, you were on your own to figure out how to add more.
With the new SF SDKs and roadmap a lot of this is changing. Particularly for the stuff SF provides out of the box like service discovery and routing, lifecycle management, and the reliable collections, we've separated these things out from the runtime. This means one of the following a) you just don't have to deal with them anymore b) we do the common thing automatically for you, or c) they're optional libraries you can include and that work anywhere you choose to run the service, even outside of SF.
As part of this, we're also introducing a bunch of different resources like routes and volumes that take a bunch of stuff that was previously much more implicit and making it explicit and declarative. The goal of the new SDK and resource model is that they're really going to help more types of workloads run on top of SF, especially ones that don't want to (or can't) use our SDK, while also making things easier to manage overall.
The goal of Service Fabric Mesh is to be an environment where Microsoft manages the cluster for you. This takes all that infrastructure goop and hides it, giving you a serverless, fully managed environment where you can run the stuff defined in that resources model. So now you get the power of the platform and some of the neat features that SF provides, and you don't have to do all that boilerplate work. This lets you focus on whatever your app/service was. And since the resources model is a core part of the product, you'll get a fairly seamless transition between building things on your dev box, running them in say some local test cluster, and running them up in Mesh in Azure.
I think it's really exciting and should help with adoption of SF since now a lot of the stuff that people had problems with is gone, and since the resources model and the changes to the APIs make it much easier to run arbitrary workloads than before.
Thanks. Is Service Fabric Mesh going to be easy to roll out without Azure? Are you going to provide a set of tools to set the cluster up and operate it, for those who need a self hosted version?
Another thing - how is the Windows build tools for SF going? Is it easy to build right now from the open source version?
Mesh is specifically the serverless environment that MS runs for you in Azure, so there's no real concept today of running Mesh somewhere else (probably except for Azure Stack) - you have to manage the servers then, so it's not serverless anymore :)
But the thing to keep in mind is that all the capabilities that you're thinking about are in the core product. So you could take SF and run it standalone and tell some people "hey you run the cluster" and another set "hey you write the services" and at least with the resource based model you have a pretty good separation of concerns. It's not Mesh because it's not serverless with MS running the cluster for you, but you can do almost all the same stuff.
Re: the Windows build, yeah what Mikkel said. Slow and steady.
(I'm Matt, from SF@MSFT, the same guy from up here [1])
To me, it makes sense that the Kubernetes services at Microsoft would be built on top of Kubernetes and expose Kubernetes to customers. That's a good way to get experience with running a service at scale, eat the dogfood, and deliver the experience people who want Kubernetes are expecting.
Just because some services pick K8s as the backend (when that's an appropriate choice!) and that Microsoft hired an expert to work on them is in no way proof that a different competing technology is dead. Plus: would you have rather we hired an amateur? :) I'm glad Brendan's here leading the charge.
The notion that promoting Docker indicates that SF doesn't have a future is also misleading - that's a container format vs. an orchestrator. Those are different layers of the stack with different jobs. Since SF has supported Docker containers for some time, we're fine with the rest of the company promoting Docker as a means of isolating workloads. Plus, before containers, isolating workloads on Windows was really tough. SF arguably made this worse by giving everyone an environment where you can run tons of different workloads SxS without giving people guidance on how to isolate things reasonably. SQL virtually had to invent their own way of doing it[2], and nobody really likes programming against Windows Job Objects (just like most people wouldn't prefer to deal with cgroups directly either).
Docker (or other) containers are a godsend for taking old leaky code and getting it running in a shared cloud environment without having to rewrite stuff. SF <3 Containers, but we also work with bare processes because, well, like Mesos, that's how we started, and also sometimes you just don't have the problems that containers solve or can't take the hit to switch to a containerized workflow at the moment.
Lots of new services get built on top of SF all the time (The video from build[3] shows the newer services built on top of SF in Azure). So in the same way, just because there are new services getting built on top of SF doesn't mean that the Kubernetes offerings at MSFT are dead either. It's a false choice.
