
ServiceFabric: a distributed platform for building microservices in the cloud - deegles
https://blog.acolyer.org/2018/06/05/servicefabric-a-distributed-platform-for-building-microservices-in-the-cloud/
======
SBArbeit
Lots of developers today just assume that microservices = containers, and if
you're already in containers, you must need Kubernetes, but, really, Service
Fabric is years ahead of Kubernetes in lots of ways. If I were starting a
major distributed systems project today that required serious uptime and
scalability, I'd start on Service Fabric.

And, yes, I know... whatever feature I could name from Service Fabric has some
sort of open-source thing for Kubernetes from someone that
probably/almost/kind-of/sort-of/some-assembly-required does the same thing as
Service Fabric. Or I could just use Service Fabric, and not play the
Kubernetes equivalent of "what's in your node_modules directory?"

Microsoft has an existential-level bet on Service Fabric. It's 1) not going
anywhere, 2) only getting better, 3) is already a better container
orchestrator than Kubernetes, if that's all you want to use it for.

~~~
zamalek
We have stuff running on SF in prod, using the SDK side of it and not the
container-like capabilities. I have some counterpoints (but my team is also
divided on SF).

First-off, you mentioned micro-services. The mechanisms that support service
and application versioning seem well-intentioned and well-designed. In many
ways the mechanisms run counter to some of the key benefits of a micro-service
architecture (especially surrounding decoupling development teams).

If your latency is high, deployment times out. We had to use a remote desktop
session to a VM near the cluster to deploy manually (solved with CI/CD). This
impedes teams who are trying it out, when they are remote.

It has a pretty hostile development experience. Before you can even open a
project in Visual Studio, you need the entire SDK installed. "Which SF
installer did you use?" was a frequent question in Slack, because there are a
bunch of places where it resides.

Whenever you add a project to your solution, you have to ensure that it
targets x64 (not AnyCpu). Strange compilation errors abound if you don't. We
get it Microsoft: the runtime only supports x64 - you don't have to enforce it
to this degree. AnyCpu runs just fine on x64. It sounds like nitpicking, but
it gets horribly frustrating.

They do (or used to do) strange things their .targets files. It doesn't behave
like the C# build system, with a very clear happy-path. I spent about a week
working around $(MSBuildProjectDir) not working. I would fix this by making
the entire thing nothing more than a Nuget package. Anyone can understand
having to have an emulator installed to debug, but the project system is
_hostile._

It's a big pity because the tech is amazing. I've personally started
championing Orleans+Docker as a more friendly alternative, but as I said:
others on my team adore it.

~~~
rraghur
We had a similar experience as well.. in our case, tried hard to like SF, but
eventually moved out. The Dev experience is sub par and even with the one node
cluster, things take a long time to come up. Also, the tooling that pushes for
services to be in a single application means that boot up times are impacted
negatively. SF's application model means that you can't just run it outside.
One of the lead devs had previously got burnt with reliable collections (no
way to have it write to an attached disk.. will only write to VM local disk..
reported to ms) and risk potential disaster if you have a cluster loss.

Eventually ported to .net core and Orleans 2 on k8s and it's been a more
pleasant experience.

~~~
algorithmsRcool
I'll add another voice on this exact same situation.

Trying to get .net core app running on SF was extremely frustrating and
confusing. K8s was a MUCH more natural experience.

I was really disappointed in the tooling. The sfproj files were this
incomprehensible mess. If the ui didn't quite work you are hosed.

I would go on but I'm on mobile

~~~
qrli
Their VS extension and libraries are always lagging behind. It seems the team
do not have enough resource on the dev tool side.

But you could use `guest executable`, which has no dependency on their
tooling. You can create exe with any language/stack.

~~~
algorithmsRcool
Yes, this is what I had to do because of .Net core. I still had some trouble
getting the cluster to deploy smoothly.

But honestly, K8s has first class support from MS, Google, AWS. Has major buy-
in from the community, good docs, plenty of recorded talks, presentations and
war stories and a very active community.

SF has basically none of that. At the end of the day, SF isn't massively
better than K8s for my use case and It would have to be to make up for all
those disadvantages.

------
wenc
It's worth noting that while Service Fabric as a technology does power a lot
of offerings at Microsoft (as noted in the article), strategically it has been
superseded somewhat by Azure Kubernetes Service (AKS). Most new development
should really be on AKS. Even Azure Container Instances (ACI), a recently
GA'ed Azure service, runs Kubernetes underneath.

Edit: As noted in the comments, SF is on Github. The following paragraph is
incorrect and I retract what I said; leaving this here for context.

 _" Also, by going the K8S route, you potentially avoid being locked into a
proprietary offering -- SF is only available from one vendor, whereas K8S is
available almost anywhere."_

~~~
manigandham
Service Fabric is different from Kubernetes. It's not a container
orchestration platform but a full-stack distributed state and processing
system with components that can serve as the foundation for building something
like Kubernetes itself.

The naming is confusing and there is a Service Mesh offering from Azure that
lets you run your own applications easily if they're using the Service Fabric
framework already, but SF can run containerized code in any language, and it
has bindings for C# and Java (for now) if you want to use the primitives
directly.

SF is also now open-source and you can run it all yourself in whatever
environment you want: [https://github.com/Microsoft/service-
fabric](https://github.com/Microsoft/service-fabric)

~~~
dewiz
SF has been opensourced very late though [1], to get some attention and trying
to survive. Legitimate but very different from being an open source project.
The cross-platform support is very weak [2][3], far from K8S, Mesos and Swarm
developer friendliness.

[1]
[https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/...](https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/14/service-
fabric-is-going-open-source)

[2] [https://docs.microsoft.com/en-us/azure/service-
fabric/servic...](https://docs.microsoft.com/en-us/azure/service-
fabric/service-fabric-get-started-mac)

[3] [https://social.msdn.microsoft.com/Forums/en-
US/b5045fe1-51a2...](https://social.msdn.microsoft.com/Forums/en-
US/b5045fe1-51a2-4c58-ae90-53c145d3ef59/using-mac-with-azure-service-fabric)

~~~
polskibus
I don't think it's buildable on Windows yet. It's been like this for a couple
of months now, not sure if MS is really trying to get others involved.

~~~
dewiz
MS is going where developers are, and outside MS there is no community around
SF. Hired one of K8S founders, built new products around k8s, advocates docker
left and right at conferences, these are clear signals that the tech doesn’t
have a future

~~~
youdontknowtho
SF can also manage docker containers. They are doing lots of work with k8s,
but this system has a huge amount of internal usage and has provably been used
to build global scale services. Its strengths are different than k8s and the
distributed system space has lots of options.

You (and others) seem to be enjoying some schadenfruede about SF not "winning"
some competition. I don't think the world works that way. Lots of people will
use it for big applications. More people will probably use something else.
That's about it.

The type of comment you made doesn't get marked by the moderators for some
reason even though it doesn't actually provide any insight. The article had
detailed information about the internals of how it worked. That's the
interesting thing here. The Ziff Davis style dev-adoption-horse race is pretty
pointless.

~~~
polskibus
I have to say, I'm evaluating some of the .net technologies for building
distributed systems, namely Akka.NET, Orleans and SF, and using any of them
requires quite a lot of investment and therefore time to learn to use them
right.

The last thing I want to experience is that after a year or two, the tech I
pick is going to be abandoned or at least put into maintenance mode.

Therefore I'd appreciate all insights (if possible data-driven) on the future
plans about SF and how does it stack against k8s investments by MS. It would
be great if MS themselves made an official announcement.

Seeing that SF doesn build on Windows for several months, makes me doubt
whether the momentum behind open sourcing SF is real.

~~~
CuriousSkeptic
Theres a good presentation from the Build conference about the future.

In summary they will add the hosted “mesh” variant. And move the SDK towards
polyglot libs for use in docker containers instead of a C# framework. The
reliable collections part of the SDK will be just a lib providing storage for
abitrarily shaped data needing replication.

They also mean to hav it all work similarity for your own clusters on-perm or
on azure.

(And some one asked, yes managed service principles and secrets is coming up
in 6.3 probably)

P.s There is also a regular q/a session over Skype they record and put on
YouTube if you want more details or ask questions directly

~~~
masnider
(I'm Matt, from SF@MSFT, the same guy from up here [1])

So about those presentations and announcements! This[2] is the main
presentation where we gave a bunch of updates and roadmap, including the
upcoming changes to the SDK and the announcement of Service Fabric Mesh.
Here's a short video that shows a demo of what that's really like[3].

So about the new programming frameworks and the changes/history here: You have
a lot of choices for frameworks. Roll your own, Spring/Steeltoe, Orleans,
Akka.NET, and the ones that SF provides, etc. For most of those though, they
are just frameworks. They help you develop services, but don't have as much to
do with how or where you run them.

...Except for the ones that came with SF, which were of course special in an
unexpected and not entirely desirable way.

Previously the SF provided frameworks (Reliable Services and Reliable Actors)
were different. They were coupled to the Service Fabric runtime environment.
You had to run them in a SF cluster, you couldn't run them anywhere else.

And managing a cluster is hard work. You had to manage all the infrastructure,
figure out the networking, figure out how you wanted to configure the drives
and storage, manage certificates, all that stuff. Managing a cluster was real
work and most of that work contributed basically 0 to how well the service
actually worked. You had to patch and secure the OS. Some of this stuff was
easier in Azure than doing it all on your own for sure. But easier still isn't
easy in a lot of situations, and it's all boilerplate.

Just as an example, since you managed the hardware, at some level, you had to
deal with scaling decisions yourself. SF could help you scale within the
cluster, but because SF tries really hard not to get too wedded to a
particular infrastructure, when you ran out of hardware, you were on your own
to figure out how to add more.

With the new SF SDKs and roadmap a lot of this is changing. Particularly for
the stuff SF provides out of the box like service discovery and routing,
lifecycle management, and the reliable collections, we've separated these
things out from the runtime. This means one of the following a) you just don't
have to deal with them anymore b) we do the common thing automatically for
you, or c) they're optional libraries you can include and that work anywhere
you choose to run the service, even outside of SF.

As part of this, we're also introducing a bunch of different resources like
routes and volumes that take a bunch of stuff that was previously much more
implicit and making it explicit and declarative. The goal of the new SDK and
resource model is that they're really going to help more types of workloads
run on top of SF, especially ones that don't want to (or can't) use our SDK,
while also making things easier to manage overall.

The goal of Service Fabric Mesh is to be an environment where Microsoft
manages the cluster for you. This takes all that infrastructure goop and hides
it, giving you a serverless, fully managed environment where you can run the
stuff defined in that resources model. So now you get the power of the
platform and some of the neat features that SF provides, and you don't have to
do all that boilerplate work. This lets you focus on whatever your app/service
was. And since the resources model is a core part of the product, you'll get a
fairly seamless transition between building things on your dev box, running
them in say some local test cluster, and running them up in Mesh in Azure.

I think it's really exciting and should help with adoption of SF since now a
lot of the stuff that people had problems with is gone, and since the
resources model and the changes to the APIs make it much easier to run
arbitrary workloads than before.

[1]:
[https://news.ycombinator.com/item?id=17257735](https://news.ycombinator.com/item?id=17257735)

[2]:
[https://www.youtube.com/watch?v=0ab2wIGMbpY](https://www.youtube.com/watch?v=0ab2wIGMbpY)

[3]:
[https://www.youtube.com/watch?v=a6GPH66i8pc&t=10m57s](https://www.youtube.com/watch?v=a6GPH66i8pc&t=10m57s)

~~~
polskibus
Thanks. Is Service Fabric Mesh going to be easy to roll out without Azure? Are
you going to provide a set of tools to set the cluster up and operate it, for
those who need a self hosted version?

Another thing - how is the Windows build tools for SF going? Is it easy to
build right now from the open source version?

~~~
masnider
Mesh is specifically the _serverless_ environment that MS runs for you in
Azure, so there's no real concept today of running Mesh somewhere else
(probably except for Azure Stack) - you have to manage the servers then, so
it's not serverless anymore :)

But the thing to keep in mind is that all the _capabilities_ that you're
thinking about are in the core product. So you could take SF and run it
standalone and tell some people "hey you run the cluster" and another set "hey
you write the services" and at least with the resource based model you have a
pretty good separation of concerns. It's not _Mesh_ because it's not
_serverless_ with MS running the cluster for you, but you can do almost all
the same stuff.

Re: the Windows build, yeah what Mikkel said. Slow and steady.

------
ossinnameonly
In my estimation, now would be a rather poor time to adopt SF.

1\. It's poorly positioned against K8s. The "it can do containers" people are
really not helping their case; SF's value proposition is clearly not in
running arbitrary containers. Unless you're writing a new .NET app and will go
all in on their SDK and replicated state model, it just doesn't make sense.

1a. It has no community. It has no presence outside of MS doc pages, the repo
they threw over the wall, and the one external user that I'm aware of (who is
already in this thread).

2\. SF was operationally not-fun. I'm being nice. Though, neither is
Kubernetes, most days.

3\. SF is losing to Kubernetes, even internally, and even despite pressure
from high-up-ish people.

4\. The rumors about SFs fate would not seem to lend themselves toward
adopting it for new projects.

~~~
tannhaeuser
I don't know the first thing about SF (the product), and you might be
absolutely right. But I find it discomforting to base engineering decisions on
the equivalent of Facebook likes and Web presences/Googling rather than merit.
For all I know, the media presence of the space of cloud orchestration
software is highly manipulated by huge PR budgets (for example by Docker) and
by media heavyweights competing in this space.

~~~
codeflo
I agree with the sentiment, but what’s sometimes overlooked is that the
community around a technology is in many ways more important than the
technology itself. Can I expect to find add-on libraries that help integrating
with the rest of my technology stack? Will I find documentation, blog posts or
StackOverflow answers that solve my specific edge case? How likely is it that
I’ll be the very first person to encounter a critical bug, because no one else
uses it at that scale?

Many of those signals are not easily faked, as Microsoft itself painfully
learned when spending billions in an attempt to artificially prop up a
developer community around Windows Phone. If, as the grandparent suggested,
Service Fabric is losing out even internally at MS where they can apply
management pressure in addition to marketing, that’s a very bad sign.

------
kerng
I like some of the metrics, e.g.

>>Azure SQL DB (100K machines, 1.82M DBs containing 3.48PB of data)

Makes you realize how different the scale of the top cloud companies is,
compared to the average service/site. Pretty amazing.

------
vshan
Very interesting read.

------
sfinternal
Not sure why this comment was banned, it has valid points worth the
conversation

—-

In my estimation, now would be a rather poor time to adopt SF.

1\. It's poorly positioned against K8s. The "it can do containers" people are
really not helping their case; SF's value proposition is clearly not in
running arbitrary containers. Unless you're writing a new .NET app and will go
all in on their SDK and replicated state model, it just doesn't make sense.

1a. It has no community. It has no presence outside of MS doc pages, the repo
they threw over the wall, and the one external user that I'm aware of (who is
already in this thread).

2\. SF was operationally not-fun. I'm being nice. Though, neither is
Kubernetes, most days.

3\. SF is losing to Kubernetes, even internally, and even despite pressure
from high-up-ish people.

4\. The rumors about SFs fate would not seem to lend themselves toward
adopting it for new projects.

------
martindale
This is pretty similar to our project, Fabric:
[https://fabric.fm](https://fabric.fm) — different tech stack, but similar
name and seemingly project goals. Should we add you to the list of "other
Fabrics" [0]?

[0]: [https://github.com/FabricLabs/fabric#other-
fabrics](https://github.com/FabricLabs/fabric#other-fabrics)

~~~
tannhaeuser
It's also not dissimilar to fabric3, a Java implementation of the Service
Component Architecture (SCA). SCA is a polyglot OASIS spec for a portable
service runtime from the height of the SOA craze, originally by David Chappell
[1].

[1]:
[http://www.davidchappell.com/writing/Introducing_SCA.pdf](http://www.davidchappell.com/writing/Introducing_SCA.pdf)

