
DNS for Service Discovery in HAProxy - phil21
https://www.haproxy.com/blog/dns-service-discovery-haproxy/
======
dcow
I've been exploring DNS PTR/SRV records for seamless service discovery
regardless of whether you have access to an authoritative DNS server
configured for a given local domain or not (so things simply devolve to
whether or not use use multicast or unicast to resolve SRV records). Basically
DNS-SD. The idea of using SRV records in general seems foreign even in the ops
realm. It's awesome to see HAProxy beginning to familiarize people with the
concept of SRV records.

~~~
jlgaddis
> _The idea of using SRV records in general seems foreign even in the ops
> realm._

I'm not suggesting you use it but (as for how it works in general) look into
how Microsoft's Active Directory works. They've had the "service discovery"
problem figured out for going on two decades, including preferences for
"closer" servers, weighting, and so on.

I'm no MS fan by any means but AD is one MS product that really is awesome.

~~~
snuxoll
And how does MS do it? SRV records, that’s right.

~~~
jlgaddis
Yeah, exactly. That's why I suggested he look into how it works (since "the
idea ... seems foreign" to him).

I didn't mean to suggest that they had some revolutionary new unique way of
handling service discovery or even that they were the first. I think maybe you
misinterpreted or misunderstood my intentions.

~~~
snuxoll
It’s all good, I got what you meant :)

LDAP and Kerberos of all implementation are probably the most common use case
for SRV records, most people probably don’t even know it. Real shame, in many
ways I wish A/AAAA records weren’t even a thing for the vast majority of
services (HTTP, I’m looking at you!)

------
otterley
Looks like both HAProxy and Nginx are going down the "pay for more advanced
features" route:

[https://www.haproxy.com/products/haproxy-enterprise-
edition/](https://www.haproxy.com/products/haproxy-enterprise-edition/)

[https://www.nginx.com/products/nginx/](https://www.nginx.com/products/nginx/)

~~~
wtarreau
What ? Are you kidding ? We're extremely open, we currently have 7 devs
working full time for most of the stuff you'll have in 1.8, we provide all the
sources with our packages (and some of our customers do their own builds), and
all the stuff we add in the enterprise version is in fact a fine selection of
stable enough features from the dev branch for people who need them right now
without waiting for the next version to be released. When a customer asks for
some specific feature we explain them that it first has to be in mainline so
we can backport it and it gets a wider exposure to spot bugs. How can you be
that insulting ? If it's a matter of unclear explanation on the web site, it's
possible and then please report this to contact@ or something like this (I
don't know the address). If it's just because you want to criticize without
knowing, well, in this case I hope I'll never count you among our users!

~~~
joeskyyy
Easy there, tiger. Not doing well for the brand of your team.

~~~
throw2016
Correcting an assumption can hardly be termed as 'lashing out', maybe a
'robust' response.

Rather have founders like wtarreau speak their mind than adopt some fake
passive aggressive tone or have a 'dev relations manager' spinning, at least
here.

~~~
abiox
> Correcting an assumption can hardly be termed as 'lashing out', maybe a
> 'robust' response.

'correcting an assumption' seems orthogonal to 'lashing out'.

also, it's not clear he even addressed the op's point, insofar as i
understand. haproxy enterprise does seem to contain features unavailable in
the 'community' version:

[http://www.haproxy.com/products/haproxy-community-vs-
enterpr...](http://www.haproxy.com/products/haproxy-community-vs-enterprise-
edition.html)

~~~
wtarreau
Sure but these are extra packages which help ease of deployment, management in
an enterprise context etc. Some of them are side projects (eg: VRRP is
provided by keepalived, route injection relies on Bird and some patches we
tried to upstream and which were rejected as useless, but which we still
provide in the source package). There are some internal development projects
for some extra products as well (which appear in blue on the web page), and
each time one of them requires any improvement from haproxy, the changes are
mainlined _before_. Any single line of code of haproxy comes from the mainline
sources, the source packages contains the patches, and all commit messages
even contain the whole cherry-picked chain leading to the master tree's commit
IDs first. We can hardly do better :-(

In fact, it's important to understand what enterprise users need : they never
want to have to deal with patches or feature backports, most of them don't
want to have to build anything, and instead they want to install regular pre-
compiled packages from a permanently updated repository. That's exactly what
we provide them. But providing pre-compiled packages is no excuse for not
giving their sources at the same time, which we do.

Let me be very clear about something : a load balancer (and haproxy is no
exception to this) is a very complex component and will _always_ have bugs.
And it happens that where it's deployed it's a critical component that must
never fail, which is a bit incompatible with the first rule. For having dealt
with proprietary LBs in the past, I'm well placed to know that you cannot and
must never trust a non-fully opensource load balancer. And we've kept this
spirit of full openness to achieve the highest possible reliability, and the
GPL maintains this guarantee here. Some of the top contributors of the project
are/have been working for enterprise customers and have brought very detailed
analysis about certain issues, sometimes even proposing patches to fix them
(or also small changes to make their life easier). As the project manager, I
don't care where a patch comes from: any user, an occasional code reviewer, a
customer, one of my coworkers etc. A fix is a fix, and if it addresses a real
bug it must be merged, and the same rule as in the kernel applies here with no
exception : mainline first. This guarantees that no patch is lost and the fact
that the patch is the most possibly correct. And in order to get these fixes,
you have to be the most open.

The situation as it is now benefits the two categories of users : \-
enterprise users don't have to wait for the next release to get certain
features they need right now : some of the features from haproxy-1.8-dev are
already present in HAPEE 1.6 or 1.7. So basically these users are encouraged
to pick the most stable version which satisfies all their needs. \- mainline
code benefits from some feedback from the field (improvements and fixes)
before the final release, which has allowed us quite a few time to adjust
certain confusing things (option names, default behaviour, etc) before
declaring them "stable" and having to maintain them forever.

The only advice I could give to people considering adopting a load balancer
from a vendor : during the evaluation, ASK FOR THE SOURCES! If you can't have
them, never use this product, one day or another you will regret it. And I'm
not saying this just to try to place us in front when many other solutions are
closed, but simply because I've been in this situation of using a closed
product quite a few times in the past and hated it. That's even what led me to
create haproxy in the first place!

------
bitly
Hi there. This is really awesome news and i am delighted to hear about this.

Any change to get some directions how to test this functionality?

So far I have currently 1.8-dev2 up and running but now I find no real
references about how to try it .. and if available in this repo. (maybe i have
to dig into the code ?)

Thanks and all the best.

~~~
bitly
[http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=e2d03d2...](http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=e2d03d2a43473a637491c0a8c9e10d5caed8e63b;hp=e70bc05b3a62ca84b5ce4440d340440f00411886)

Thanks and sorry for the noise

------
doublerebel
Interesting that HAProxy believes it is incumbent enough to charge for
directly copying a feature Consul has had for years. When I added Consul to my
stack 3 years ago HAProxy was already looking behind the times... I don't
think they can catch up for any new installation.

~~~
wtarreau
To be honest it's not a matter of duplicating, catching up or anything. It's
just that when a significant number of our users ask for exactly the same
stuff with similar and sometimes different justifications, it becomes quite
hard to avoid adding it. And if they want it there, they very likely have
their own very valid reasons. Baptiste (the DNS subsystem maintainer) has a
significant amount of use cases in mind and could likely much better explain
than me.

~~~
bedis9
We first developped DNS in HAProxy for our AWS users who wanted HAProxy to
follow-up a node when it is restarted (and it's IP is changed)... That was the
very first request from both community and customers. After we released this
feature, the community came back with some requirements regarding the ability
to use DNS resolution to resolve all the servers (or a set of servers at list)
of a backend. So we had to improve a lot the first dev we did to make this
possible. Support for SRV record is just the last stone we put on top of many
other devs :)

We're quite proud of the result because it makes HAProxy able to scale up /
down at run time without being reloaded and compatible with any service
registry able to export a list of nodes delivering a same service through DNS.

HAProxy can use multiple name for the resolution, pointing to different set of
DNS servers, enforcing custom "hold" timers (to bypass server's TTL or
negative TTLs in case of NX, etc...) and mix all of this with "old style
hardcoded" servers in the backend...

HAProxy is flexible :)

------
arianvanp
I hope the non-blocking DNS Client will be released separately as well. This
seems very useful on its own

~~~
otterley
Have you tried c-ares? It's venerable and useful.

~~~
clan
For the lazy (like me!): [https://c-ares.haxx.se](https://c-ares.haxx.se)

------
Whitespace
Anyone here ever used SRV records for local development? I really like the
idea of running a local DNS server with port information built in instead of a
full-blown service discovery/routing/service mesh setup.

~~~
doublerebel
Yes, try Consul. It's extremely easy to configure and run.

~~~
bedis9
I do agree on this statement!!! I personally validated HAProxy SRV records
with both Kubernetes and Consul and I was positively supersized of how simple
is consul. (Well, it does not cover all the stuff kubernetes does, you may
need nomad for this purpose).

------
linkmotif
Maybe there will there be a Kubernetes a Ingress Controller?? :)

~~~
aiharos
Kubernetes Ingress Controllers for HAProxy already exist. A nice one is at
[https://github.com/jcmoraisjr/haproxy-
ingress](https://github.com/jcmoraisjr/haproxy-ingress) Other examples of
usage are spread around in the examples section of the ingress repo, like here
[https://github.com/kubernetes/ingress/tree/master/examples/d...](https://github.com/kubernetes/ingress/tree/master/examples/deployment/haproxy)

The DNS-SD code will ship with HAProxy 1.8, and we will gladly help out by
adding it to the above ingress controller after the release. A WIP of that
code is up on github in case you’re interested. In fact, HAProxy is full of
surprises and little known features (like the dynamic scaling via haproxy
runtime api we already contributed to the controller) and we're happy to share
them with everyone!

