

Ask YC: Primer on DNS? - brandnewlow

I'm looking to move a few projects from shared hosting to VPS...and I can't get a straight answer out of my host as to what to expect as far as downtime with setting up nameservers and pointing my domains to them.<p>As far as I can tell, I'm supposed to set up my new nameservers and aim the relevant domains to these new nameservers.<p>What I don't get is what happens next.<p>The name servers have to propigate?  And this can take 6-24 hours?  Then the domain redirection has to propigate and this too can take 6-24 hours?<p>Is there any way to know when this propigation has happened?  It seems like I'll be flying blind and possible incurring up to 48 hours of downtime on these sites in order to get them working on the new nameservers.  Is this normal?  Seems awfully inefficient and mysterious.  I don't like the idea of passively waiting around with all my sites down while some mysterious process happens "across the internet".<p>Can anyone clear this up?  (I realize this is elementary stuff for this forum, but perhaps helpful for some?)
======
brk
Yeah, DNS is a bitch.

Best approach would be to host your DNS completely outside of your hosting
company. This provides a certain level of resiliency.

Changing just about anything with DNS can take 6-24+ hours to be fully
reflected on the Intartubes.

Just in summary, the DNS flow looks something like this:

Client asks DNS server what IP is www.foo.com DNS Server has no idea (it could
be in cache, but this is worst-case). DNS Server looks up the root name server
for foo.com and sees that NS1.BAR.COM is the nameserver for foo.com. DNS
server finds glue record for NS1.BAR.COM and finds that it has IP address
1.2.3.4 DNS asks 1.2.3.4 what IP is www.foo.com DNS Caches answer based on TTL
in response AND/OR sometimes its own settings to override too-low default TTLs
DNS Server tells Client PC www.foo.com == 1.2.3.4

Moving your domain from being handled by NS1.BAR.COM to NS1.BAZ.COM can take
6-12 hours to take effect, and then another 6-24 hours for most other servers
to expire their caches.

Changing www.foo.com to move from 10.10.10.10 to 10.12.13.14 takes effect
immediately on the nameserver (NS1...), but various other servers along the
way (often including the client PC) will have cached the old information, and
may not go out to see if the IP has changed for some period of time.

All in all, a "seemless" DNS move takes about 1 week end to end and goes
through a few iterations in the middle.

I personally would probably setup a new server _somewhere_ with HAProxy setup
to shoot connections back to my "old" server. Then make nameserver changes to
move DNS someplace better (this step might be able to be eliminated). Stage
site on new server. Change DNS to send traffic to haproxy IP. Set haproxy to
push traffic to new site. If things work as expected, set DNS to reflect IP of
new server, and watch haproxy logs to see when traffic stops coming in (3-5
days typically).

The reason for the haproxy bit is that you can "steer" the traffic at will and
immediately, no propagation changes. Many people have gotten burned on a new
site/server where something doesn't work and DNS changes aren't immediate, so
you're stuck with all these hits that can't be fulfilled.

Sorry if my answer is non-readable... I'm writing this while on a conference
call :)

I can help you more if you want, email in my profile...

------
bprater
You can hold downtime to a few minutes for a good chunk of your traffic.

How can you speed up the time it takes for DNS to propagate? Set your TTL low
now, before you the move. (Set it to about 300 seconds.) This means that many
DNS servers will query your DNS box every 5 minutes, rather than waiting for a
day or two.

You cannot know when propogation is "officially complete" as DNS servers
around the world may not respect your TTL and serve old results for a long
bit.

What you can do is put up a blanket webpage at the old host saying, "Hey
folks, we are moving hosts. If you want to visit the new site click here..."
and you can use the new IP address if you have your host configured properly.

If you don't "have" nameservers, you'll want to find a company like
dnsmadeeasy.com that can do this for you. Many hosts will do this though.
Slicehost, a VPS, has a DNS tab in their admin area. Find out if they can do
this for you.

So the steps:

1) Get your new site set up. (Use step 5 liberally to test if it is working
correctly without doing any switching yet.)

2) If your VPS does DNS, set up your records. If not, get one like
DNSMadeEasy.com and set up your records. Remember to set the TTL to 300
seconds. BUT set the records to point to the IP address of your OLD host.

3) Go to your domain registrar like GoDaddy and change the nameserver to
whatever DNS site you used in step 2. They will TELL you the names to enter,
it'll be like: ns1.namehost.com, ns2.namehost.com.

4) Give it a couple days for the switchover.

5) Ok, it's the day to switch. First update your hosts file on your PC, to do
a last minute test of your new site before the switch. (Add the domain and the
new IP address, just google to see how this is done. This allows you to test
in advance of the move.)

5) Now switch the A records from your old host to your new host. Tail the log
file. You should see traffic in a few minutes.

6) Put up a webpage on the old host pointing to the new host.

~~~
iigs
I had written up a long comment about what to do, but this more or less nails
it. Here's what I had to say from a more DNS centric perspective:

You can do this with no downtime if you don't have any other outside
circumstances forcing your hand, such as your current web hosting expiring
very soon.

Hint: find a web based DNS test tool. Google "DNS Report", the dnsstuff.com
one used to be really nice but they got too ambitious with a bunch of crap on
their site and I think it's a pay service now, the dnscolos.com one looks like
it's pretty good, I can't tell about the checkdns.net one. Also if you'll be
running your own DNS services (as opposed to outsourcing them where someone
else maintains the hosts) you'll want to find a DNS server package that has
good documentation and good tutorials. My personal bias is toward BIND to get
things going right away and then migrate to something else (PowerDNS, maybe)
after you get it working because BIND is very, um, featureful, and needs
updated a lot. :)

As you've determined, DNS servers are like web servers for static content in
that you can have two different sets of them running that think they're the
owners of the domain. If you give your new servers data identical to the old
ones, it won't matter which set is serving your records, as the client will
end up with the same data.

There is a notion, as you've probably read by now, of a TTL, or Time-To-Live.
This is a number that goes along side your DNS response and tells the computer
that your record will be staying the same for the next X seconds, so it
doesn't have to ask again. Managing this in all of the records is crucial to a
smooth migration.

You need to know the TTL of your current records. It sounds like you have a
UNIXy machine at your disposal in the VPS. Log in and execute:

dig -t NS google.com

You will see a bunch of goop, but you really only care about:

;; ANSWER SECTION: google.com. 345600 IN NS ns4.google.com. ...

The 345600 is the number of seconds that the current name servers can be
cached out on the internet. You won't be able to cleanly move your services
faster than that if you don't control your current nameservers.

dig google.com @ns4.google.com

;; ANSWER SECTION: google.com. 300 IN A 64.233.187.99 ...

if you care about your email you're going to need to track your MX records as
well, using:

dig -t MX google.com @ns4.google.com

wherein you will see:

;; ANSWER SECTION: google.com. 10800 IN MX 10 smtp4.google.com. ...

;; ADDITIONAL SECTION: smtp1.google.com. 3600 IN A 209.85.237.25

Again, in all of these answers field #2 (300, 10800, 3600) are the number of
seconds that each record type (A, MX, A for smtp) are cacheable by servers
across the internet. You will control these on your new server.

You will want to set the TTL of your A (and MX) records very low on your new
server. 300 seconds would be a 5 minute migration from site to site.

Once you have your DNS servers properly set up (dig example.com
@new.srvr.ip.addr) update your information with your registrar, giving them
your new name servers as the only nameservers for the account. Then the NS TTL
countdown begins. In our google test case you'd have four days before you
could move the site cleanly. I'm not sure if all registrars push updates
hourly or not -- it used to be that they only did it once daily. I'd wait NS
TTL + 24 hours to be safe, if you have the time.

Once everyone is visiting your new name servers, you now can migrate your non-
DNS servers. All you need to do is properly update the IP addresses, which can
include incrementing the zone serial numbers if you're using a BIND-alike.
After updating the A (and MX) IPs you have their old TTL seconds to wait
before the migration is complete and everyone should be viewing things on your
new servers.

After your migration is complete you can (at your option) turn your TTLs back
up so that the load is reduced on your DNS servers. It's not really critical
to do this, but it can carry you through if your DNS servers ever have a short
network connectivity outage.

Good luck.

I've glossed over a bunch of stuff (some root NS stuff, DNS is badly
implemented in a lot of places, some things don't honor TTLs, and so on) but
these steps should be enough to do an upgrade in the common case with no
problems.

------
mhud
You should not need to endure any downtime, even though DNS changes can take
days to propagate. The trick is to get DNS working at your new host BEFORE you
point your domains at the new host's servers. This way any given person will
be hitting the new server or the old server, but it won't matter -- both
should be serving the same information.

Other suggestions in this thread, such as reducing the TTL beforehand, will
help drastically if any issues arise.

------
jm4
I assume you're using your shared host's DNS servers and that a move to a VPS
also requires a DNS move. If it's possible to keep the same nameservers it's
just a matter of changing the record and everything will work right away.

Switching nameservers is what takes a long time.

It can take 24 hours, but it's usually much faster. There have been times that
I've seen the change take place in just a few minutes, but that only means
that the nameservers I'm using have picked up the change. Other nameservers
may take longer. The result is that you're not dealing with a total outtage.
Some users will get pointed to the new location while others will get pointed
to the old one.

You obviously want to do this when you expect usage to be at it's lowest and
it wouldn't be a bad idea to have a message on the old site letting users know
what's going on.

Also, as long as you're moving it's probably worth looking into a more
permanent DNS solution so that you don't need to switch nameservers again next
time you switch hosts.

~~~
maw
If you know you need to make to your network, you can set TTLs to be lower,
several days in advance. Problem solved.

(It's true that some crappy DNS resolvers don't honor TTLs, so you'll need to
retain control of old servers for at least several days afterwards, to set up
some automagic forwarding, receive mail from senders who haven't caught up
with new MX records, etc. But none of this should be too difficult.)

~~~
brk
True, but if you have a reasonably popular site SOMEONE has always broken DNS
TTL's along the way. I've been through this more times than I can count... If
your site is still small/early, it's less of an issue.

~~~
maw
Right. So, like I said, you need to control the old servers for at least a few
days after the cutoff date.

------
jmount
DNS and BIND, Third Edition By Paul Albitz, Cricket Liu Third Edition Pages:
499 ISBN 10: 1-56592-512-2 | ISBN 13: 9781565925120

~~~
tptacek
Or <http://cr.yp.to/djbdns.html> \--- no book required, and you avoid a really
spotty security track record.

DJBDNS will hurt a bit if you're not Unix-y, but it will hurt less than
managing zone files if you're not BIND-y. No sequence numbers, TTLs, or
cryptic "IN A blah blah terminating period" syntax.

------
jjguy
Uhm... Lots of good suggestions here. Given the innomence of your question,
I'll take the liberty of assuming you won't take offense to me questioning
some of the more fundamental assumptions. Why do you need to host your own
DNS? Honestly, it's a real pain. And if you're asking questions like this, you
probably don't need anything fancy. The control panels at the major registrars
are pretty advanced these days.

Apologies if you're a DNS rockstar, no offense intended.

~~~
brandnewlow
Definitely NOT hosting my own DNS. I've got my domains registered at
invisiondomains.com and am hosting someplace else. Over at invisiondomains, I
set up the nameservers and aimed my domains at the new nameservers. My initial
post was an attempt to find out if the downtime I was experiencing was normal
or just my host and registrar giving me a line. Not it seems that what I'm
seeing is normal for someone who doesn't take some of these more intricate
measures beforehand. All of this is really useful info. Like I said, I was
looking for a primer - to know if I should call bullshit on my host and
registrar or not.

------
conorh
I just did this recently. I reduced the TTL a few days before the move and I
also set up Apache on the old server to proxy all requests to the new server.
Look up Reverse Proxy for Apache. I turned that on when I changed the IP
addresses in the DNS records. So there was no downtime. I left this up for a
week and then shut it down. Worked great. Note: This site doesn't use SSL.
That would make it a little more complicated.

------
tptacek
Honestly, the "high end" DNS hosting services cost ~$15/mo. If you DIY, during
setup you're going to incur many times more than that in time and effort
costs, and you'll do it again every time you need to move DNS. If I were you,
I'd just use a good hosting service with a web interface.

------
brandnewlow
Ok. So I'm not crazy. It's incredibly helpful to know that 2 days of downtime
doesn't mean my host is a lemon per se and that I should reconcile myself to
some serious downtime over the weekend (which is the best time to be down for
me).

