DNS server infrastructure

by Kurt Seifried, [email protected]

It's funny to see the Internet sneaking up on businesses and becoming a critical component, without many people seeming to realize it. Many businesses now rely heavily on email as a messaging system, the use of web servers to distribute corporate data and allow access to a variety of services. All these services rely on DNS. DNS is the directory to the Internet, it maps names such as seifried.org to IP addresses such as 1.2.3.4. To see a fortune 100 company, such as Microsoft, suffer a multi-day outage because their DNS infrastructure is not up to the task is disturbing indeed.

DNS Security and availability

In the past we've run several articles on DNS security, and by and large most organizations have begun to pay attention to these issues and address them. However a significant part of security is availability. While preventing an attacker from accessing sensitive data is definitely a security goal, allowing an attacker to deny access to that resource, for everybody, can be a significant problem. Even if your webservers are in tip-top shape, the firewalls are doing their job, and your backend application servers and databases are in perfect order it doesn't matter if an attacker manages to take out your DNS servers. Without DNS servers no-one on the Internet will be able to find your servers, would you be able to remember 199.81.203.50, 153.2.228.50 or 56.0.78.101? Of course not, but you can remember fedex.com, ups.com and usps.com. Worse yet without your DNS servers internal services may not work properly, email deliveries can fail and access to servers will timeout as DNS queries fail.

How DNS works

The first step is to contact a DNS registrar such as Network Solutions and pony up $35 for a DNS name (good for one year). If you forget to pay your registration fees your DNS name will be taken out of the database and DNS will stop working (this has happened to Microsoft in the past). So make sure you document somewhere when your DNS fees need to be renewed. While you can pay up to 10 years in advance I would not recommend it, making DNS renewal a yearly ritual is probably better then leaving it for 10 years (at which point your contact with the Internic may have retired) and hoping that the reminder gets to the right person.

Once this is done you can register up to 6 DNS servers for a domain name. NetBSD for example has six servers listed, if five fail in some manner DNS queries will still work (albeit slowly, since clients will have to try several servers before finding the working one). This is your first line of defense against an attacker, since hopefully no attacker will be able to take out all the root servers you can use them to do limited load balancing, but more importantly to list multiple servers.

The DNS client will then query one of the DNS servers listed by the Internic, so having numerous servers spread out (different locations, backbone providers, etc.) will provide a good degree of redundancy. The cost to host several machines at a major co-location provider to host DNS services is trivial compared to the cost of having an extended outage for most organizations. At a bare minimum you'll need a 1U rack mount server with enough ram to handle the database in memory (in most cases 512 megs will be plenty), you may additionally want to place a firewall in front of it to prevent attacks directly on the firewall.

By having several sets of DNS servers at different provides a much more robust system, an attacker must take out more then one target, however if all your DNS servers are running at 100% capacity and an attacker takes one server out that may cause enough added load on the other server to make them unresponsive. Ideally a single server should be able to handle the full load, realistically you should be able to lose at least one, and probably two servers without overloading the remaining servers. Microsoft, like a wounded gazelle, was attacked, most likely the chokepoint router was targeted, which would have the effect of blocking access to the four DNS servers behind it. It doesn't matter how powerful and fast and well secured those four DNS servers were, the router in front of it was most likely dead (traceroute response was very sporadic).

Below is a table listing public information about various major companies DNS configuration. This information was gleaned by using:

whois [email protected]
dig -t ns example.org
traceroute foo.example.org

Microsoft has made some serious improvements in their DNS configuration, by dint of outsourcing some of it to Akamai, how is extremely experienced at this sort of thing (oddly enough the Akamai server handling Microsoft's DNS appears to be running Linux). I also expect to see Microsoft do some reorganization of their own servers in the future. The following companies rely very heavily on the Internet, from providing simple product information to product updates, critical security fixes, and so forth. In general they are doing a good job, most have DNS servers in three or more separate locations. This chart reflect data from Jan. 31 of 2001.

Two vendors stand out as having particularly poor DNS infrastructure. Caldera by far has the worst, with only two DNS servers hosted at the same site, in fact this is the site that hosts most of their servers, email, ftp and so on. Essentially they have a network link to their offices with all their infrastructure based their, if someone were to flood a router on that link they could likely take out Caldera entirely (DNS, email, secondary email, ftp, etc.). Another vendor with a poor DNS infrastructure is Mandrake, while not nearly as bad as Caldera it is far from perfect. Mandrake appears to host two DNS servers with Level3 (a major provider), and while it is unlikely they can be simply flooded, it appears that they are not firewalled from the Internet, meaning that an attack on the DNS server themselves is possible (they are running several services not related to DNS on the servers).

And neither of these companies are alone, according to an article in TheRegister (URL below) about 25% of fortune 1000 companies are vulnerable in similar ways to Microsoft (and Caldera / Mandrake). However as you can see from the table it is possible to do it correctly, many major companies (such as IBM and Novell) as well as mid sized companies (Red Hat and SuSE) and even several free community efforts (such as OpenBSD and NetBSD) have solid DNS infrastructure. This is not a network task that should be put off, if you do not have DNS servers in at least two (or preferably three or more) separate locations then you should start on this immediately. While I do not advise completely outsourcing your DNS (the provider may not have properly secured DNS servers, etc.) co-locating machines at a major ISP is a reasonable solution. One thing to remember is that zone transfers are not encrypted, so some form of VPN (such as IPSec) is recommended to secure the path between your primary DNS servers and backup servers. It can be done properly, and should be done properly, because your companies reliance on the Internet is not going to decrease anytime soon.

Reference Links:

MS DNS mess matched by 25% of Fortune 1000 - http://www.theregister.co.uk/content/6/16410.html

Back

Last updated on 3/24/2002

Copyright Kurt Seifried 2002