By the Tweakers editorial team.
DNS anycast is now proven technology in the global Domain Name System. For some years, we’ve partnered with peers and service providers around the world to deliver the resilience benefits of anycast in the .nl domain. Although we’ve wanted to have an anycast set-up of our own for a while, we’ve so far been deterred by the complexity of co-locating hardware at sites around the globe.
The global Domain Name System (DNS) operates so unobtrusively that most internet users barely give it a second thought. Just as phones nowadays save us the trouble of remembering our friends’ numbers, so the DNS has for many years enabled us to find places on the internet without having the addresses. The DNS is a delegated system: your browser doesn’t normally know the IP address that goes with a URL that you type or click. It therefore starts by looking in the root of the DNS. In the case of a .nl address, the root will refer your browser to a server operated by SIDN (the Foundation for Internet Domain Registration in the Netherlands). SIDN’s server will in turn refer your browser to the server for the relevant domain. In the blink of an eye, the chain of referrals will end with your browser getting the IP address for the website or service you’re trying to reach.
It all sounds very neat, but under the hood the process is highly complex. Suppose that you’re in Sydney, Australia, when you try to reach that .nl address. If your Australian access provider’s resolver has to get the information it needs all the way from the Netherlands, that will slow down the lookup process considerably. Another, more pressing problem is that name servers have to be publicly known, and being conspicuous makes them prone to attack. It’s a risk that we’re acutely aware of. “A DDoS attack can have a serious effect on the quality of our services,” acknowledges Niek Willems, Systems Administrator at SIDN. “That’s obviously something we want to avoid, because it could mean that .nl domains are harder to reach for a while.”
Cornerstone of the internet
Clearly, the DNS is vital to the internet. If the DNS isn’t working, the internet isn’t working. No trouble or expense is spared, therefore, to keep the .nl domain in the air. In the past, for example, we used to have an arrangement where other registries operated secondary name servers for us, and we did the same for them. Unfortunately, a set-up like that can’t fully protect against modern DDoS attacks. That’s why, in recent years, anycast technology has come to the fore. Anycast actually works like a hack. “You have multiple name servers with the same IP address at various locations around the world. So, in our example, you have one in Sydney. The global DNS traffic is then spread across all your servers, making it much harder for an attack to succeed. Anycast is therefore a great way to build resilience against DDoS attacks, while also improving performance for users.” SIDN’s research team SIDN Labs is working with the ICT team to optimise the use of anycast and identify anything that can be improved. “We’re neatly keeping a top-level domain in the air and experimenting with new technologies at the same time,” says Willems.
BGP hack for resilience
Anycast is essentially a BGP mechanism. In other words, it’s based on the Border Gateway Protocol, the routing protocol that ensures the reachability of all IP addresses. “The great thing about BGP is that it allows for traffic to reach a given address by various routes. The system directs traffic via whichever route works out best. If you’re clever about how you set things up, you can have multiple distributed systems that use the same IP address, without the protocol choking. BGP simply sees multiple routes to the same server and flexibly redirects traffic so that it’s always taking the best route from source to destination,” explains Marco Davids, Research Engineer at SIDN Labs. “Anycast makes intelligent use of BGP’s inherent characteristics. Strictly speaking, having multiple servers with the same IP address at various locations goes against internet principles. But it means we can take the BGP’s ability to see multiple routes to a single system, and use it to enable traffic to reach any one of multiple systems. So, if someone in Florida, in the US, sends a DNS query for a .nl domain, it doesn’t go to a name server in the Netherlands. It goes to somewhere closer instead, maybe Miami. That means a faster response and a much better user experience.”
Global and local resilience
As mentioned earlier, anycast technology has been around for years. If, for example, someone in the Netherlands sends a query to 8.8.8.8 (the Google resolver), or to 1.1.1.1 or 9.9.9.9 (Cloudflare and Quad9), it arrives at a server in the same country, not a server in the States. That’s down to anycast. Cloudflare is one of several CDNs that use anycast. Popular media sites that generate high traffic volumes also employ various technologies, including anycast, to ensure that the traffic is handled more locally. For our part, we use anycast both on the global level (we have servers in multiple geographical regions) and on the local level (we have .nl name servers located with several Dutch service providers). “If, say, a Ziggo customer or KPN customer mounts a DDoS attack on one of our name servers, only their local server will be hit,” says Davids. “People accessing the internet through other service providers won’t be affected. And vice versa. If there’s a global attack from sources in Asia and the US, the associated DDoS traffic won’t ever reach the local nodes operated by Ziggo and KPN.” Our anycast set-up operates on the basis of cooperation with partners in other countries. At present, we work with the Swedish internet exchange Netnod (which has traditionally been very DNS focused and operates one of the root servers) and Nic.at in Austria. Outside Europe, CIRA of Canada provides us with anycast as a service. An anycast network of our own is something we’ve wanted for a while, but haven’t felt ready to realise. Locating, operating and maintaining physical servers at multiple sites around the world is a significant challenge. “Until now, the main thing we’ve lacked is the time to set up a network of our own.”
Testbed
An experimental Anycast 2020 Testbed has now been created to look at the problem from a new angle. The testbed is divided across three cloud service providers and features a total of nineteen nodes at seventeen locations. Explaining why SIDN has decided to explore a fresh approach, Davids says, “The market has moved on considerably in recent years. We’re interested to see whether we’ve now reached the point where highly mission-critical services can be run at least partially in the cloud. We also want to compare a cloud-based set-up with the traditional model.” Three service providers — Packet, Vultr and Heficed — have been selected for the testbed, giving a mix of ‘bare metal’ and virtual servers. “When choosing providers, we placed particular emphasis on connectivity. Heficed, for example, covers our connectivity requirements for South America and Africa. The choice was also limited by the fact that relatively few service providers let customers announce their own IP address space via the BGP. Several of the big tech service providers don’t allow it.”
Our Anycast 2020 Testbed will be used to assess the quality of the connectivity delivered by cloud service providers. SIDN could in principle arrange its own connectivity using servers operating at an internet exchange, for example. “What we’re particularly interested to know, though, is how connectivity is affected by reliance on a cloud service provider,” clarifies Davids. To run the project, SIDN Labs has teamed up with the operations staff in SIDN’s ICT Department. “The stability of the .nl domain is operationally vital, and joint initiatives like this can help us maximise stability. Collaboration has the added advantage of covering blind spots, because the combined project team includes people with expertise in a variety of different fields. From networking to Unix, and from office automation to security.”
Self-operated virtual anycast
Adding nodes to the testbed has proved to be very straightforward, but, as Willems points out, it involves a hidden risk. “In the test phase, you’re free to try things out. A certain amount of manual software installation isn’t an issue. But, in an operational setting, it’s not like that. Everything has to follow strict protocols to prevent mistakes. Before we can transition to the operational phase, therefore, we need to look at automating configuration management, monitoring and backups. There’s no room for error where operationally critical applications are concerned.” The initial goal is therefore to progress a single node from testing to production within a reasonable time frame. “At the moment, we’re still using fairly lightweight equipment and small zone files. However, the plan is to acquire a heavy-duty server and set it up to meet our quality requirements, so that it’s capable of handling the .nl zone file, which is much bigger. If everything goes well, we’ll connect the new server to the existing .nl DNS platform. And, if that’s a success, then the further transition to self-operated virtual anycast is largely a question of scaling up.” How will the anycast project make the .nl domain more resilient? “Diversity is a major contributor to resilience,” responds Davids. “In our testbed, we’ve got a particular type of name server running, whereas our existing anycast partners may be running another type, or they may have the ability to switch software packages in the event of a security issue. To some extent, therefore, building resilience is about diversification. For that reason, it’s certainly not the idea that we stop working with outside partners. What we want is to have our own systems as well. Our main aim is to deliver the best-quality service we can. After all, if the .nl domain ever went down, the damage to the Dutch economy would be enormous. We’re determined to do whatever it takes to prevent that.” The early signs from the anycast project are promising, which is good news for the Dutch internet. And the partnership between SIDN Labs and ICT is a great example of how research and operations teams can complement each other.
This knowledge article was originally published on Tweakers on 1 July 2020.