Distance Between Datacenters

One of the most common questions when designing a IT service continuity plan is distance between datacenters. Answer is easy: a distance that would be far enough to grant that an event is not going to affect to the principal and the alternate data center, but close enough to ensure that services are going to be delivered in a proper way, specifically in those services in which people are involved.

IT industry maturity has caused that, in most companies, IT service process has a high level of automation or are remotely managed and cloud computing is a clear example of this. This fact has and caused that risk is  considered as the only aspect to take into account when selecting an alternative data center location, so let put the data center far far away.

However there are two issues that affect when determining the distance: Cost and Technology.

Technology

Telecommunications between data center are done using optical fiber in most cases, although depending on applications requirements, different protocols will be used. Basically there are three types of data centers communications:

• Level 3 interconnections: are used to connect data centers with few latency and transparency requirements. MacroLan by Telefónica, LanLink by Colt or LanExtesion by BT are the typical level 3 operators products .
• Level 2 interconnections: Are based in link protocols, for example, and over all Ethernet. This kind of connections are more transparent than the lever 3 connections and, with techniques like  FCoE, allow to cover most of connectivity scenarios if applications has not a few latency.
• Level 1 interconnections: signal transmission through optical fibber multiplexed with DWDM or CWDM that allow interconnection high capacity interconnections (nowadays 40 Gb and up to 100Gb). with a very few latency. It'll be more important the latency due to light transmission.

Most data centers are connected using this technologies, being between 10 and 80 kms the usual distances among them. For longer distances, projects are more complex, because long distance networks has to be used, with, for example, SDH protocol that allow a continuous signal regeneration, although latency are introduced.

Most multinationals have chosen an architecture that offers a lot of technical functionalities with the minimum risk, that is to have two data centers one close to the other (1 to 5 kms) operating in an active-active configuration and the alternative data center for disaster or disruptive events is located hundreds of miles away, even in another continent. For example, Santander bank has in this architecture datacenters in Madrid, Cantabria ( upcoming opening ), London, Sao Paulo (Brasil) and Querétaro (México). The following video shows a virtual tour for the Brazilian datacenter of Santander.

Cost

It's easy to conclude that the longer the distance, the higher the cost. Depending on fiber attenuation interconnections has to be implemented with more or less power lasers. For distances longer than 80kms regeneration or amplification of signal will be required, so more equipment and more cost. Talking about very long distances, like London - Cantabria of Santander bank, international lamdas will be involved and the bill could be very expensive and difficult to justify.

Another fact to take into account is that the longer the distance, the higher probability to be intercepted and confidential information to be disclosed, and thats why cipher technologies has to be used, as I explain in this post in secuirtybydefault blog.