Power failures have always been one of the top causes of serious IT service outages. The loss of power to a data center can be devastating, and its consequences have fully justified the huge expense and effort that go into preventing such events.
But in recent years, other causes are catching up, with networking issues now emerging as one of the more common — if not the most common — causes of downtime. In our most recent survey of nearly 300 data center and IT service operators, network issues were cited as the most common reason for any IT service outage — more common even than power problems (see Figure 1).
Figure 1. Among survey respondents, networking issues were the most common cause of IT service outages in the past three years.
The reasons are clear enough: modern applications and data are increasingly spread across and among data centers, with the network ever more critical. To add to the mix, software-defined networks have added great flexibility and programmability, but they have also introduced failure-prone complexity.
Delving a little deeper confirms the complexity diagnosis. Configuration errors, firmware errors, and corrupted routing tables all play a big role, while the more traditional worries of weather and cable breaks are a relatively minor concern. Congestion and capacity issues also cause failures, but these are often themselves the result of programming/configuration issues.
Networks are complex not only technically, but also operationally. While enterprise data centers may be served by only one or two providers, multi-carrier colocation hubs can be served by many telecommunications providers. Some of these links may, further down the line, share cables or facilities — adding possible overlapping points of failure or capacity pinch points. Ownership, visibility and accountability can also be complicated. These factors combined help account for the fact that 39% of survey respondents said they had experienced an outage caused by a third-party networking issue — something over which they had little control (see Figure 2).
Figure 2. Configuration/change management failures caused almost half of all network-related outages reported by survey respondents.
Among those who avoided any downtime from networking-related outages, what was the most important reason why?
A few of those organizations that avoided any network-related downtime put this down to luck (!). (We know of one operator who suffered two separate, unrelated critical cable breaks at the same time.) But the majority of those who avoided downtime credited a factor that is more controllable: investment in systems and training (see Figure 3).
Figure 3.. Over three-quarters of survey respondents that avoided network-related downtime attributed it to their investments in resiliency and training.
The Bottom Line: As with the prevention of power issues, money spent on expertise, redundancy, monitoring, diagnostics and recovery — along with staff training and processes — will be paid back with more hours of uptime.
https://journal.uptimeinstitute.com/wp-content/uploads/2021/02/dd2.jpg8992034Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2021-02-22 05:44:002021-05-04 10:27:15Network problems causing ever more outages
The pandemic has led to a renewed interest by data center managers in remote monitoring, management and automation. Uptime Institute has fielded dozens of inquiries about these approaches in recent months, but one in particular stands out: What will operational automation and machine learning mean for on-site staff requirements?
With greater automation, the expectation is a move toward light staffing models, with just one or a handful of technicians on-site. These technicians will need to be able to respond to a range of potential situations: electrical or mechanical issues; software administration problems; break/fix needs on the computer-room floor; the ability to configure equipment, including servers, switches and routers; and so on. Do people with these competencies exist? How long does it take to train them?
Our experts agree: the requirement of on-site staff is shifting — from electrical and mechanical specialists to more generalist technicians whose primary role is to monitor and control data center activities to prevent incidents (especially outages).
Even before the pandemic, most data center technicians on-site did not carry out major preventative maintenance activities (although some do conduct low-level preventative maintenance); they support and escort vendors who do this work. The pandemic has accelerated this trend. On-site technicians today are typically trained as operational coordinators, switching and isolating equipment when necessary, ensuring adequate monitoring, reacting to unexpected and emergency conditions with the goal of getting a situation under control and returning the data center to a stable state.
Staffing costs have always been a major item in data center operations budgets. With the advent of better remote monitoring and resiliency of the data center in recent years, the perceived need for larger numbers of on-site data center staffing has diminished, particularly during off hours when activity is low. This trend is unlikely to reverse even after pandemic times.
One of our members, for example, runs an extremely large data center site that can be described as being built with a Tier III (concurrently maintainable) intent. It is mostly leased by hyperscale internet/cloud companies. On-site technicians are trained as generalists for operations and emergency coverage, and they work 12-hour shifts. A separate 8-hour day shift is staffed more heavily with engineers to handle customer projects and to assist with other operator activities as needed. All preventative maintenance is conducted by third-party vendors, who are escorted on-site by the staff technicians. Management anticipates moving to an automated, condition maintenance-based approach in the future, with the aim of lowering the number of on-site technical staff over time. The expectation is that on-site 24/7 staff will always be required to meet client service level agreements, but by lowering their numbers there will be meaningful operational savings.
However, this will not be a swift change (for this or any other data center). Implementing automated, software-driven systems is an iterative — and human-driven — process that takes time, ongoing investment and, critically, organizational and process change.
Technologies and services for remote data center monitoring and management are available and continue to develop. As they are (slowly and carefully) implemented, managers will feel more comfortable not having personnel on-site 24/7. In time, management focus will likely shift from ensuring round-the-clock staffing to developing more of an on-call approach. Already, more data centers are employing technicians and engineers who support multiple sites rather than having a fully staffed, dedicated team for each individual data center. These technicians have general knowledge of electrical and mechanical systems, and they coordinate the preventive and corrective maintenance activities, which are mostly performed by vendors.
Today, however, because of the pandemic, there is generally a greater reliance in on-site staffing, whereby technicians in the data center are providing managers with security/comfort and insurance in case there is an incident or outage. This is likely a short-term reaction.
In the medium term — say, in the next three to five years or so — we expect there will be increased use of plug-and-play data center and IT components and systems, so generalist site staffers can readily remove and replace modules as needed, without extensive training.
In the long term, more managers will seek to enhance and ensure data center and application resiliency and automation. This will involve the technical development of more self-healing systems/networks and redundancies (driven by software) that allow for reduced levels of on-site staff and reduced expertise of those personnel. If business functions can continue in the face of a failure without human intervention, then mean-time-to-repair becomes far less critical — a technician or vendor can be dispatched in due course with a replacement component to restore full functionality of the site. This type of self-healing approach has been discussed in earnest for at least the past decade but has not yet been realized — in no small part because of the operational change and new operational processes needed. A self-healing, autonomous operational approach would be an overhaul of today’s decades-long, industry-wide practices. Change is not always easy, and rarely is it inexpensive.
What is likely to (finally) propel the development of and move to self-healing technologies is the expected demand for large numbers of lights-out edge data centers. These small facilities will increasingly be designed to be plug-and-play and to be serviced by people with little specialized skills/training. On-site staff will be trained primarily to reliably follow directions from remote technical experts. These remote experts will be responsible for analyzing monitored data and providing specific instructions for the staffer at the site. It is possible, if not likely, that most people dispatched on-site to edge facilities will be vendors swapping out components. And increasingly, the specialist mechanical and electrical staff will not only be remote, but also trained experts in real-time monitoring and management software and software-driven systems.
Recent events have heightened concerns around physical security for many data center operators, and with good reason: the pandemic means many data centers may still be short-staffed, less time may have been available for review of and training on routine procedures, and vendor substitutes may be more common than under non-pandemic conditions. Add the usual “unusuals” that affect operations (e.g., severe storms causing staff absences and increasing the likelihood of utility failures), and normal precautions may fall by the wayside.
For most data centers, much of physical security starts at site selection and design. The typical layered (“box inside a box”) security strategy adopted by most facilities handles many concerns. If a data center has vulnerabilities (e.g., dark fiber wells beyond the perimeter), they’re generally known and provisions have been made to monitor them. Routine security standards are in place, emergency procedures are established, and all employees are trained.
But what keeps data center operators up at night is the unexpected. The recent bombing in Nashville, Tennessee (US), which disrupted internet and wireless services, and new threats to Amazon Web Services facilities as a result of their decision to suspend hosting the social media platform Parler are a stark reminder that extreme events can occur.
A December 2019 report from Uptime Institute summed it up best, by stating that IT security is one of the big issues of the information age. Billions of dollars are spent protecting the integrity and availability of data against the actions of malign agents. But while cybersecurity is a high-profile issue, all information lives in a physical data center somewhere, and much of it needs the highest order of protection. Data center owners/operators employ a wide range of tactics to maintain a perimeter against intruders and to regulate the activities of clients and visitors inside the data center. The full report assesses operator security spending, concerns, management and best practices.
Key Findings from December 2019 report: • Spending on physical security is commonly around 5% of the operations budget but in extremecases can be as high as 30%. • Data centers employ a range of common technologies and techniques to control access to the facility, but there is no “one size fits all” solution to physical security: each organization must tailor their approach to fit their circumstances. • Neither cloud-based data replication nor the threat of cybersecurity to both IT systems and facilities equipment have significantly diminished the need for physical security. • Most data center owners and operators consider unauthorized activity in the data center to be the greatest physical threat to IT. • Access to the data center property is governed by policies that reflect the business requirements of the organization and establish the techniques and technologies used to ensure the physical security of the facility. These policies should be reviewed regularly and benchmarked against those of similar organizations. • Data centers commonly employ third-party security services to enforce physical security policies. • Attempts at unwarranted entry do occur. In a recent study, about one in five data centers experienced some form of attempted access in a 5-year period. • Drones, infrared cameras, thermal scanners and video analytics are promising new technologies. • Biometric recognition is still viewed skeptically by many operators.
Now is the time to review your security plans and emergency operations procedures and to brief staff. Ensure they know the organization’s strategies and expectations. If your facility is in an area where many data centers are clustered together, consider collaborating with them to develop a regional plan.
https://journal.uptimeinstitute.com/wp-content/uploads/2021/01/sec1.jpg8041857Rhonda Ascierto, Vice President, Research, Uptime Institutehttps://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngRhonda Ascierto, Vice President, Research, Uptime Institute2021-01-25 04:34:002021-01-12 14:56:35Ensuring physical security in uncertain times
Data center operators (and enterprise IT) are generally cautious adopters of new technologies. Only a few (beyond hyperscale operators) try to gain a competitive advantage through their early use of technology. Rather, they have a strong preference toward technologies that are proven, reliable and well-supported. This reduces risks and costs, even if it means opportunities to jump ahead in efficiency, agility or functionality are missed.
But innovation does occur, and sometimes it comes in waves, perhaps triggered by the opportunity for a significant leap forward in efficiency, the sudden maturing of a technology, or some external catalyst. The threat of having to close critical data centers to move workloads to the public cloud may be one such driver; the need to operate a facility without staff during a weather event, or a pandemic crisis, may be another; the need to operate with far fewer carbon emissions may be yet another. Sometimes one new technology needs another to make it more economic.
The year 2021 may be one of those standouts in which a number of emerging technologies begin to gain traction. Among the technologies on the edge of wider adoption are:
Storage-class memory – A long-awaited class of semiconductors with ramifications for server performance, storage strategies and power management.
Silicon photonics – A way of connecting microchips that may revolutionize server and data center design.
ARM servers – Low-powered compute engines that, after a decade of stuttering adoption, are now attracting attention.
Software-defined power – A way to unleash and virtualize power assets in the data center.
All of these technologies are complementary; all have been much discussed, sampled and tested for several years, but so far with limited adoption. Three of these four were identified as highly promising technologies in the Uptime Institute/451 Research Disrupted Data Center research project summarized in the report Disruptive Technologies in the Datacenter: 10 Technologies Driving a Wave of Change, published in 2017. As the disruption profile below shows, these technologies were clustered to the left of the timeline, meaning they were, at that time, not yet ready for widespread adoption.
In 2017, Uptime Institute and 451 Research identified 10 technologies that had the potential to make a big, disruptive impact on the data center. Those to the right of the chart would do so sooner than those on the left, while those nearest the top would be most likely to have a disruptive impact. The size of the circle indicates how large an impact the technology would likely make.
Now the time may be coming, with hyperscale operators particularly interested in storage-class memory and silicon photonics. But small operators, too, are trying to solve new problems — to match the efficiency of their larger counterparts, and, in some cases, to deploy highly efficient, reliable, and powerful edge data centers.
Storage-class memory
Storage-class memory (SCM) is a generic label for emerging types of solid-state media that offer the same or similar performance as dynamic random access memory or static random access memory, but at lower cost and with far greater data capacities. By allowing servers to be fitted with larger memories, SCM promises to heavily boost processing speeds. SCM is also nonvolatile or persistent — it retains data even if power to the device is lost and promises greater application availability by allowing far faster restarts of servers after reboots and crashes.
SCM can be used not just as memory, but also as an alternative to flash for high-speed data storage. For data center operators, the (widespread) use of SCM could reduce the need for redundant facility infrastructure, as well as promote higher-density server designs and more dynamic power management (software-defined power is discussed below).
However, the continuing efforts to develop commercially viable SCM have faced major technical challenges. Currently only one SCM exists with the potential to be used widely in servers. That memory was jointly developed by Intel and Micron Technology, and is now called Optane by Intel, and 3D XPoint by Micron. Since 2017, it has powered storage drives made by Intel that, although far faster than flash equivalents, have enjoyed limited sales because of their high cost. More promisingly, Intel last year launched the first memory modules powered by Optane.
Software suppliers such as Oracle and SAP are changing the architecture of their databases to maximize the benefits of the SCM devices, and major cloud providers are offering services based on Optane used as memory. Meanwhile a second generation of Optane/3D XPoint is expected to ship soon, and by reducing prices is expected to be more widely used in storage drives.
Silicon photonics
Silicon photonics enables optical switching functions to be fabricated on silicon substrates. This means electronic and optical devices can be combined into a single connectivity/processing package, reducing transceiver/switching latency, costs, size and power consumption (by up to 40%). While this innovation has uses across the electronics world, data centers are expected to be the biggest market for the next decade.
In the data center, silicon photonics allows components (such as processors, memory, input/output [I/O]) that are traditionally packaged on one motherboard or within one server to be optically interconnected, and then spread across a data hall — or even far beyond. Effectively, it has the potential to turn a data center into one big computer, or for data centers to be built out in a less structured way, using software to interconnect disaggregated parts without loss of performance. The technology will support the development of more powerful supercomputers and may be used to support the creation of new local area networks at the edge. Networking switches using the technology can also save 40% on power and cooling (this adds up in large facilities, which can have up to 50,000 switches).
Acquisitions by Intel (Barefoot Networks), Cisco (Luxtera, Acacia Communications) and Nvidia (Mellanox Networking) signal a much closer integration between network switching and processors in the future. Hyperscale data center operators are the initial target market because the technology can combine with other innovations (as well as with Open Compute Project rack and networking designs). As a result, we expect to see the construction of flexible, large-scale networks of devices in a more horizontal, disaggregated way.
ARM servers
The Intel x86 processor family is one of the building blocks of the internet age, of data centers and of cloud computing. Whether provided by Intel or a competitor such as Advanced Micro Devices, almost every server in every data center is built around this processor architecture. With its powerful (and power-hungry) cores, its use defines the motherboard and the server design and is the foundation of the software stack. Its use dictates technical standards, how workloads are processed and allocated, and how data centers are designed, powered and organized.
This hegemony may be about to break down. Servers based on the ARM processor design — the processors used in billions of mobile phones and other devices (and soon, in Apple MacBooks) — are now being used by Amazon Web Services (AWS) in its proprietary designs. Commercially available ARM systems offer dramatic price, performance and energy consumption improvements over current Intel x86 designs. When Nvidia announced its (proposed) $40 billion acquisition of ARM in early 2020, it identified the data center market as its main opportunity. The server market is currently worth $67 billion a year, according to market research company IDC (International Data Corporation).
Skeptics may point out that there have been many servers developed and offered using alternative, low-power and smaller processors, but none have been widely adopted to date. Hewlett Packard Enterprise’s Moonshot server system, initially launched using low-powered Intel Atom processors, is the best known but, due to A variety of factors, market adoption has been low.
Will that change? The commitment to use ARM chips by Apple (currently for MacBooks) and AWS (for cloud servers) will make a big difference, as will the fact that even the world’s most powerful supercomputer (as of mid-2020) uses an ARM Fujitsu microprocessor. But innovation may make the biggest difference. The UK-based company Bamboo Systems, for example, designed its system to support ARM servers from the ground up, with extra memory, connectivity and I/O processors at each core. It claims to save around 60% of the costs, 60% of the energy and 40% of the space when compared with a Dell x86 server configured for the same workload.
Software-defined power
In spite of its intuitive appeal and the apparent importance of the problems it addresses, the technology that has come to be known as “software-defined power” has to date received little uptake among operators. Software-defined power, also known as “smart energy,” is not one system or single technology but a broad umbrella term for technologies and systems that can be used to intelligently manage and allocate power and energy in the data center.
Software-defined power systems promise greater efficiency and use of capacity, more granular and dynamic control of power availability and redundancy, and greater real-time management of resource use. In some instances, it may reduce the amount of power that needs to be provisioned, and it may allow some energy storage to be sold back to the grid, safely and easily.
Software-defined power adopts some of the architectural designs and goals of software-defined networks, in that it virtualizes power switches as if they were network switches. The technology has three components: energy storage, usually lithium-ion (Li-ion) batteries; intelligently managed power switches or breakers; and, most importantly, management software that has been designed to automatically reconfigure and allocate power according to policies and conditions. (For a more detailed description, see our report Smart energy in the data center).
Software-defined power has taken a long time to break into the mainstream — and even 2021 is unlikely to be the breakthrough year. But a few factors are swinging in its favor. These include the widespread adoption of Li-ion batteries for uninterruptible power supplies, an important precondition; growing interest from the largest operators and the biggest suppliers (which have so far assessed technology, but viewed the market as unready); and, perhaps most importantly, an increasing understanding by application owners that they need to assess and categorize their workloads and services for differing resiliency levels. Once they have done that, software-defined power (and related smart energy technologies) will enable power availability to be applied more dynamically to the applications that need it, when they need it.
The full report Five data center trends for 2021 is available to members of the Uptime Institute community which can be found here.
Through 2021 and beyond, the world will begin to recover from its acute crisis — COVID-19 — and will turn its attention to other matters. Few if any of these issues will be as important as climate change, a chronic condition that will become more pressing and acute as each year passes.
In the critical digital infrastructure sector, as in all businesses, issues arising directly or indirectly from climate change will play a significant role in strategic decision-making and technical operations in the years ahead. And this is regardless of the attitude or beliefs of senior executives; stakeholders, governments, customers, lobbyists and watchdogs all want and expect to see more action. The year 2021 will be critical, with governments expected to act with greater focus and unity as the new US government rejoins the global effort.
We can group the growing impact of climate change into four areas:
Extreme weather/climate impact – As discussed in our report The gathering storm: Climate change and data center resiliency, extreme weather and climate change present an array of direct and indirect threats to data centers. For example, extreme heatwaves — which will challenge many data center cooling systems — are projected to occur once every three or four years, not once in 20.
Legislation and scrutiny – Nearly 2,000 pieces of climate-related legislation have been passed globally to date (covering all areas). Many more, along with more standards and customer mandates, can be expected in the next several years.
Litigation and customer losses – Many big companies are demanding rigorous standards through their supply chains — or their contracts will be terminated. Meanwhile, climate activists, often well-resourced, are filing lawsuits against technology companies and digital infrastructure operators to cover everything from battery choices to water consumption.
The need for new technologies – Management will be under pressure to invest more, partly to protect against weather events, and partly to migrate to cleaner technologies such as software-defined power or direct liquid cooling.
In the IT sector generally — including in data centers — it has not all been bad news to date. Led by the biggest cloud and colo companies, and judged by several metrics, the data center sector has made good progress in curtailing carbon emissions and wasteful energy use. According to the Carbon Trust, a London-based body focused on reducing carbon emissions, the IT sector is on course to meet its science-based target for 2030 — a target that will help keep the world to 1.5 degrees Celsius (34.7 degrees Fahrenheit) warming (but still enough warming to create huge global problems). Its data shows IT sector carbon emissions from 2020 to 2030 are on a trajectory to fall significantly in five key areas – data centers, user devices, mobile networks, and fixed and enterprise networks. Overall, the IT sector needs to cut carbon emissions by 50% from 2020 to 2030.
Data centers are just a part of this, accounting for more carbon emissions than mobile, fixed or enterprise networks, but significantly less than all the billions of user devices. Data center energy efficiency has been greatly helped by facility efficiencies, such as economizer cooling, improvements in server energy use, and greater utilization through virtualization and other IT/software improvements. Use of renewables has also helped: According to Uptime Institute data (our 2020 Climate Change Survey) over a third of operators now largely power their data centers using renewable energy sources or offset their carbon use (see figure below). Increasing availability of renewable power in the grid will help to further reduce emissions.
But there are some caveats to the data center sector’s fairly good performance. First, the reduction in carbon emissions achieved to date is contested by many who think the impact of overall industry growth on energy use and carbon emissions has been understated (i.e., energy use/carbon emissions are actually quite a lot higher than widely accepted models suggest — a debatable issue that Uptime Institute continues to review). Second, at an individual company or data center level, it may become harder to achieve carbon emissions reductions in the next decade than it has been in the past decade — just as the level of scrutiny and oversight, and the penalty for not doing enough, ratchets up. Why? There are many possible reasons, including the following:
Many of the facilities-level improvements in energy use at data centers have been achieved already — indeed, average industry power usage effectiveness values show only marginal improvements over the last five years. Some of these efficiencies may even go into reverse if other priorities, such as water use or resiliency, take precedence (economizers may have to be supplemented with mechanical chillers to reduce water use, for example).
Improvements in IT energy efficiency have also slowed — partly due to the slowing or even ending of Moore’s Law (i.e., IT performance doubling every two years) — and because the easiest gains in IT utilization have already been achieved.
Some of the improvements in carbon emissions over the next decade require looking beyond immediate on-site emissions, or those from energy supplies. Increasingly, operators of critical digital infrastructure — very often under external pressure and executive mandate — must start to record the embedded carbon emissions (known as Scope 3 emissions) in the products and services they use. This requires skills, tools and considerable administrative effort.
The biggest operators of digital infrastructure — among them Amazon, Digital Realty, Equinix, Facebook, Google and Microsoft — have made ambitious and specific commitments to achieve carbon neutrality in line with science-based targets within the next two decades. That means, first, they are setting standards that will be difficult for many others to match, giving them a competitive advantage; and second, these companies will put pressure on their supply chains — including data center partners — to minimize emissions.
The full report Five data center trends for 2021 is available to members of Uptime Institute which can be obtained here.
https://journal.uptimeinstitute.com/wp-content/uploads/2021/01/sustainability2.jpg242700Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2021-01-11 05:42:002021-05-04 10:25:22Sustainability: More challenging, more transparent
One of the most widely anticipated trends in IT and infrastructure is significant new demand for edge computing, fueled by technologies such as 5G, IoT and AI. To date, net new demand for edge computing — processing, storing and integrating data close to where it is generated — has built slowly. As a result, some suppliers of micro data center and edge technologies have had to lower their investors’ expectations.
This slow build-out, however, does not mean that it will not happen. Demand for decentralized IT will certainly grow. There will be more workloads that need low latency, such as healthcare tech, high performance computing (notably more AI), critical IoT, and virtual and augmented reality, as well as more traffic from latency-sensitive internet companies (as Amazon famously said 10 years ago, every 100 milliseconds of latency costs them one percent in sales). There will also be more data generated by users and “things” at the edge, which will be too expensive to transport across long distances to large, centralized data centers (the “core”).
For all these reasons, new edge data center and connectivity capacity will be needed, and we expect a wave of new partnerships and deals in 2021. Enterprises will connect to clouds via as-a-service (on-demand, software-driven) interconnections at the edge, and the internet will extend its reach with new exchange points. Just as the internet is a network of tens of thousands of individual networks connected together, the edge will require not just new capacity but also a new ecosystem of suppliers working together. The year 2021 will likely see intense activity — but the long-expected surge in demand may have to wait.
The edge build-out will be uneven, in part because the edge is not a monolith. Different edge workloads need different levels of latency, bandwidth and resiliency, as shown in the data center schema below. Requirements for data transit and exchanges will also vary. Edge infrastructure service providers will need to rely on many partners, including specialist vendors that will serve different customer requirements. Enterprise customers will become increasingly dependent on third-party connections to different services.
So far, much attention has been focused on the local edge, where connectivity and IT capacity are sited within a kilometer or so from devices and users. In urban areas, where 5G is (generally) expected to flourish, and in places where a lot of IoT data is generated, such as factories and retail stores, we are slowly seeing more micro data centers being deployed. These small facilities can act either as private connections or internet exchange points (or both), handing off wireless data to a fiber connection and creating new “middle-mile” connections.
We expect that edge micro data centers will be installed both privately and as shared infrastructure, including for cloud providers, telcos and other edge platform providers, to reduce latency and keep transit costs in check. To get closer to users and “things,” fiber providers will also partner with more wireless operators.
In 2021, most of the action is likely to be one step further back from the edge, in regional locations where telcos, cloud providers and enterprises are creating — or consuming — new interconnections in carrier-neutral data centers such as colo and wholesale facilities. All major cloud providers are increasingly creating points of presence (PoPs) in more colos, creating software-defined WANs of public (internet) and private (enterprise) connections. Colo customers are then able to connect to various destinations, depending on their business needs, via software, hardware and networks that colos are increasingly providing. These interconnections are making large leased facilities a preferred venue for other suppliers to run edge infrastructure-as-a-service offerings, including for IoT workloads. For enterprises and suppliers alike, switching will become as important as power and space.
We expect more leased data centers will be built (and bought) in cities and suburbs in 2021 and beyond. Large and small colos alike will place more PoPs in third-party facilities. And more colos will provide more software-driven interconnection platforms, either via internal development, partnerships or acquisitions.
At the same time, CDNs that already have large edge footprints will further exploit their strong position by offering more edge services on their networks directly to enterprises. We’re also seeing more colos selling “value-add” IT and infrastructure-as-a-service products — and we expect they will extend further up the IT stack with more compute and storage capabilities.
The edge build-out will clearly lead to increased operational complexity, whereby suppliers will have to manage hundreds of application program interfaces and multiple service level agreements. For these reasons, the edge will need to become increasingly software-defined and driven by AI. We expect investment and partnerships across all these areas.
How exactly it will play out remains unclear; it is simply too early. Already we have seen major telco and data center providers pivot their edge strategies, including moving from partnerships to acquisitions.
One segment we are watching particularly closely is the big internet and cloud companies. Having built significant backbone infrastructure, they have made little or only modest investments to date at the edge. With their huge workloads and deep pockets, their appetite for direct ownership of edge infrastructure is not yet known but could significantly shape the ecosystem around them.
The full report Five data center trends for 2021 is available to members of Uptime Institute, guest membership can be found here.
https://journal.uptimeinstitute.com/wp-content/uploads/2020/12/Edge5sm.jpg235645Rhonda Ascierto, Vice President, Research, Uptime Institutehttps://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngRhonda Ascierto, Vice President, Research, Uptime Institute2021-01-04 05:34:002021-05-04 10:24:50Edge Computing – The Next Frontier
Network problems causing ever more outages
/in Design, Executive/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]Power failures have always been one of the top causes of serious IT service outages. The loss of power to a data center can be devastating, and its consequences have fully justified the huge expense and effort that go into preventing such events.
But in recent years, other causes are catching up, with networking issues now emerging as one of the more common — if not the most common — causes of downtime. In our most recent survey of nearly 300 data center and IT service operators, network issues were cited as the most common reason for any IT service outage — more common even than power problems (see Figure 1).
Figure 1. Among survey respondents, networking issues were the most common cause of IT service outages in the past three years.
The reasons are clear enough: modern applications and data are increasingly spread across and among data centers, with the network ever more critical. To add to the mix, software-defined networks have added great flexibility and programmability, but they have also introduced failure-prone complexity.
Delving a little deeper confirms the complexity diagnosis. Configuration errors, firmware errors, and corrupted routing tables all play a big role, while the more traditional worries of weather and cable breaks are a relatively minor concern. Congestion and capacity issues also cause failures, but these are often themselves the result of programming/configuration issues.
Networks are complex not only technically, but also operationally. While enterprise data centers may be served by only one or two providers, multi-carrier colocation hubs can be served by many telecommunications providers. Some of these links may, further down the line, share cables or facilities — adding possible overlapping points of failure or capacity pinch points. Ownership, visibility and accountability can also be complicated. These factors combined help account for the fact that 39% of survey respondents said they had experienced an outage caused by a third-party networking issue — something over which they had little control (see Figure 2).
Among those who avoided any downtime from networking-related outages, what was the most important reason why?
A few of those organizations that avoided any network-related downtime put this down to luck (!). (We know of one operator who suffered two separate, unrelated critical cable breaks at the same time.) But the majority of those who avoided downtime credited a factor that is more controllable: investment in systems and training (see Figure 3).
The Bottom Line: As with the prevention of power issues, money spent on expertise, redundancy, monitoring, diagnostics and recovery — along with staff training and processes — will be paid back with more hours of uptime.
Data center staff on-site: engineering specialists or generalists?
/in Executive, Operations/by Rhonda Ascierto, Vice President, Research, Uptime InstituteThe pandemic has led to a renewed interest by data center managers in remote monitoring, management and automation. Uptime Institute has fielded dozens of inquiries about these approaches in recent months, but one in particular stands out: What will operational automation and machine learning mean for on-site staff requirements?
With greater automation, the expectation is a move toward light staffing models, with just one or a handful of technicians on-site. These technicians will need to be able to respond to a range of potential situations: electrical or mechanical issues; software administration problems; break/fix needs on the computer-room floor; the ability to configure equipment, including servers, switches and routers; and so on. Do people with these competencies exist? How long does it take to train them?
Our experts agree: the requirement of on-site staff is shifting — from electrical and mechanical specialists to more generalist technicians whose primary role is to monitor and control data center activities to prevent incidents (especially outages).
Even before the pandemic, most data center technicians on-site did not carry out major preventative maintenance activities (although some do conduct low-level preventative maintenance); they support and escort vendors who do this work. The pandemic has accelerated this trend. On-site technicians today are typically trained as operational coordinators, switching and isolating equipment when necessary, ensuring adequate monitoring, reacting to unexpected and emergency conditions with the goal of getting a situation under control and returning the data center to a stable state.
Staffing costs have always been a major item in data center operations budgets. With the advent of better remote monitoring and resiliency of the data center in recent years, the perceived need for larger numbers of on-site data center staffing has diminished, particularly during off hours when activity is low. This trend is unlikely to reverse even after pandemic times.
One of our members, for example, runs an extremely large data center site that can be described as being built with a Tier III (concurrently maintainable) intent. It is mostly leased by hyperscale internet/cloud companies. On-site technicians are trained as generalists for operations and emergency coverage, and they work 12-hour shifts. A separate 8-hour day shift is staffed more heavily with engineers to handle customer projects and to assist with other operator activities as needed. All preventative maintenance is conducted by third-party vendors, who are escorted on-site by the staff technicians. Management anticipates moving to an automated, condition maintenance-based approach in the future, with the aim of lowering the number of on-site technical staff over time. The expectation is that on-site 24/7 staff will always be required to meet client service level agreements, but by lowering their numbers there will be meaningful operational savings.
However, this will not be a swift change (for this or any other data center). Implementing automated, software-driven systems is an iterative — and human-driven — process that takes time, ongoing investment and, critically, organizational and process change.
Technologies and services for remote data center monitoring and management are available and continue to develop. As they are (slowly and carefully) implemented, managers will feel more comfortable not having personnel on-site 24/7. In time, management focus will likely shift from ensuring round-the-clock staffing to developing more of an on-call approach. Already, more data centers are employing technicians and engineers who support multiple sites rather than having a fully staffed, dedicated team for each individual data center. These technicians have general knowledge of electrical and mechanical systems, and they coordinate the preventive and corrective maintenance activities, which are mostly performed by vendors.
Today, however, because of the pandemic, there is generally a greater reliance in on-site staffing, whereby technicians in the data center are providing managers with security/comfort and insurance in case there is an incident or outage. This is likely a short-term reaction.
In the medium term — say, in the next three to five years or so — we expect there will be increased use of plug-and-play data center and IT components and systems, so generalist site staffers can readily remove and replace modules as needed, without extensive training.
In the long term, more managers will seek to enhance and ensure data center and application resiliency and automation. This will involve the technical development of more self-healing systems/networks and redundancies (driven by software) that allow for reduced levels of on-site staff and reduced expertise of those personnel. If business functions can continue in the face of a failure without human intervention, then mean-time-to-repair becomes far less critical — a technician or vendor can be dispatched in due course with a replacement component to restore full functionality of the site. This type of self-healing approach has been discussed in earnest for at least the past decade but has not yet been realized — in no small part because of the operational change and new operational processes needed. A self-healing, autonomous operational approach would be an overhaul of today’s decades-long, industry-wide practices. Change is not always easy, and rarely is it inexpensive.
What is likely to (finally) propel the development of and move to self-healing technologies is the expected demand for large numbers of lights-out edge data centers. These small facilities will increasingly be designed to be plug-and-play and to be serviced by people with little specialized skills/training. On-site staff will be trained primarily to reliably follow directions from remote technical experts. These remote experts will be responsible for analyzing monitored data and providing specific instructions for the staffer at the site. It is possible, if not likely, that most people dispatched on-site to edge facilities will be vendors swapping out components. And increasingly, the specialist mechanical and electrical staff will not only be remote, but also trained experts in real-time monitoring and management software and software-driven systems.
Ensuring physical security in uncertain times
/in Executive/by Rhonda Ascierto, Vice President, Research, Uptime InstituteRecent events have heightened concerns around physical security for many data center operators, and with good reason: the pandemic means many data centers may still be short-staffed, less time may have been available for review of and training on routine procedures, and vendor substitutes may be more common than under non-pandemic conditions. Add the usual “unusuals” that affect operations (e.g., severe storms causing staff absences and increasing the likelihood of utility failures), and normal precautions may fall by the wayside.
For most data centers, much of physical security starts at site selection and design. The typical layered (“box inside a box”) security strategy adopted by most facilities handles many concerns. If a data center has vulnerabilities (e.g., dark fiber wells beyond the perimeter), they’re generally known and provisions have been made to monitor them. Routine security standards are in place, emergency procedures are established, and all employees are trained.
But what keeps data center operators up at night is the unexpected. The recent bombing in Nashville, Tennessee (US), which disrupted internet and wireless services, and new threats to Amazon Web Services facilities as a result of their decision to suspend hosting the social media platform Parler are a stark reminder that extreme events can occur.
A December 2019 report from Uptime Institute summed it up best, by stating that IT security is one of the big issues of the information age. Billions of dollars are spent protecting the integrity and availability of data against the actions of malign agents. But while cybersecurity is a high-profile issue, all information lives in a physical data center somewhere, and much of it needs the highest order of protection. Data center owners/operators employ a wide range of tactics to maintain a perimeter against intruders and to regulate the activities of clients and visitors inside the data center. The full report assesses operator security spending, concerns, management and best practices.
Key Findings from December 2019 report:
• Spending on physical security is commonly around 5% of the operations budget but in extremecases can be as high as 30%.
• Data centers employ a range of common technologies and techniques to control access to the facility, but there is no “one size fits all” solution to physical security: each organization must tailor their approach to fit their circumstances.
• Neither cloud-based data replication nor the threat of cybersecurity to both IT systems and facilities equipment have significantly diminished the need for physical security.
• Most data center owners and operators consider unauthorized activity in the data center to be the greatest physical threat to IT.
• Access to the data center property is governed by policies that reflect the business requirements of the organization and establish the techniques and technologies used to ensure the physical security of the facility. These policies should be reviewed regularly and benchmarked against those of similar organizations.
• Data centers commonly employ third-party security services to enforce physical security policies.
• Attempts at unwarranted entry do occur. In a recent study, about one in five data centers experienced some form of attempted access in a 5-year period.
• Drones, infrared cameras, thermal scanners and video analytics are promising new technologies.
• Biometric recognition is still viewed skeptically by many operators.
Now is the time to review your security plans and emergency operations procedures and to brief staff. Ensure they know the organization’s strategies and expectations. If your facility is in an area where many data centers are clustered together, consider collaborating with them to develop a regional plan.
A Surge of Innovation
/in Design, Executive/by Rhonda Ascierto, Vice President, Research, Uptime InstituteData center operators (and enterprise IT) are generally cautious adopters of new technologies. Only a few (beyond hyperscale operators) try to gain a competitive advantage through their early use of technology. Rather, they have a strong preference toward technologies that are proven, reliable and well-supported. This reduces risks and costs, even if it means opportunities to jump ahead in efficiency, agility or functionality are missed.
But innovation does occur, and sometimes it comes in waves, perhaps triggered by the opportunity for a significant leap forward in efficiency, the sudden maturing of a technology, or some external catalyst. The threat of having to close critical data centers to move workloads to the public cloud may be one such driver; the need to operate a facility without staff during a weather event, or a pandemic crisis, may be another; the need to operate with far fewer carbon emissions may be yet another. Sometimes one new technology needs another to make it more economic.
The year 2021 may be one of those standouts in which a number of emerging technologies begin to gain traction. Among the technologies on the edge of wider adoption are:
All of these technologies are complementary; all have been much discussed, sampled and tested for several years, but so far with limited adoption. Three of these four were identified as highly promising technologies in the Uptime Institute/451 Research Disrupted Data Center research project summarized in the report Disruptive Technologies in the Datacenter: 10 Technologies Driving a Wave of Change, published in 2017. As the disruption profile below shows, these technologies were clustered to the left of the timeline, meaning they were, at that time, not yet ready for widespread adoption.
Now the time may be coming, with hyperscale operators particularly interested in storage-class memory and silicon photonics. But small operators, too, are trying to solve new problems — to match the efficiency of their larger counterparts, and, in some cases, to deploy highly efficient, reliable, and powerful edge data centers.
Storage-class memory
Storage-class memory (SCM) is a generic label for emerging types of solid-state media that offer the same or similar performance as dynamic random access memory or static random access memory, but at lower cost and with far greater data capacities. By allowing servers to be fitted with larger memories, SCM promises to heavily boost processing speeds. SCM is also nonvolatile or persistent — it retains data even if power to the device is lost and promises greater application availability by allowing far faster restarts of servers after reboots and crashes.
SCM can be used not just as memory, but also as an alternative to flash for high-speed data storage. For data center operators, the (widespread) use of SCM could reduce the need for redundant facility infrastructure, as well as promote higher-density server designs and more dynamic power management (software-defined power is discussed below).
However, the continuing efforts to develop commercially viable SCM have faced major technical challenges. Currently only one SCM exists with the potential to be used widely in servers. That memory was jointly developed by Intel and Micron Technology, and is now called Optane by Intel, and 3D XPoint by Micron. Since 2017, it has powered storage drives made by Intel that, although far faster than flash equivalents, have enjoyed limited sales because of their high cost. More promisingly, Intel last year launched the first memory modules powered by Optane.
Software suppliers such as Oracle and SAP are changing the architecture of their databases to maximize the benefits of the SCM devices, and major cloud providers are offering services based on Optane used as memory. Meanwhile a second generation of Optane/3D XPoint is expected to ship soon, and by reducing prices is expected to be more widely used in storage drives.
Silicon photonics
Silicon photonics enables optical switching functions to be fabricated on silicon substrates. This means electronic and optical devices can be combined into a single connectivity/processing package, reducing transceiver/switching latency, costs, size and power consumption (by up to 40%). While this innovation has uses across the electronics world, data centers are expected to be the biggest market for the next decade.
In the data center, silicon photonics allows components (such as processors, memory, input/output [I/O]) that are traditionally packaged on one motherboard or within one server to be optically interconnected, and then spread across a data hall — or even far beyond. Effectively, it has the potential to turn a data center into one big computer, or for data centers to be built out in a less structured way, using software to interconnect disaggregated parts without loss of performance. The technology will support the development of more powerful supercomputers and may be used to support the creation of new local area networks at the edge. Networking switches using the technology can also save 40% on power and cooling (this adds up in large facilities, which can have up to 50,000 switches).
Acquisitions by Intel (Barefoot Networks), Cisco (Luxtera, Acacia Communications) and Nvidia (Mellanox Networking) signal a much closer integration between network switching and processors in the future. Hyperscale data center operators are the initial target market because the technology can combine with other innovations (as well as with Open Compute Project rack and networking designs). As a result, we expect to see the construction of flexible, large-scale networks of devices in a more horizontal, disaggregated way.
ARM servers
The Intel x86 processor family is one of the building blocks of the internet age, of data centers and of cloud computing. Whether provided by Intel or a competitor such as Advanced Micro Devices, almost every server in every data center is built around this processor architecture. With its powerful (and power-hungry) cores, its use defines the motherboard and the server design and is the foundation of the software stack. Its use dictates technical standards, how workloads are processed and allocated, and how data centers are designed, powered and organized.
This hegemony may be about to break down. Servers based on the ARM processor design — the processors used in billions of mobile phones and other devices (and soon, in Apple MacBooks) — are now being used by Amazon Web Services (AWS) in its proprietary designs. Commercially available ARM systems offer dramatic price, performance and energy consumption improvements over current Intel x86 designs. When Nvidia announced its (proposed) $40 billion acquisition of ARM in early 2020, it identified the data center market as its main opportunity. The server market is currently worth $67 billion a year, according to market research company IDC (International Data Corporation).
Skeptics may point out that there have been many servers developed and offered using alternative, low-power and smaller processors, but none have been widely adopted to date. Hewlett Packard Enterprise’s Moonshot server system, initially launched using low-powered Intel Atom processors, is the best known but, due to A variety of factors, market adoption has been low.
Will that change? The commitment to use ARM chips by Apple (currently for MacBooks) and AWS (for cloud servers) will make a big difference, as will the fact that even the world’s most powerful supercomputer (as of mid-2020) uses an ARM Fujitsu microprocessor. But innovation may make the biggest difference. The UK-based company Bamboo Systems, for example, designed its system to support ARM servers from the ground up, with extra memory, connectivity and I/O processors at each core. It claims to save around 60% of the costs, 60% of the energy and 40% of the space when compared with a Dell x86 server configured for the same workload.
Software-defined power
In spite of its intuitive appeal and the apparent importance of the problems it addresses, the technology that has come to be known as “software-defined power” has to date received little uptake among operators. Software-defined power, also known as “smart energy,” is not one system or single technology but a broad umbrella term for technologies and systems that can be used to intelligently manage and allocate power and energy in the data center.
Software-defined power systems promise greater efficiency and use of capacity, more granular and dynamic control of power availability and redundancy, and greater real-time management of resource use. In some instances, it may reduce the amount of power that needs to be provisioned, and it may allow some energy storage to be sold back to the grid, safely and easily.
Software-defined power adopts some of the architectural designs and goals of software-defined networks, in that it virtualizes power switches as if they were network switches. The technology has three components: energy storage, usually lithium-ion (Li-ion) batteries; intelligently managed power switches or breakers; and, most importantly, management software that has been designed to automatically reconfigure and allocate power according to policies and conditions. (For a more detailed description, see our report Smart energy in the data center).
Software-defined power has taken a long time to break into the mainstream — and even 2021 is unlikely to be the breakthrough year. But a few factors are swinging in its favor. These include the widespread adoption of Li-ion batteries for uninterruptible power supplies, an important precondition; growing interest from the largest operators and the biggest suppliers (which have so far assessed technology, but viewed the market as unready); and, perhaps most importantly, an increasing understanding by application owners that they need to assess and categorize their workloads and services for differing resiliency levels. Once they have done that, software-defined power (and related smart energy technologies) will enable power availability to be applied more dynamically to the applications that need it, when they need it.
The full report Five data center trends for 2021 is available to members of the Uptime Institute community which can be found here.
Sustainability: More challenging, more transparent
/in Design, Executive/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]Through 2021 and beyond, the world will begin to recover from its acute crisis — COVID-19 — and will turn its attention to other matters. Few if any of these issues will be as important as climate change, a chronic condition that will become more pressing and acute as each year passes.
In the critical digital infrastructure sector, as in all businesses, issues arising directly or indirectly from climate change will play a significant role in strategic decision-making and technical operations in the years ahead. And this is regardless of the attitude or beliefs of senior executives; stakeholders, governments, customers, lobbyists and watchdogs all want and expect to see more action. The year 2021 will be critical, with governments expected to act with greater focus and unity as the new US government rejoins the global effort.
We can group the growing impact of climate change into four areas:
In the IT sector generally — including in data centers — it has not all been bad news to date. Led by the biggest cloud and colo companies, and judged by several metrics, the data center sector has made good progress in curtailing carbon emissions and wasteful energy use. According to the Carbon Trust, a London-based body focused on reducing carbon emissions, the IT sector is on course to meet its science-based target for 2030 — a target that will help keep the world to 1.5 degrees Celsius (34.7 degrees Fahrenheit) warming (but still enough warming to create huge global problems). Its data shows IT sector carbon emissions from 2020 to 2030 are on a trajectory to fall significantly in five key areas – data centers, user devices, mobile networks, and fixed and enterprise networks. Overall, the IT sector needs to cut carbon emissions by 50% from 2020 to 2030.
Data centers are just a part of this, accounting for more carbon emissions than mobile, fixed or enterprise networks, but significantly less than all the billions of user devices. Data center energy efficiency has been greatly helped by facility efficiencies, such as economizer cooling, improvements in server energy use, and greater utilization through virtualization and other IT/software improvements. Use of renewables has also helped: According to Uptime Institute data (our 2020 Climate Change Survey) over a third of operators now largely power their data centers using renewable energy sources or offset their carbon use (see figure below). Increasing availability of renewable power in the grid will help to further reduce emissions.
But there are some caveats to the data center sector’s fairly good performance. First, the reduction in carbon emissions achieved to date is contested by many who think the impact of overall industry growth on energy use and carbon emissions has been understated (i.e., energy use/carbon emissions are actually quite a lot higher than widely accepted models suggest — a debatable issue that Uptime Institute continues to review). Second, at an individual company or data center level, it may become harder to achieve carbon emissions reductions in the next decade than it has been in the past decade — just as the level of scrutiny and oversight, and the penalty for not doing enough, ratchets up. Why? There are many possible reasons, including the following:
The biggest operators of digital infrastructure — among them Amazon, Digital Realty, Equinix, Facebook, Google and Microsoft — have made ambitious and specific commitments to achieve carbon neutrality in line with science-based targets within the next two decades. That means, first, they are setting standards that will be difficult for many others to match, giving them a competitive advantage; and second, these companies will put pressure on their supply chains — including data center partners — to minimize emissions.
The full report Five data center trends for 2021 is available to members of Uptime Institute which can be obtained here.
Edge Computing – The Next Frontier
/in Design, Executive/by Rhonda Ascierto, Vice President, Research, Uptime InstituteOne of the most widely anticipated trends in IT and infrastructure is significant new demand for edge computing, fueled by technologies such as 5G, IoT and AI. To date, net new demand for edge computing — processing, storing and integrating data close to where it is generated — has built slowly. As a result, some suppliers of micro data center and edge technologies have had to lower their investors’ expectations.
This slow build-out, however, does not mean that it will not happen. Demand for decentralized IT will certainly grow. There will be more workloads that need low latency, such as healthcare tech, high performance computing (notably more AI), critical IoT, and virtual and augmented reality, as well as more traffic from latency-sensitive internet companies (as Amazon famously said 10 years ago, every 100 milliseconds of latency costs them one percent in sales). There will also be more data generated by users and “things” at the edge, which will be too expensive to transport across long distances to large, centralized data centers (the “core”).
For all these reasons, new edge data center and connectivity capacity will be needed, and we expect a wave of new partnerships and deals in 2021. Enterprises will connect to clouds via as-a-service (on-demand, software-driven) interconnections at the edge, and the internet will extend its reach with new exchange points. Just as the internet is a network of tens of thousands of individual networks connected together, the edge will require not just new capacity but also a new ecosystem of suppliers working together. The year 2021 will likely see intense activity — but the long-expected surge in demand may have to wait.
The edge build-out will be uneven, in part because the edge is not a monolith. Different edge workloads need different levels of latency, bandwidth and resiliency, as shown in the data center schema below. Requirements for data transit and exchanges will also vary. Edge infrastructure service providers will need to rely on many partners, including specialist vendors that will serve different customer requirements. Enterprise customers will become increasingly dependent on third-party connections to different services.
So far, much attention has been focused on the local edge, where connectivity and IT capacity are sited within a kilometer or so from devices and users. In urban areas, where 5G is (generally) expected to flourish, and in places where a lot of IoT data is generated, such as factories and retail stores, we are slowly seeing more micro data centers being deployed. These small facilities can act either as private connections or internet exchange points (or both), handing off wireless data to a fiber connection and creating new “middle-mile” connections.
We expect that edge micro data centers will be installed both privately and as shared infrastructure, including for cloud providers, telcos and other edge platform providers, to reduce latency and keep transit costs in check. To get closer to users and “things,” fiber providers will also partner with more wireless operators.
In 2021, most of the action is likely to be one step further back from the edge, in regional locations where telcos, cloud providers and enterprises are creating — or consuming — new interconnections in carrier-neutral data centers such as colo and wholesale facilities. All major cloud providers are increasingly creating points of presence (PoPs) in more colos, creating software-defined WANs of public (internet) and private (enterprise) connections. Colo customers are then able to connect to various destinations, depending on their business needs, via software, hardware and networks that colos are increasingly providing. These interconnections are making large leased facilities a preferred venue for other suppliers to run edge infrastructure-as-a-service offerings, including for IoT workloads. For enterprises and suppliers alike, switching will become as important as power and space.
We expect more leased data centers will be built (and bought) in cities and suburbs in 2021 and beyond. Large and small colos alike will place more PoPs in third-party facilities. And more colos will provide more software-driven interconnection platforms, either via internal development, partnerships or acquisitions.
At the same time, CDNs that already have large edge footprints will further exploit their strong position by offering more edge services on their networks directly to enterprises. We’re also seeing more colos selling “value-add” IT and infrastructure-as-a-service products — and we expect they will extend further up the IT stack with more compute and storage capabilities.
The edge build-out will clearly lead to increased operational complexity, whereby suppliers will have to manage hundreds of application program interfaces and multiple service level agreements. For these reasons, the edge will need to become increasingly software-defined and driven by AI. We expect investment and partnerships across all these areas.
How exactly it will play out remains unclear; it is simply too early. Already we have seen major telco and data center providers pivot their edge strategies, including moving from partnerships to acquisitions.
One segment we are watching particularly closely is the big internet and cloud companies. Having built significant backbone infrastructure, they have made little or only modest investments to date at the edge. With their huge workloads and deep pockets, their appetite for direct ownership of edge infrastructure is not yet known but could significantly shape the ecosystem around them.
The full report Five data center trends for 2021 is available to members of Uptime Institute, guest membership can be found here.