Two persistent trends in data center staffing are in apparent tension. The 2023 Uptime Institute Global Data Center Survey confirmed, once again, that operations teams are struggling to attract and retain qualified staff. The severity of this shortage should justify aggressive hiring from all available labor sources — yet data centers still employ shockingly few women. The average proportion of female employees at respondents’ organizations is just 8% — lower than in many physically demanding, conventionally male-dominated industries, such as construction, mining and manufacturing.
In the Uptime Institute Global Data Center Survey 2023 report, Uptime details the staffing shortage that has frustrated the data center industry for more than a decade. Operator responses show that the past four years have been particularly trying for the sector. About half of the survey’s respondents reported difficulty in filling open job positions, and one in four have seen their staff hired away, with most being poached by competitors. Skills shortages affect virtually all job roles but are the most acute among operations, mechanical and electrical staff.
While researching the staffing demands of data center design, build, and operations teams, Uptime has collected gender data since 2018. Figure 1 shows that the majority (81%) of data center teams are overwhelmingly male, with women making up one in 10 workers or fewer. On average, a meager 8% of all teams are made up of women. This year, Uptime calculated this weighted average for the first time, and five years of data reveals no significant change.
Figure 1 Data center teams lack gender diversity
In the 2019 questionnaire regarding hiring initiatives, good intentions abound. Almost three in four respondents agreed the industry would benefit from hiring more women — and 45% said the gender imbalance was a threat to the industry, contributing to the dry talent pipeline and even technical stagnation. Nearly half of respondents were planning initiatives to hire more women — or had such programs already in place. These results suggested a will to invest in hiring women, and an expectation of that investment paying off.
Four years on, the staff shortfall looms large in keynote presentations and data halls alike. So, where are the women? Did the time and money invested in hiring initiatives fail to attract female candidates, or were these plans and initiatives quietly shelved? Regardless of the causes, the gender divide persists, and the benefits of mitigating it remain hypothetical.
Identifying the most relevant influences on the gender balance of the workforce is difficult without any motion in the data — but some explanations can be ruled out by comparison against other industries.
Anecdotally, women are disinclined to pursue manual labor — and labor statistics confirm that physically demanding job roles typically attract fewer women. However, this alone cannot account for the disparity in the data halls. Uptime’s findings suggest data center operations teams employ fewer women than manufacturing (29%), mining (15%) and construction (11%) — according to figures from a 2021 publication by the US Bureau of Labor Statistics (Employment and Wages Online Annual Averages). The construction workforce in western Europe is about 9% women, as per government statistical offices, and the International Energy Agency reports at least 10% women in the mining industry in all regions.
In occupational health and safety research, women describe construction sites as hostile workplaces. Women endure isolation, sexual harassment, and bullying, as well as injuries from tools and personal protective equipment designed for the average size, weight, and grip strength of a male worker. In interviews, women say they experience little job security in construction, and this is discouraging timely reporting of harassment or injuries, and frustrating efforts at improvement. If female workers are choosing such punishing job sites over data centers (even by a small margin), there must be other factors influencing the minimal presence of women — beyond the assumed aversion to physical labor.
Data center operations are not a highly visible career path, and the problem is more pronounced for women. With few female role models in the industry, women may perceive a data center career as unwelcoming or unsafe and may feel discouraged from applying. This cycle is likely to perpetuate itself unless more resources are devoted specifically to closing the gender gap.
Many data center operators now desire better visibility in the labor market and are bringing their organizations out of their habit of comfortable obscurity. To draw in more workers, the industry will need to effectively communicate that it has rewarding careers available for qualified women. An uptick in female representation would strengthen this message and could act as an indicator of early successes for recruitment efforts at large.
To draw future workers from universities and trade schools, some data center organizations have formed educational partnerships and programs — often working with high school students, or younger. Efforts in education to attract more women and girls to careers in data centers must begin today, to produce more gender diversity when these students grow into job seekers.
The data center industry also seeks out workers who are changing careers and can bring some applicable skills with them. Operators can smooth the transition into a data center career by revisiting the advertised job requirements — lowering the bar for entry so candidates can supplement their existing skill sets with training on the job and mentoring programs. Active outreach to women in this talent pool can ensure that fewer qualified candidates are overlooked.
This avenue of recruitment can expand to include “career returners.” Many in this group are women who left the workforce for childcare or family care duties — and operators offering benefits suited to workers with family obligations may gain an advantage in recruitment.
Operators desperate for staff would benefit from understanding what went wrong with female representation in the industry. Ongoing efforts to recruit women must be re-examined and held to account for a lack of return on time and money invested.
To lessen the data center staffing struggle, operators will need to draw from every available labor pool. Women and other underrepresented groups are underutilized resources — and the industry likely will not escape its staffing crisis until it can bring them in.
The Uptime Intelligence View
Gender balance in the data center lags many physically demanding, conventionally male-dominated industries such as construction, mining, and manufacturing — a sign the industry is not doing enough to advertise rewarding career opportunities to women. Data centers cannot afford to let female candidates remain left out. Outreach efforts to career changers, universities, and vocational training could maximize their returns by seeking out more women.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/10/Data-Center-Are-Short-Staffed-Boys-Clubs-featured.jpg5391030Jacqueline Davis, Research Analyst, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngJacqueline Davis, Research Analyst, Uptime Institute, [email protected]2023-11-01 12:00:002023-10-31 11:10:50Data centers are short-staffed boys’ clubs
In recent reports, Uptime Institute Intelligence has warned that a wave of resiliency, security and sustainability legislation is making its way toward the statute books. Governments around the world — aware that digital infrastructure is increasingly critical to economic and national security (and consumes a lot of power) — have decided the sector cannot be left unwatched and unmanaged.
New laws relating to data center sustainability have attracted the most attention, partly because some of the provisions will likely prove expensive or difficult to meet. In Germany, for example, the Energy Efficiency Act, which was passed by Germany’s lower house of parliament in September 2023, requires all new data centers to reuse a set proportion of their waste heat (with some exceptions) and have a power usage effectiveness (PUE) of 1.3 or below. The legislation also specifies that older data centers will be required to reach this level by 2026.
The act, which has been dubbed by some as the “data center prevention act,” is the first of many new laws planned by European governments. It anticipates (and adds to) the requirements of the EU’s Energy Efficiency Directive (EED), which comes into force in the latter part of 2023. The EED contains a long list of onerous reporting and improvement requirements for data centers that will be transposed into national law in all 27 EU member states.
Sustainability is not, however, the focus of a lot of the upcoming regulation. A recent Uptime Institute report, Digital resiliency: global trends in regulation, looks at how legislation addressing resiliency across the digital supply chain is being implemented or planned in the US, EU, UK, Singapore, Australia and elsewhere. At least some of these laws will have far-reaching effects on the digital infrastructure ecosystem.
While rules to ensure resiliency are not new, the latest wave signals a significant extension of regulatory oversight. The new laws are a response to the growing threat of complex, systemic or even irrecoverable outages and are a recognition that data center services now play a critical role in a modern economy — a lesson underlined during the COVID-19 pandemic.
A common theme of these new rules is that, effectively, governments are now classifying digital infrastructure as part of the critical national infrastructure. This is a specific term that means operators of critical services are subject to security, availability, transparency and reporting mandates. Under these regulations, all participants in the infrastructure supply chain — whether software, hosting, colocation, cloud or networking — need to be transparent and accountable.
Uptime Intelligence research suggests that few operators are up to date with pending legislation or the requirements and costs of compliance. In the area of sustainability, surveys show that most data center / digital infrastructure operators are not collecting the data they will soon need to report.
Operators in North America (primarily the US) tend to be much more wary of regulation than their counterparts in other parts of the world — wariness that applies both to sustainability and to resiliency / transparency. In a 2021 Uptime Institute climate change survey, for example, three-quarters of European and Asian data center operators said they thought data center sustainability laws were needed — but only about 40% of US operators agreed (see Figure 1).
Figure 1. Majority invite more regulation for sustainability
In the 2023 Uptime Institute Global Data Center Survey, operators were asked about their attitude toward resiliency laws that would reveal more details of data center infrastructures and would enable certain customers (i.e., clients with critical requirements) to visit or assess facilities for resiliency. Operators everywhere were broadly supportive, but once again, those in North America were the most wary (see Figure 2).
Figure 2. North American operators least likely to favor transparency laws
Differences in attitudes toward the role of regulation are not limited to businesses; regulators too differ in their aims and methods. The US, for example, generally puts more emphasis on economic incentives, partly through the tax system, such as the Inflation Reduction Act, while Europe favors the stick — rules must be followed, or penalties will follow.
In the US, when resiliency laws are introduced or proposed, for example by the Securities and Exchange Commission, they are not expected to be backed up by tough sanctions for failures to comply — unlike in the EU, where penalties can be high. Instead, organizations are encouraged to conform, or they will be prevented from bidding for certain government contracts.
And while Europe and parts of Asia (such as China and Singapore) have tough sustainability laws in the pipeline, the US has no federal laws planned.
There is, of course, a long-running debate over whether expensive carrots (the US model) or punitive, bureaucratic sticks are the most effective methods in facilitating change. Evidence from the Uptime Institute annual survey does show that regulations drive some investments (see Regulations drive investments in cybersecurity and efficiency). But, of course, so do rules that require investments in equipment.
For data center owners and operators, the end result may not be so different. High levels of resiliency and transparency are mostly expected to be rewarded in law and in the market, as are energy efficiency and low carbon emissions.
However, the incentive model may cost less for operators because of generous rebates and exemptions — Uptime estimates that fulfilling the emerging reporting requirements for sustainability and resiliency can cost upwards of $100,000.
The Uptime Intelligence View
The role and extent of regulation — and of incentives — is likely to change constantly in the next decade, making it difficult for data center operators to formulate a clear strategy. The most successful data center owners and operators will be those that aim for high standards at an early stage, in both areas of resiliency and sustainability, and invest accordingly. The business case for such investments is becoming ever stronger — in all geographies.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/10/Consensus-on-regulatory-goals-hides-national-differences-featured.jpg5391030Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2023-10-18 15:00:002023-10-16 12:07:26Consensus on regulatory goals hides national differences
Legislative requirements for data center resiliency, operational transparency and energy performance are tightening worldwide — putting data centers under greater regulatory scrutiny. In response, organizations are either starting or stepping up their efforts to achieve compliance in these areas, and findings from the Uptime Institute Global Data Center Survey 2023 reveal that most are prioritizing cybersecurity (see Figure 1).
Figure 1. Regulations drive security, hardware and efficiency investments
Since 2020, several countries have introduced laws with strict cybersecurity demands for data center operators to combat the rise in cyber threats (see Table 1) — especially if they host or manage critical national infrastructure (CNI) workloads. As CNI entities become more reliant on digital services, they are increasingly exposed to cyber risks that could result in severe consequences. For example, a compromised facility managing applications for a utility risks widespread power and communications outages, threatening the physical safety of citizens.
Table 1. Regulations that mandate enhanced cybersecurity measures
Cyberattacks are becoming increasingly sophisticated as the digital infrastructure becomes more interconnected. For example, operational technology systems for power and cooling optimization are routinely connected to the internet (either directly or indirectly), which creates a broader “attack surface,” giving more access points for cyberattacks. Operators are also increasingly deploying Internet of Things devices and applications. These are used for asset tracking, predictive maintenance and capacity planning, but they require network connectivity and can lack robust cybersecurity features.
Measures aimed at improving energy efficiency rank as the second and third most popular responses to new regulations (see Figure 1). To evaluate their progress, data center operators may add new energy management systems and network connections to the power infrastructure, potentially complicating existing cybersecurity programs.
Alongside the risks to CNI, cyberattacks could lead to significant financial losses for organizations through data breaches, reputational damage, customer lawsuits, ransom payments and regulatory fines. Governments are particularly concerned about systemic risks: the knock on or “domino effect” when parts of the digital infrastructure supply chain go offline, causing others to fail or putting new traffic loads of entirely separate systems.
Privacy is also a major issue beginning to affect infrastructure operators — although this is mostly an issue at the application / data storage level. For example, the US Health Insurance Portability and Accountability Act (HIPAA) mandates that data center operators meet specific security standards if their facilities process private healthcare information — and noncompliance can cost $50,000 per violation. Such financial risks often fuel the business case for cybersecurity investments.
What do these investments look like? Many organizations start by conducting cybersecurity risk assessments, which often show that traditional and partial solutions such as firewalls and basic security is not enough. They may also hire new or additional cybersecurity staff and systems to patch vulnerable systems and applications, deploy network segmentation, set up protection against distributed denial-of-service attacks and deploy multifactor authentication for users. Once established, these measures need to be checked against specific regulatory requirements, which may call for specialized software or compliance audits.
The cost of compliance can be significant and recurring because of frequent regulatory and technological changes. Furthermore, the cybersecurity field is currently facing a labor shortage. According to the International Information System Security Certification Consortium (ISC2), there are more than 700,000 unfilled cybersecurity positions in the US alone, which is likely driving the costs higher.
While these investments can be significant for some organizations, there are many potential benefits that extend beyond regulatory compliance. Combined with other investments prompted by regulations, including energy performance improvements, these may pay dividends in preventing potential outages and play a role in elevating the overall resiliency and efficiency of all the systems involved.
The Uptime Intelligence View
Regulatory concerns over resiliency and energy use have led to a wave of new and updated requirements for data centers. Organizations are starting efforts to achieve compliance — and most are prioritizing cybersecurity. While investments in cybersecurity can carry significant costs, threats by malicious actors and financial penalties from noncompliance with regulatory requirements have bolstered the business case for these efforts.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/10/Regulations-drive-investments-in-cybersecurity-and-efficiency-featured.jpg5391030Douglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngDouglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]2023-10-04 13:00:002023-10-02 16:38:42Regulations drive investments in cybersecurity and efficiency
The testing of backup power systems is crucial for ensuring that data center operations remain available through power interruptions. By cutting all power to the facility and replicating a real-world electrical grid failure, pull-the-plug testing provides the most comprehensive assessment of these systems. However, there are some differing opinions on the best way to perform the test and whether the electrical utility company needs to be involved.
Results from the Uptime Institute Data Center Resiliency Survey 2023 found that more than 70% of organizations perform pull-the-plug tests (Figure 1), and of this group, roughly 95% do so at least annually. At the same time, less than half of operators involve their utility company in the process — raising questions over the best practices and the value of some approaches to performing the test.
Operators are not required to notify the utility company of these tests in most cases. This is because it is unlikely that a sudden drop in demand, even from larger data centers, would impact an average-sized grid.
Successful pull-the-plug tests assess a range of operations, including power-loss detection, switchgear, backup generation and the controls needed to connect to on-site power production systems. Depending on the facility design, it may not be possible to fully test all these functions without coordinating with the electrical utility company.
Therefore, organizations that interrupt their power supply independently, without the involvement of the utility, are at risk of performing an incomplete test. And this may give a false sense of security about the facility’s ability to ride through a power outage.
Figure 1. Most data center operators perform pull-the-plug tests
Below are three of the most common approaches for performing a pull-the-plug test and the key considerations for operators when determining which type of test is best suited to their facility.
Coordinating with the electrical utility provider
For this test, the grid provider cuts all incoming power to the data center, prompting the backup power controls to start.
Although this approach guarantees an interruption to the power supply, it can create challenges with costs and scheduling. Because this is a full test of all backup functions, there are some risks. This means it is crucial to have staff with the necessary skills on-site during the test to monitor each step of the procedure and ensure it runs smoothly. This can create scheduling challenges since the test may be constrained by staff availability, including those from suppliers. And because utility providers typically charge fees for their technicians, the costs can increase if unforeseen events, such as severe weather, occur that result in a call for a rescheduling of the test.
Typically, operators have to use this approach when they lack an isolation device — but these carry their own set of challenges.
Using an isolation device to interrupt power
A pull-the-plug test may also be carried out using an isolation device. These are circuit breakers or switches that are deployed upstream of the power transformers. Opening the isolation device cuts the power from the grid to the facility without requiring coordination with the electrical utility company. This approach can cut costs and remove some of the scheduling challenges listed in the previous section, but may not be feasible for some facility designs.
For example, if the opened hardware is monitored by a programmable logic controller (PLC), the generators may start automatically without using (and therefore testing) the controls linked to the power transformer. In this case, the testing of the power-loss detection, the processes for switching devices to on-site power use, and the controls used to coordinate these steps can be bypassed, leading to an incomplete test.
The use of an isolation device can also create new risks. Human error or hardware malfunctions of the device can result in unintended power interruptions or failures to interrupt the power when necessary. Other factors can add to these risks, such as installing the device outside a building and exposing it to extreme weather.
Data center operators that have deployed isolation devices in the initial facility’s design are the most likely to use them to conduct pull-the-plug tests. Those operators that do not have the devices already installed may not want to have them retrofitted due to new concerns, such as spatial challenges — some standards, such as the National Electrical Code in the US, require additional open space around such deployments. Any new installations would also require testing, which would carry all the risks and costs associated with pull-the-plug tests.
Pulling the power transformer fuses
Pulling the power transformer fuses tests all the PLC and backup system hardware required for responding to utility power failures and does not require coordination with the grid provider. However, the power loss to the facility is only simulated and not experienced. The PLC reacts as if an interruption to the power has happened, but a true loss of grid power only occurs once the generator power is active and backup systems are online.
In this case, the uninterruptible power supply (UPS) batteries only discharge for a fraction of the time that they would normally in an actual power outage and are therefore not fully tested. Depending on the PLC design, other ancillary processes may also be skipped and not tested.
However, this approach has many advantages that offset these limitations. It is widely used, particularly by operators of facilities that are the most sensitive to risk. Because the grid power is not interrupted, it can be restored quickly if the equipment malfunctions or human error occurs during the test. And because the UPS batteries are discharged only for a short time, there is less stress and impact on their overall life expectancy.
Facilities that have difficulties with interrupting the power, such as coordinating with the utility or have designs that place staff at risk while opening breakers and switches, also benefit from this approach.
While data center operators have options for pulling the plug, many are unwilling or unable to perform the test. For example, colocation providers and operators of facilities that process critical infrastructure workloads may be restricted over how and when they can pull the plug due to customer contracts.
The Uptime Intelligence View
Uptime Intelligence data has consistently shown that power is the most common cause behind the most significant data center outages, with the failure to switch from the electrical grid to on-site a recurrent problem. At the same time, electrical grids are set to become less reliable. As a result, all operators can benefit from reviewing their pull-the-plug testing procedures with their clients, regardless of whether they involve the energy provider or not, to help ensure resilient backup power systems.
For more details on data center resiliency and outage prevention, Uptime Institute’s Annual Outages Analysis 2023 is available here.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/09/Utility-companies-pull-plug-testing-featured.jpg5391030Douglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngDouglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]2023-09-20 15:00:002023-09-19 15:14:28Are utility companies needed for pull-the-plug testing?
The topic of artificial intelligence (AI) has captured the public’s imaginations, and now barely a week goes by without reports of another breakthrough. Among the many, sometimes dramatic predictions made by experts and non-experts alike is the potential elimination of some, or even many, jobs.
These expectations are partly — but only partly — mirrored in the data center industry: a quarter of respondents to the 2023 Uptime Institute annual data center survey believe that AI will reduce their data center operations staffing levels within the next five years. A much larger group, however, are more cautious, with nearly half believing that jobs will only be displaced over a longer period of time.
These views are inevitably speculative, but a measure of skepticism in the sector is understandable. Despite the hype surrounding large language models, such as ChatGPT, and other generative AI applications, the use cases for these AI tools in data center operations currently appear limited. There are, however, other forms of AI that are already in use in the data center — and have proved valuable — but have not affected any jobs.
AI-based technologies have been the subject of several hype cycles in the past, with their immediate impact always smaller than predicted. This supports the view that the transition to a new generation of software tools is unlikely to be quick, or as far-reaching in the near term, as some AI enthusiasts think.
There are two factors that will likely slow the impact of AI on data center jobs:
The risk profile of most AI-based technologies is currently unacceptable to data center operators.
Those AI-based technologies that have made their way into the data center appear to augment, rather than replace employees.
AI in context
AI is an inconveniently broad umbrella term used to describe computer software that is capable of exhibiting what humans perceive as intelligent behavior. The term includes disciplines such as machine learning (ML), which is concerned with developing complex mathematical models that can learn from data to improve model performance over time.
ML is important in the automation of certain tasks. Once trained on data center operational data, such models can react to events much faster and with more granularity than human employees. This attribute is the foundation for most of the current-generation AI-based data center applications, such as dynamic cooling optimization and equipment health monitoring.
But the term AI also includes plenty of other concepts that span a wide range of applications. The most hotly pursued approaches fashion deep neural networks into complex logic using a training process. Such systems address computational problems that cannot be explicitly expressed in manual programming. High-profile examples are natural language processing, computer vision, search and recommendation systems, and, more recently, generative content systems, such as ChatGPT (text) and Stable Diffusion (text-to-image).
While the current wave of interest, investment and application is unprecedented, there are reasons to look at the latest resurgence of AI with a degree of skepticism. AI is one of the few technologies to have gone through several hype cycles since its origins as an academic discipline in 1956. These periods are often referred to as “AI summers” at the height of excitement and investment in the technology and “AI winters” during the lows.
With faster and more affordable computers, new sources of data for model training, and sensors that enable machines to better understand the physical world, new and innovative AI applications emerge. When researchers reach the technological limits of the day, the funding and interest dries out.
AI applications that prove to be useful are integrated into mainstream software and often stop being considered AI, as part of a phenomenon called “the AI effect.” In the past, this has happened to computers playing chess, optical character recognition, machine translation, email spam filters, satellite navigation systems, and personal digital assistants, such as Siri and Alexa. Applications of AI with bad product-market fit are abandoned.
Data center operators, like other managers across industry, tend to react to the hype cycles with waves of inflated or dampened expectations. In 2019, 29% of respondents to Uptime’s annual survey said they believed that AI would reduce the need for data center staff within the next five years (Figure 1). Nearly five years later, we don’t see any evidence of this reduction taking place.
Figure 1: More operators expect AI to reduce staffing requirements in the near term
AI in the data center
Some AI-based applications have made it into the data center. AI is currently used for dynamic power and cooling optimization, in anomaly detection, predictive maintenance, and other types of predictive analytics.
AI is rarely integrated into data center management tools as a control mechanism. Instead, it is used to advise facility operators. Ceding control of the facility to algorithms or models might make the infrastructure more efficient, but it would also expose the data center to new types of risk — and arguably new single points of failure in the AI mechanism itself. Any mistakes in model design or operation could result in prolonged outages, which could cost millions of dollars. This is not a gamble that operators are currently willing to take.
Increased media coverage of AI has also created more awareness of the faults and limitations that exist within the current generation of AI-based tools, which drives further caution. One such fault that has gained prominence in 2023 is the concept of “artificial hallucinations,” which describes the tendency of generative AI models to occasionally produce confident but inaccurate responses on factual matters. Other issues include the lack of decision-making transparency and accountability (often described as the “black box” problem), and concerns over the security of the data that is provided to train the models.
Nothing is new
It is worth noting that AI has had plenty of time to make inroads into the data center: US company Vigilent — an AI-based tool developer focused on digital infrastructure — has been applying ML in its cooling equipment optimization system since 2008. Some of the vendors to integrate this technology in their data center management tools include Schneider Electric, Siemens and Hitachi Vantara.
Vigilent is not alone in offering this kind of service. Recent entries in the cooling optimization product category include Phaidra in the US (established in 2019) and Coolgradient in Europe (founded in 2021). The former was founded by some members of the DeepMind team, which built an ML model for a Google data center that reportedly cut down the power consumption of cooling equipment by 40%.
What these tools, which represent some of the most successful implementations of AI in data center operations, have in common is their intention to augment humans rather than replace them — they drive cooling systems with a level of detail that the human brain alone would find difficult, if not impossible, to achieve.
The impact on jobs
According to Data Center Career Pathfinder — the tool developed in collaboration between Uptime Institute and some of the world’s largest data center operators — there are at least 25 distinct career options in data center operations and another 25 in operations engineering. These roles include engineers, mechanics, electricians, HVAC technicians, supplier quality managers, environmental health and safety coordinators, and cleaning specialists.
Most operations jobs require a physical presence at the site and interaction with physical equipment within the data center. No matter how intelligent, a software system cannot install a server or fix an ailing generator set.
There are, however, a few data center positions that may be at immediate risk from AI tools. The need for preventative maintenance planners might be reduced since the current generation of AI-based tools can predict failure rates and suggest optimal, condition-based maintenance schedules through advanced statistical methods. There may also be less need for physical security staff: CCTV systems equipped with detection, recognition and tracking features are able to alert the front desk if someone is in the facility without the right credentials. In the future, such systems will get smarter and cover a growing number of threat types through complex pattern recognition in and around the data center.
At the same time, the data center industry is suffering from severe staffing shortages. Half of the respondents to Uptime’s annual survey said they are experiencing difficulties in finding qualified candidates for open jobs. Even if AI-based tools become reliable enough to take over some of the duties of human employees, the likely impact would be to reduce the need for additional hires, offsetting the deficit in staff recruitment, rather than replace those already employed in data centers.
On balance, AI is not going to devour many, if any, jobs in data centers. Equally, it is premature to look to AI as a short-term fix to the industry’s staffing issues. Instead, the data center industry needs to take steps to draw in more staff through advertising the benefits of working in the sector to qualified job seekers, and particularly through attracting the younger cohorts by offering a clear path for training and career progression.
Perhaps this time around AI will really change the fabric of society and the nature of work. In the meantime, developing and deploying smarter AI systems will require a great deal more infrastructure capacity, which will generate new data center jobs before the technology displaces any.
The Uptime Intelligence View
Large language models and generative AI applications are making the headlines but are unlikely to find many uses in data center management and operation. Instead, the current hype cycle might make operators more amenable to better established and understood types of AI — those that have been deployed in data centers over the past 15 years but haven’t reached mainstream adoption. There is little doubt that, eventually, some jobs will be automated out of existence through AI-based software. However, data centers will continue to provide secure employment and operational staff, in particular, will continue to be in high demand.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/09/AI-limited-role-for-now-featured.jpg5391030Max Smolaks, Research Analyst, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngMax Smolaks, Research Analyst, [email protected]2023-09-06 15:00:002023-09-06 09:22:08AI will have a limited role in data centers — for now
In a recent report on server energy efficiency, Uptime Intelligence’s Dr. Tomas Rahkonen analyzed data from 429 servers and identified five key insights (see Server energy efficiency: five key insights). All were valuable observations for better managing (and reducing) IT power consumption, but one area of his analysis stood out: the efficiency benefits of IT power management.
IT power management holds a strange position in modern IT. The technology is mature, well understood, clearly explained by vendors, and is known to reduce IT energy consumption effectively at certain points of load. Many guidelines (including the 2022 Best Practice Guidelines for the EU Code of Conduct on Data Centre Energy Efficiency and Uptime Institute’s sustainability series Digital infrastructure sustainability — a manager’s guide) strongly advocate for its use. And yet very few operators use it.
The reason for this is also widely understood: operational IT managers make the decision to use IT power management and their main task is to ensure IT processing performance is optimized at all times so that it never becomes a problem. Power management, however, is a technology that involves a trade-off between processing power and energy consumption, and this compromise will almost always affect performance. When it comes to power consumption versus compute power, IT managers will almost always favor compute in order to protect application performance.
This is where Dr. Rahkonen’s study becomes important. His analysis (see below for the key findings) details these trade-offs and shows how performance might be affected. Such analysis, Uptime Intelligence believes, should be part of the discussions between facilities and IT managers at the point of procurement and as part of regular efficiency or sustainability reviews.
Power management — the context
The goal of power management is simple: reduce server energy consumption by applying voltage or frequency controls in various ways. The trick is finding the right approach so that IT performance is only minimally affected. That requires some thought about processor types, likely utilization and application needs.
Analysis shows that power management lengthens processing times and latency, but most IT managers have little or no idea by how much. And because most IT managers are striving to consolidate their IT loads and work their machines harder, power management seems like an unnecessary risk.
In his report, Dr. Rahkonen analyzed the data on server performance and energy use from The Green Grid’s publicly available Server Efficiency Rating Tool (SERT) database and drew out two key findings.
First, that server power management can improve server efficiency — which is based on the SERT server-side Java (SSJ) worklet and defined in terms of the SERT energy efficiency metric (SSJ transactions per second per watt) — by up to 19% at the most effective point in the 25% to 60% utilization range. This is a useful finding, not least because it shows the biggest efficiency improvements occur in the utilization range that most operators are striving for.
Despite this finding’s importance, many IT managers won’t initially pay too much attention to the efficiency metric. They care more about absolute performance. This is where Dr. Rahkonen’s second key finding (see Figure 1) is important: even at the worst points of utilization, power management only reduced work (in terms of SSJ transactions per second) by up to 6.4%. Power use reductions were more likely to be in the range of 15% to 21%.
Figure 1. Power management reduces server power and work capacity
What should IT managers make of this information? The main payoff is clear: power management’s impact on performance is demonstrably low, and in most cases, customers will probably not notice that it is turned on. Even at higher points of utilization, the impact on performance is minimal, which suggests that there are likely to be opportunities to both consolidate servers and utilize power management.
It is, of course, not that simple. Conservative IT managers may argue that they still cannot take the risk, especially if certain applications might be affected at key times. This soon becomes a more complex discussion that spans IT architectures, capacity use, types of performance measurement and economics. And latency, not just processor performance, is certainly a worry — more so for some applications and businesses than others.
Such concerns are valid and should be taken into consideration. However, seen through the lens of sustainability and efficiency, there is a clear case for IT operators to evaluate the impact of power management and deploy it where it is technically practical — which will likely be in many situations.
The economic case is possibly even stronger, especially given the recent rises in energy prices. Even at the most efficient facilities, aggregated savings will be considerable, easily rewarding the time and effort spent deploying power management (future Uptime Intelligence reports will have more analysis on the cost impacts of better IT power management).
IT power management has long been overlooked as a means of improving data center efficiency. Uptime Intelligence’s data shows that in most cases, concerns about IT performance are far outweighed by the reduction in energy use. Managers from both IT and facilities will benefit from analyzing the data, applying it to their use cases and, unless there are significant technical and performance issues, using power management as a default.
https://journal.uptimeinstitute.com/wp-content/uploads/2023/08/The-strong-case-for-power-management-featured.jpg5391030Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2023-08-23 15:00:002023-08-22 09:25:21The strong case for power management
Data centers are short-staffed boys’ clubs
/in Executive, Operations/by Jacqueline Davis, Research Analyst, Uptime Institute, [email protected]Two persistent trends in data center staffing are in apparent tension. The 2023 Uptime Institute Global Data Center Survey confirmed, once again, that operations teams are struggling to attract and retain qualified staff. The severity of this shortage should justify aggressive hiring from all available labor sources — yet data centers still employ shockingly few women. The average proportion of female employees at respondents’ organizations is just 8% — lower than in many physically demanding, conventionally male-dominated industries, such as construction, mining and manufacturing.
In the Uptime Institute Global Data Center Survey 2023 report, Uptime details the staffing shortage that has frustrated the data center industry for more than a decade. Operator responses show that the past four years have been particularly trying for the sector. About half of the survey’s respondents reported difficulty in filling open job positions, and one in four have seen their staff hired away, with most being poached by competitors. Skills shortages affect virtually all job roles but are the most acute among operations, mechanical and electrical staff.
While researching the staffing demands of data center design, build, and operations teams, Uptime has collected gender data since 2018. Figure 1 shows that the majority (81%) of data center teams are overwhelmingly male, with women making up one in 10 workers or fewer. On average, a meager 8% of all teams are made up of women. This year, Uptime calculated this weighted average for the first time, and five years of data reveals no significant change.
Figure 1 Data center teams lack gender diversity
In the 2019 questionnaire regarding hiring initiatives, good intentions abound. Almost three in four respondents agreed the industry would benefit from hiring more women — and 45% said the gender imbalance was a threat to the industry, contributing to the dry talent pipeline and even technical stagnation. Nearly half of respondents were planning initiatives to hire more women — or had such programs already in place. These results suggested a will to invest in hiring women, and an expectation of that investment paying off.
Four years on, the staff shortfall looms large in keynote presentations and data halls alike. So, where are the women? Did the time and money invested in hiring initiatives fail to attract female candidates, or were these plans and initiatives quietly shelved? Regardless of the causes, the gender divide persists, and the benefits of mitigating it remain hypothetical.
Identifying the most relevant influences on the gender balance of the workforce is difficult without any motion in the data — but some explanations can be ruled out by comparison against other industries.
Anecdotally, women are disinclined to pursue manual labor — and labor statistics confirm that physically demanding job roles typically attract fewer women. However, this alone cannot account for the disparity in the data halls. Uptime’s findings suggest data center operations teams employ fewer women than manufacturing (29%), mining (15%) and construction (11%) — according to figures from a 2021 publication by the US Bureau of Labor Statistics (Employment and Wages Online Annual Averages). The construction workforce in western Europe is about 9% women, as per government statistical offices, and the International Energy Agency reports at least 10% women in the mining industry in all regions.
In occupational health and safety research, women describe construction sites as hostile workplaces. Women endure isolation, sexual harassment, and bullying, as well as injuries from tools and personal protective equipment designed for the average size, weight, and grip strength of a male worker. In interviews, women say they experience little job security in construction, and this is discouraging timely reporting of harassment or injuries, and frustrating efforts at improvement. If female workers are choosing such punishing job sites over data centers (even by a small margin), there must be other factors influencing the minimal presence of women — beyond the assumed aversion to physical labor.
Data center operations are not a highly visible career path, and the problem is more pronounced for women. With few female role models in the industry, women may perceive a data center career as unwelcoming or unsafe and may feel discouraged from applying. This cycle is likely to perpetuate itself unless more resources are devoted specifically to closing the gender gap.
Many data center operators now desire better visibility in the labor market and are bringing their organizations out of their habit of comfortable obscurity. To draw in more workers, the industry will need to effectively communicate that it has rewarding careers available for qualified women. An uptick in female representation would strengthen this message and could act as an indicator of early successes for recruitment efforts at large.
To draw future workers from universities and trade schools, some data center organizations have formed educational partnerships and programs — often working with high school students, or younger. Efforts in education to attract more women and girls to careers in data centers must begin today, to produce more gender diversity when these students grow into job seekers.
The data center industry also seeks out workers who are changing careers and can bring some applicable skills with them. Operators can smooth the transition into a data center career by revisiting the advertised job requirements — lowering the bar for entry so candidates can supplement their existing skill sets with training on the job and mentoring programs. Active outreach to women in this talent pool can ensure that fewer qualified candidates are overlooked.
This avenue of recruitment can expand to include “career returners.” Many in this group are women who left the workforce for childcare or family care duties — and operators offering benefits suited to workers with family obligations may gain an advantage in recruitment.
Operators desperate for staff would benefit from understanding what went wrong with female representation in the industry. Ongoing efforts to recruit women must be re-examined and held to account for a lack of return on time and money invested.
To lessen the data center staffing struggle, operators will need to draw from every available labor pool. Women and other underrepresented groups are underutilized resources — and the industry likely will not escape its staffing crisis until it can bring them in.
The Uptime Intelligence View
Gender balance in the data center lags many physically demanding, conventionally male-dominated industries such as construction, mining, and manufacturing — a sign the industry is not doing enough to advertise rewarding career opportunities to women. Data centers cannot afford to let female candidates remain left out. Outreach efforts to career changers, universities, and vocational training could maximize their returns by seeking out more women.
Consensus on regulatory goals hides national differences
/in Executive, Operations/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]In recent reports, Uptime Institute Intelligence has warned that a wave of resiliency, security and sustainability legislation is making its way toward the statute books. Governments around the world — aware that digital infrastructure is increasingly critical to economic and national security (and consumes a lot of power) — have decided the sector cannot be left unwatched and unmanaged.
New laws relating to data center sustainability have attracted the most attention, partly because some of the provisions will likely prove expensive or difficult to meet. In Germany, for example, the Energy Efficiency Act, which was passed by Germany’s lower house of parliament in September 2023, requires all new data centers to reuse a set proportion of their waste heat (with some exceptions) and have a power usage effectiveness (PUE) of 1.3 or below. The legislation also specifies that older data centers will be required to reach this level by 2026.
The act, which has been dubbed by some as the “data center prevention act,” is the first of many new laws planned by European governments. It anticipates (and adds to) the requirements of the EU’s Energy Efficiency Directive (EED), which comes into force in the latter part of 2023. The EED contains a long list of onerous reporting and improvement requirements for data centers that will be transposed into national law in all 27 EU member states.
Sustainability is not, however, the focus of a lot of the upcoming regulation. A recent Uptime Institute report, Digital resiliency: global trends in regulation, looks at how legislation addressing resiliency across the digital supply chain is being implemented or planned in the US, EU, UK, Singapore, Australia and elsewhere. At least some of these laws will have far-reaching effects on the digital infrastructure ecosystem.
While rules to ensure resiliency are not new, the latest wave signals a significant extension of regulatory oversight. The new laws are a response to the growing threat of complex, systemic or even irrecoverable outages and are a recognition that data center services now play a critical role in a modern economy — a lesson underlined during the COVID-19 pandemic.
A common theme of these new rules is that, effectively, governments are now classifying digital infrastructure as part of the critical national infrastructure. This is a specific term that means operators of critical services are subject to security, availability, transparency and reporting mandates. Under these regulations, all participants in the infrastructure supply chain — whether software, hosting, colocation, cloud or networking — need to be transparent and accountable.
Uptime Intelligence research suggests that few operators are up to date with pending legislation or the requirements and costs of compliance. In the area of sustainability, surveys show that most data center / digital infrastructure operators are not collecting the data they will soon need to report.
Operators in North America (primarily the US) tend to be much more wary of regulation than their counterparts in other parts of the world — wariness that applies both to sustainability and to resiliency / transparency. In a 2021 Uptime Institute climate change survey, for example, three-quarters of European and Asian data center operators said they thought data center sustainability laws were needed — but only about 40% of US operators agreed (see Figure 1).
Figure 1. Majority invite more regulation for sustainability
In the 2023 Uptime Institute Global Data Center Survey, operators were asked about their attitude toward resiliency laws that would reveal more details of data center infrastructures and would enable certain customers (i.e., clients with critical requirements) to visit or assess facilities for resiliency. Operators everywhere were broadly supportive, but once again, those in North America were the most wary (see Figure 2).
Figure 2. North American operators least likely to favor transparency laws
Differences in attitudes toward the role of regulation are not limited to businesses; regulators too differ in their aims and methods. The US, for example, generally puts more emphasis on economic incentives, partly through the tax system, such as the Inflation Reduction Act, while Europe favors the stick — rules must be followed, or penalties will follow.
In the US, when resiliency laws are introduced or proposed, for example by the Securities and Exchange Commission, they are not expected to be backed up by tough sanctions for failures to comply — unlike in the EU, where penalties can be high. Instead, organizations are encouraged to conform, or they will be prevented from bidding for certain government contracts.
And while Europe and parts of Asia (such as China and Singapore) have tough sustainability laws in the pipeline, the US has no federal laws planned.
There is, of course, a long-running debate over whether expensive carrots (the US model) or punitive, bureaucratic sticks are the most effective methods in facilitating change. Evidence from the Uptime Institute annual survey does show that regulations drive some investments (see Regulations drive investments in cybersecurity and efficiency). But, of course, so do rules that require investments in equipment.
For data center owners and operators, the end result may not be so different. High levels of resiliency and transparency are mostly expected to be rewarded in law and in the market, as are energy efficiency and low carbon emissions.
However, the incentive model may cost less for operators because of generous rebates and exemptions — Uptime estimates that fulfilling the emerging reporting requirements for sustainability and resiliency can cost upwards of $100,000.
The Uptime Intelligence View
The role and extent of regulation — and of incentives — is likely to change constantly in the next decade, making it difficult for data center operators to formulate a clear strategy. The most successful data center owners and operators will be those that aim for high standards at an early stage, in both areas of resiliency and sustainability, and invest accordingly. The business case for such investments is becoming ever stronger — in all geographies.
Regulations drive investments in cybersecurity and efficiency
/in Executive, Operations/by Douglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]Legislative requirements for data center resiliency, operational transparency and energy performance are tightening worldwide — putting data centers under greater regulatory scrutiny. In response, organizations are either starting or stepping up their efforts to achieve compliance in these areas, and findings from the Uptime Institute Global Data Center Survey 2023 reveal that most are prioritizing cybersecurity (see Figure 1).
Figure 1. Regulations drive security, hardware and efficiency investments
Since 2020, several countries have introduced laws with strict cybersecurity demands for data center operators to combat the rise in cyber threats (see Table 1) — especially if they host or manage critical national infrastructure (CNI) workloads. As CNI entities become more reliant on digital services, they are increasingly exposed to cyber risks that could result in severe consequences. For example, a compromised facility managing applications for a utility risks widespread power and communications outages, threatening the physical safety of citizens.
Table 1. Regulations that mandate enhanced cybersecurity measures
Cyberattacks are becoming increasingly sophisticated as the digital infrastructure becomes more interconnected. For example, operational technology systems for power and cooling optimization are routinely connected to the internet (either directly or indirectly), which creates a broader “attack surface,” giving more access points for cyberattacks. Operators are also increasingly deploying Internet of Things devices and applications. These are used for asset tracking, predictive maintenance and capacity planning, but they require network connectivity and can lack robust cybersecurity features.
Measures aimed at improving energy efficiency rank as the second and third most popular responses to new regulations (see Figure 1). To evaluate their progress, data center operators may add new energy management systems and network connections to the power infrastructure, potentially complicating existing cybersecurity programs.
Alongside the risks to CNI, cyberattacks could lead to significant financial losses for organizations through data breaches, reputational damage, customer lawsuits, ransom payments and regulatory fines. Governments are particularly concerned about systemic risks: the knock on or “domino effect” when parts of the digital infrastructure supply chain go offline, causing others to fail or putting new traffic loads of entirely separate systems.
Privacy is also a major issue beginning to affect infrastructure operators — although this is mostly an issue at the application / data storage level. For example, the US Health Insurance Portability and Accountability Act (HIPAA) mandates that data center operators meet specific security standards if their facilities process private healthcare information — and noncompliance can cost $50,000 per violation. Such financial risks often fuel the business case for cybersecurity investments.
What do these investments look like? Many organizations start by conducting cybersecurity risk assessments, which often show that traditional and partial solutions such as firewalls and basic security is not enough. They may also hire new or additional cybersecurity staff and systems to patch vulnerable systems and applications, deploy network segmentation, set up protection against distributed denial-of-service attacks and deploy multifactor authentication for users. Once established, these measures need to be checked against specific regulatory requirements, which may call for specialized software or compliance audits.
The cost of compliance can be significant and recurring because of frequent regulatory and technological changes. Furthermore, the cybersecurity field is currently facing a labor shortage. According to the International Information System Security Certification Consortium (ISC2), there are more than 700,000 unfilled cybersecurity positions in the US alone, which is likely driving the costs higher.
While these investments can be significant for some organizations, there are many potential benefits that extend beyond regulatory compliance. Combined with other investments prompted by regulations, including energy performance improvements, these may pay dividends in preventing potential outages and play a role in elevating the overall resiliency and efficiency of all the systems involved.
The Uptime Intelligence View
Regulatory concerns over resiliency and energy use have led to a wave of new and updated requirements for data centers. Organizations are starting efforts to achieve compliance — and most are prioritizing cybersecurity. While investments in cybersecurity can carry significant costs, threats by malicious actors and financial penalties from noncompliance with regulatory requirements have bolstered the business case for these efforts.
Are utility companies needed for pull-the-plug testing?
/in Executive, Operations/by Douglas Donnellan, Senior Research Associate, Uptime Institute, [email protected]The testing of backup power systems is crucial for ensuring that data center operations remain available through power interruptions. By cutting all power to the facility and replicating a real-world electrical grid failure, pull-the-plug testing provides the most comprehensive assessment of these systems. However, there are some differing opinions on the best way to perform the test and whether the electrical utility company needs to be involved.
Results from the Uptime Institute Data Center Resiliency Survey 2023 found that more than 70% of organizations perform pull-the-plug tests (Figure 1), and of this group, roughly 95% do so at least annually. At the same time, less than half of operators involve their utility company in the process — raising questions over the best practices and the value of some approaches to performing the test.
Operators are not required to notify the utility company of these tests in most cases. This is because it is unlikely that a sudden drop in demand, even from larger data centers, would impact an average-sized grid.
Successful pull-the-plug tests assess a range of operations, including power-loss detection, switchgear, backup generation and the controls needed to connect to on-site power production systems. Depending on the facility design, it may not be possible to fully test all these functions without coordinating with the electrical utility company.
Therefore, organizations that interrupt their power supply independently, without the involvement of the utility, are at risk of performing an incomplete test. And this may give a false sense of security about the facility’s ability to ride through a power outage.
Figure 1. Most data center operators perform pull-the-plug tests
Below are three of the most common approaches for performing a pull-the-plug test and the key considerations for operators when determining which type of test is best suited to their facility.
Coordinating with the electrical utility provider
For this test, the grid provider cuts all incoming power to the data center, prompting the backup power controls to start.
Although this approach guarantees an interruption to the power supply, it can create challenges with costs and scheduling. Because this is a full test of all backup functions, there are some risks. This means it is crucial to have staff with the necessary skills on-site during the test to monitor each step of the procedure and ensure it runs smoothly. This can create scheduling challenges since the test may be constrained by staff availability, including those from suppliers. And because utility providers typically charge fees for their technicians, the costs can increase if unforeseen events, such as severe weather, occur that result in a call for a rescheduling of the test.
Typically, operators have to use this approach when they lack an isolation device — but these carry their own set of challenges.
Using an isolation device to interrupt power
A pull-the-plug test may also be carried out using an isolation device. These are circuit breakers or switches that are deployed upstream of the power transformers. Opening the isolation device cuts the power from the grid to the facility without requiring coordination with the electrical utility company. This approach can cut costs and remove some of the scheduling challenges listed in the previous section, but may not be feasible for some facility designs.
For example, if the opened hardware is monitored by a programmable logic controller (PLC), the generators may start automatically without using (and therefore testing) the controls linked to the power transformer. In this case, the testing of the power-loss detection, the processes for switching devices to on-site power use, and the controls used to coordinate these steps can be bypassed, leading to an incomplete test.
The use of an isolation device can also create new risks. Human error or hardware malfunctions of the device can result in unintended power interruptions or failures to interrupt the power when necessary. Other factors can add to these risks, such as installing the device outside a building and exposing it to extreme weather.
Data center operators that have deployed isolation devices in the initial facility’s design are the most likely to use them to conduct pull-the-plug tests. Those operators that do not have the devices already installed may not want to have them retrofitted due to new concerns, such as spatial challenges — some standards, such as the National Electrical Code in the US, require additional open space around such deployments. Any new installations would also require testing, which would carry all the risks and costs associated with pull-the-plug tests.
Pulling the power transformer fuses
Pulling the power transformer fuses tests all the PLC and backup system hardware required for responding to utility power failures and does not require coordination with the grid provider. However, the power loss to the facility is only simulated and not experienced. The PLC reacts as if an interruption to the power has happened, but a true loss of grid power only occurs once the generator power is active and backup systems are online.
In this case, the uninterruptible power supply (UPS) batteries only discharge for a fraction of the time that they would normally in an actual power outage and are therefore not fully tested. Depending on the PLC design, other ancillary processes may also be skipped and not tested.
However, this approach has many advantages that offset these limitations. It is widely used, particularly by operators of facilities that are the most sensitive to risk. Because the grid power is not interrupted, it can be restored quickly if the equipment malfunctions or human error occurs during the test. And because the UPS batteries are discharged only for a short time, there is less stress and impact on their overall life expectancy.
Facilities that have difficulties with interrupting the power, such as coordinating with the utility or have designs that place staff at risk while opening breakers and switches, also benefit from this approach.
While data center operators have options for pulling the plug, many are unwilling or unable to perform the test. For example, colocation providers and operators of facilities that process critical infrastructure workloads may be restricted over how and when they can pull the plug due to customer contracts.
The Uptime Intelligence View
Uptime Intelligence data has consistently shown that power is the most common cause behind the most significant data center outages, with the failure to switch from the electrical grid to on-site a recurrent problem. At the same time, electrical grids are set to become less reliable. As a result, all operators can benefit from reviewing their pull-the-plug testing procedures with their clients, regardless of whether they involve the energy provider or not, to help ensure resilient backup power systems.
For more details on data center resiliency and outage prevention, Uptime Institute’s Annual Outages Analysis 2023 is available here.
AI will have a limited role in data centers — for now
/in Design, Executive, Operations/by Max Smolaks, Research Analyst, [email protected]The topic of artificial intelligence (AI) has captured the public’s imaginations, and now barely a week goes by without reports of another breakthrough. Among the many, sometimes dramatic predictions made by experts and non-experts alike is the potential elimination of some, or even many, jobs.
These expectations are partly — but only partly — mirrored in the data center industry: a quarter of respondents to the 2023 Uptime Institute annual data center survey believe that AI will reduce their data center operations staffing levels within the next five years. A much larger group, however, are more cautious, with nearly half believing that jobs will only be displaced over a longer period of time.
These views are inevitably speculative, but a measure of skepticism in the sector is understandable. Despite the hype surrounding large language models, such as ChatGPT, and other generative AI applications, the use cases for these AI tools in data center operations currently appear limited. There are, however, other forms of AI that are already in use in the data center — and have proved valuable — but have not affected any jobs.
AI-based technologies have been the subject of several hype cycles in the past, with their immediate impact always smaller than predicted. This supports the view that the transition to a new generation of software tools is unlikely to be quick, or as far-reaching in the near term, as some AI enthusiasts think.
There are two factors that will likely slow the impact of AI on data center jobs:
AI in context
AI is an inconveniently broad umbrella term used to describe computer software that is capable of exhibiting what humans perceive as intelligent behavior. The term includes disciplines such as machine learning (ML), which is concerned with developing complex mathematical models that can learn from data to improve model performance over time.
ML is important in the automation of certain tasks. Once trained on data center operational data, such models can react to events much faster and with more granularity than human employees. This attribute is the foundation for most of the current-generation AI-based data center applications, such as dynamic cooling optimization and equipment health monitoring.
But the term AI also includes plenty of other concepts that span a wide range of applications. The most hotly pursued approaches fashion deep neural networks into complex logic using a training process. Such systems address computational problems that cannot be explicitly expressed in manual programming. High-profile examples are natural language processing, computer vision, search and recommendation systems, and, more recently, generative content systems, such as ChatGPT (text) and Stable Diffusion (text-to-image).
While the current wave of interest, investment and application is unprecedented, there are reasons to look at the latest resurgence of AI with a degree of skepticism. AI is one of the few technologies to have gone through several hype cycles since its origins as an academic discipline in 1956. These periods are often referred to as “AI summers” at the height of excitement and investment in the technology and “AI winters” during the lows.
With faster and more affordable computers, new sources of data for model training, and sensors that enable machines to better understand the physical world, new and innovative AI applications emerge. When researchers reach the technological limits of the day, the funding and interest dries out.
AI applications that prove to be useful are integrated into mainstream software and often stop being considered AI, as part of a phenomenon called “the AI effect.” In the past, this has happened to computers playing chess, optical character recognition, machine translation, email spam filters, satellite navigation systems, and personal digital assistants, such as Siri and Alexa. Applications of AI with bad product-market fit are abandoned.
Data center operators, like other managers across industry, tend to react to the hype cycles with waves of inflated or dampened expectations. In 2019, 29% of respondents to Uptime’s annual survey said they believed that AI would reduce the need for data center staff within the next five years (Figure 1). Nearly five years later, we don’t see any evidence of this reduction taking place.
Figure 1: More operators expect AI to reduce staffing requirements in the near term
AI in the data center
Some AI-based applications have made it into the data center. AI is currently used for dynamic power and cooling optimization, in anomaly detection, predictive maintenance, and other types of predictive analytics.
AI is rarely integrated into data center management tools as a control mechanism. Instead, it is used to advise facility operators. Ceding control of the facility to algorithms or models might make the infrastructure more efficient, but it would also expose the data center to new types of risk — and arguably new single points of failure in the AI mechanism itself. Any mistakes in model design or operation could result in prolonged outages, which could cost millions of dollars. This is not a gamble that operators are currently willing to take.
Increased media coverage of AI has also created more awareness of the faults and limitations that exist within the current generation of AI-based tools, which drives further caution. One such fault that has gained prominence in 2023 is the concept of “artificial hallucinations,” which describes the tendency of generative AI models to occasionally produce confident but inaccurate responses on factual matters. Other issues include the lack of decision-making transparency and accountability (often described as the “black box” problem), and concerns over the security of the data that is provided to train the models.
Nothing is new
It is worth noting that AI has had plenty of time to make inroads into the data center: US company Vigilent — an AI-based tool developer focused on digital infrastructure — has been applying ML in its cooling equipment optimization system since 2008. Some of the vendors to integrate this technology in their data center management tools include Schneider Electric, Siemens and Hitachi Vantara.
Vigilent is not alone in offering this kind of service. Recent entries in the cooling optimization product category include Phaidra in the US (established in 2019) and Coolgradient in Europe (founded in 2021). The former was founded by some members of the DeepMind team, which built an ML model for a Google data center that reportedly cut down the power consumption of cooling equipment by 40%.
What these tools, which represent some of the most successful implementations of AI in data center operations, have in common is their intention to augment humans rather than replace them — they drive cooling systems with a level of detail that the human brain alone would find difficult, if not impossible, to achieve.
The impact on jobs
According to Data Center Career Pathfinder — the tool developed in collaboration between Uptime Institute and some of the world’s largest data center operators — there are at least 25 distinct career options in data center operations and another 25 in operations engineering. These roles include engineers, mechanics, electricians, HVAC technicians, supplier quality managers, environmental health and safety coordinators, and cleaning specialists.
Most operations jobs require a physical presence at the site and interaction with physical equipment within the data center. No matter how intelligent, a software system cannot install a server or fix an ailing generator set.
There are, however, a few data center positions that may be at immediate risk from AI tools. The need for preventative maintenance planners might be reduced since the current generation of AI-based tools can predict failure rates and suggest optimal, condition-based maintenance schedules through advanced statistical methods. There may also be less need for physical security staff: CCTV systems equipped with detection, recognition and tracking features are able to alert the front desk if someone is in the facility without the right credentials. In the future, such systems will get smarter and cover a growing number of threat types through complex pattern recognition in and around the data center.
At the same time, the data center industry is suffering from severe staffing shortages. Half of the respondents to Uptime’s annual survey said they are experiencing difficulties in finding qualified candidates for open jobs. Even if AI-based tools become reliable enough to take over some of the duties of human employees, the likely impact would be to reduce the need for additional hires, offsetting the deficit in staff recruitment, rather than replace those already employed in data centers.
On balance, AI is not going to devour many, if any, jobs in data centers. Equally, it is premature to look to AI as a short-term fix to the industry’s staffing issues. Instead, the data center industry needs to take steps to draw in more staff through advertising the benefits of working in the sector to qualified job seekers, and particularly through attracting the younger cohorts by offering a clear path for training and career progression.
Perhaps this time around AI will really change the fabric of society and the nature of work. In the meantime, developing and deploying smarter AI systems will require a great deal more infrastructure capacity, which will generate new data center jobs before the technology displaces any.
The Uptime Intelligence View
Large language models and generative AI applications are making the headlines but are unlikely to find many uses in data center management and operation. Instead, the current hype cycle might make operators more amenable to better established and understood types of AI — those that have been deployed in data centers over the past 15 years but haven’t reached mainstream adoption. There is little doubt that, eventually, some jobs will be automated out of existence through AI-based software. However, data centers will continue to provide secure employment and operational staff, in particular, will continue to be in high demand.
The strong case for power management
/in Design, Executive, Operations/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]ANALYST OPINION
In a recent report on server energy efficiency, Uptime Intelligence’s Dr. Tomas Rahkonen analyzed data from 429 servers and identified five key insights (see Server energy efficiency: five key insights). All were valuable observations for better managing (and reducing) IT power consumption, but one area of his analysis stood out: the efficiency benefits of IT power management.
IT power management holds a strange position in modern IT. The technology is mature, well understood, clearly explained by vendors, and is known to reduce IT energy consumption effectively at certain points of load. Many guidelines (including the 2022 Best Practice Guidelines for the EU Code of Conduct on Data Centre Energy Efficiency and Uptime Institute’s sustainability series Digital infrastructure sustainability — a manager’s guide) strongly advocate for its use. And yet very few operators use it.
The reason for this is also widely understood: operational IT managers make the decision to use IT power management and their main task is to ensure IT processing performance is optimized at all times so that it never becomes a problem. Power management, however, is a technology that involves a trade-off between processing power and energy consumption, and this compromise will almost always affect performance. When it comes to power consumption versus compute power, IT managers will almost always favor compute in order to protect application performance.
This is where Dr. Rahkonen’s study becomes important. His analysis (see below for the key findings) details these trade-offs and shows how performance might be affected. Such analysis, Uptime Intelligence believes, should be part of the discussions between facilities and IT managers at the point of procurement and as part of regular efficiency or sustainability reviews.
Power management — the context
The goal of power management is simple: reduce server energy consumption by applying voltage or frequency controls in various ways. The trick is finding the right approach so that IT performance is only minimally affected. That requires some thought about processor types, likely utilization and application needs.
Analysis shows that power management lengthens processing times and latency, but most IT managers have little or no idea by how much. And because most IT managers are striving to consolidate their IT loads and work their machines harder, power management seems like an unnecessary risk.
In his report, Dr. Rahkonen analyzed the data on server performance and energy use from The Green Grid’s publicly available Server Efficiency Rating Tool (SERT) database and drew out two key findings.
First, that server power management can improve server efficiency — which is based on the SERT server-side Java (SSJ) worklet and defined in terms of the SERT energy efficiency metric (SSJ transactions per second per watt) — by up to 19% at the most effective point in the 25% to 60% utilization range. This is a useful finding, not least because it shows the biggest efficiency improvements occur in the utilization range that most operators are striving for.
Despite this finding’s importance, many IT managers won’t initially pay too much attention to the efficiency metric. They care more about absolute performance. This is where Dr. Rahkonen’s second key finding (see Figure 1) is important: even at the worst points of utilization, power management only reduced work (in terms of SSJ transactions per second) by up to 6.4%. Power use reductions were more likely to be in the range of 15% to 21%.
Figure 1. Power management reduces server power and work capacity
What should IT managers make of this information? The main payoff is clear: power management’s impact on performance is demonstrably low, and in most cases, customers will probably not notice that it is turned on. Even at higher points of utilization, the impact on performance is minimal, which suggests that there are likely to be opportunities to both consolidate servers and utilize power management.
It is, of course, not that simple. Conservative IT managers may argue that they still cannot take the risk, especially if certain applications might be affected at key times. This soon becomes a more complex discussion that spans IT architectures, capacity use, types of performance measurement and economics. And latency, not just processor performance, is certainly a worry — more so for some applications and businesses than others.
Such concerns are valid and should be taken into consideration. However, seen through the lens of sustainability and efficiency, there is a clear case for IT operators to evaluate the impact of power management and deploy it where it is technically practical — which will likely be in many situations.
The economic case is possibly even stronger, especially given the recent rises in energy prices. Even at the most efficient facilities, aggregated savings will be considerable, easily rewarding the time and effort spent deploying power management (future Uptime Intelligence reports will have more analysis on the cost impacts of better IT power management).
IT power management has long been overlooked as a means of improving data center efficiency. Uptime Intelligence’s data shows that in most cases, concerns about IT performance are far outweighed by the reduction in energy use. Managers from both IT and facilities will benefit from analyzing the data, applying it to their use cases and, unless there are significant technical and performance issues, using power management as a default.