When the PUE (power usage effectiveness) metric was first discussed at a meeting of The Green Grid in Santa Clara, back in 2007, a microphone stand was placed in each aisle of the auditorium. The importance of the initiative was understood even then: experts, including the founders of Uptime Institute, formed lines to give their considered input. And if there was one point that came across, it was that the industry should not treat PUE as a comparative metric, a gold standard that every designer, operator and planning authority must chase down, for comparison, presentation and applause.
Thirteen years on, that is almost exactly what has happened. PUE, thanks to its simplicity and universal applicability, has become the critical benchmark for scoring data centers for efficiency and “greenness” (environmental responsibility). But it is often used uncritically.
This has had both positive and negative consequences. The positive side is that PUE values have been forced down across the world, as operators strive to reduce waste and operating costs. To meet their goals, suppliers have focused on improving electrical efficiency of equipment and have developed nonmechanical alternatives (such as free cooling). PUEs have fallen from an average of 2.5 in 2007 to around 1.6 today. In an Uptime Institute survey, around 95% of respondents said it is important that colocation companies have a low PUE.
So what is the negative side? Its simplicity. The very quality that led to PUE’s near universal adoption also may have led to its being over-used and misapplied — increasingly, with unintended consequences.
This observation is hardly new — indeed, this is exactly what those experts feared back in 2007 as they lined up to voice their concerns. And it is why PUE was developed into a standard, with rules on how it should be measured, and why Partial PUE variants were developed.
But the unintended consequences are becoming more serious and more concerning. The metric may be driving the wrong behaviors. For this reason, it is very possible that PUE values may begin to be de-emphasized in the near future and more operators may allow their number to rise a little. (In 2019, Uptime Institute did note a marginal rise in PUE values.) If this happens, some operators may find themselves in conflict with certain regulators and planners, for whom PUE has been an easily understood but simplistic way of enforcing environmental goals (policing this is another matter). These planning authorities, then, may also need to review their policies around driving down the PUE numbers. (California, Singapore, Amsterdam, take note.)
There are, we think, three strong and strengthening reasons why focusing on the PUE of the data center might be counter-productive:
Resiliency (N to N+1). There is an industry-wide move to increase the resiliency of data centers, with the growing dependency on IT and the impact of the pandemic and extreme weather among the drivers. At the physical site level, it is difficult to improve redundancy and resiliency without adding power-consuming equipment — which drives up the PUE. (Similarly, one operator installed the highest level of MERV [Minimum Efficiency Reporting Value] air filters to filter out viruses — but fan energy use rose by 8% and the PUE value rose accordingly.)
Water use. The drive to improve energy efficiency has encouraged operators to use economizers and, in particular, adiabatic and evaporative cooling and chillers. But this has often involved a trade-off: energy use drops, but water use rises. That compromise is becoming problematic in some regions, where climate change means water is becoming more scarce and, in some instances, a bigger local environmental issue than on-site energy use. Again, PUE may be the wrong metric to track — WUE (water usage effectiveness) is a more important one.
IT efficiency. It is widely known that while facilities have become ever more energy efficient, IT use can be extremely inefficient and wasteful, with processors and memory often drawing energy while doing relatively little work. Energy-saving efficiency investments on the IT side, however, are not rewarded with a lower PUE, but rather a higher one (as PUE is the ratio of the IT to non-IT energy figures). This has been known from the outset. With a slew of new technologies becoming available to lower the overall energy figure, it is important that the need to meet a given PUE value does not discourage investments and innovation on the IT side.
There is, to be fair, a good counter-argument to these three points: that by searching for ways to lower the PUE regardless of these constraints, data center operators can find ways to meet all the goals. Liquid cooling, for example, can be highly resilient and uses little water. The argument here, then, is not for anyone to abandon PUE, or to stop watching and measuring it, but to apply it more cautiously than ever, and more flexibly.
https://journal.uptimeinstitute.com/wp-content/uploads/2020/11/PUE-Rusty.jpg380895Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2020-11-02 07:32:042020-10-26 12:18:10PUE: The golden metric is looking rusty
When Uptime Institute recently asked over 300 data center managers how the pandemic would change their operations, one answer stood out: Two-thirds expect to increase the resiliency of their core data center(s) in the years ahead. Many said they expected their costs to increase as a result.
The reasoning is clear: the pandemic — or any future one — can mean operating with fewer staff, and possibly with disrupted service and supply chains. Remote monitoring and preventive maintenance will help to reduce the likelihood of an incident, but machines will always fail. It makes sense to reduce the impact of failure by increasing system redundancy.
But even before the pandemic, there was a trend toward higher levels of redundancy. As shown in the figure below, roughly half of those participating in the Uptime Institute 2020 global survey of suppliers, designers and advisors reported their customers have increased redundancy levels in the last three to five years.
This trend may seem unsurprising to some, but it was not entirely predictable. The growth of cloud has been accompanied by the much greater use of multisite resiliency and regional availability zones. In theory, at least, these substantially reduce the impact of single-site facility outages, because traffic and workloads can be diverted elsewhere. Backed by this capability, some operators — Facebook is an example — have proceeded with lower levels of redundancy than was common in the past (thereby saving costs and energy).
The use of availability zones, however, has come with its own problems, with networking and software issues often causing service interruptions. And the loss of one data center immediately places capacity and traffic demand on others, heightening risks. For this reason, even the big cloud providers and internet application operators manage mostly concurrently maintainable facilities, and it is common for them to stipulate that colocation partners have N+2 level facilities.
With a variety of options, the overall shift to increased resiliency is still slow and quite nuanced, with designers mostly favoring either N+1 or N+2 configurations, according to site and business needs, and, often, according to the creativity of the designers. Overall, there is actually a marginal decrease in the number of data centers that are 2N, but a steady three-year shift from N+1 to N+2 — not only in power, but also in cooling (see figure below). There is also an increase in the use of active-active availability zones, as discussed in our recent report Uptime Institute global data center survey 2020.
Demand patterns and growing IT dependency partly account for these higher levels of redundancy/resiliency. The level of resiliency needed for each service or by each customer is dictated by business requirements, but this is not fixed in time. The growing criticality of many IT services highlights the importance of mitigating risk through increased resiliency. “Creeping criticality” — a situation in which infrastructure and processes have not been upgraded or updated to reflect the growing criticality of the applications or business processes they support — may require redundancy upgrades.
Uptime Institute expects operators to make more use of distributed resiliency in the future — especially as more workloads are designed using cloud or microservices architectures (workloads are more portable, and instances are more easily copied). But there is no sign that this is diminishing the need for site-level resiliency. The software running these distributed services is often opaque, complex and may be prone to errors of programming or configuration. Annual outage data shows these types of issues are proliferating. Further, any big component failures can cascade, making recovery difficult and expensive, with data and applications synchronized across multiple sites.
The trend for now is clear: more resiliency at every level is the least risky approach — even if it means some extra expense and duplication of effort.
https://journal.uptimeinstitute.com/wp-content/uploads/2020/10/type1a.jpg3551475Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]https://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngAndy Lawrence, Executive Director of Research, Uptime Institute, [email protected]2020-10-26 06:00:342021-05-04 10:19:35Why data center operators are investing in more redundancy
In mid-September 2020, The US Bureau of Labor and Statistics published their updated 2019-2029 Employment Projection summary news release and the associated handbook which discusses various job roles and hiring and salary expectations over the next decade. It identified a number of easy to consume trends which caught my attention and perhaps worth your consideration.
In the report, they state that in 2019 we had 162.8 million workers in the USA which they project to grow by 6-8 million jobs (in total) over the next ten years, an annual growth of less than HALF of 1%, compared to the last decade’s growth of 1.3%. Not surprising (and due to the aging of America), 60% of those new jobs will be in the health and medical fields. You can browse through the interactive handbook and click on any number of filters and report types, but BE SURE to try the links in the “Browse Occupations” section on the bottom half of the first screen. Try “Most New Jobs” or “Highest Paid Jobs”. (Spoiler alerts: Highest-Paid Jobs has Psychiatrists listed at the top, and while Most-New Jobs is listed as “Home Health”, the second biggest increase in jobs is Fast Food workers!)
BLS talks about some notable related and supporting projections which can be summarized as follows:
More people over 55 years old will continue to be employed full-time (financial stability has decreased)
Fewer people below 34 years old will choose to be employed full-time (the ‘desk job’ has less appeal to many in this age group who want freelance or “gig” style work)
About 57% of women will work full-time outside the home, compared to 66% of men (and both will be down slightly from today)
The BLS offers their narrative: “The decline in labor force participation is due to the aging of the babyboom generation, a continuation of the declining trend in men’s participation, and a slight decline in women’s participation”. And throughout the report they cite the aging of the baby-boomers as a root cause.
BLS spends billions of dollars and countless resources gathering data and analyzing it and ultimately come to the same conclusion and projections that we all implicitely feel: Baby-Boomers are timing out, or as I like to say ‘Greying out’. Couple this workforce aging with the URGENT macro topics of:
COVID – all of the long-term shifts and economics caused by it
Trade Policies – shifting of raw materials/goods/transportation/macro-economics
‘GIG work Mentality’ – intentionally skipping full-time employment altogether
Online everything – The death of brick and mortar retail
and we have a crazy ride over the next 10 years to be sure. The world we knew has changed, and those changes are STRUCTURAL. We will never go back to pre-2020 life…. This is not a fad, nor a short-term manageable incident. The world has changed and the quicker each of us decides how they wish to participate in the new structure, the easier life will be for that person. And when enough people have gotten on board, life will become easier once again.
Note: While the report above is USA-centric, similiar patterns are already being seen worldwide.
https://journal.uptimeinstitute.com/wp-content/uploads/2020/09/BLS-blog.jpg8202320Mark Harrishttps://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngMark Harris2020-10-19 06:00:012020-10-20 09:38:45Job Projections, 2019-2029, Macro shifts, Gig work and the Baby-Boomers
Periodically we get questions about data center temperatures and energy savings. The questions usually ask about possible energy savings in commercial data centers by raising the temperature of the data hall. Some form of this question surfaces regularly over time because it just seems to be a common sense opportunity. But there is alot to consider before running out and adjusting your “thermostat”.
The most recent question took the form of “I have followed ASHRAE for some years and have increased temperature thresholds in rooms over which I have full control. If there are cost and efficiency gains by accepting higher temperatures, is there talk of marketing this — i.e., segregated/containerized hotter rooms — to colo customers who believe they can take the heat to save on monthly recurring charges? If so, I have not seen it yet.”.
This really is a great question, because it means we are thinking about energy conservation and optimization. At first glance this seems like easy avenue to pursue. In fact many of the IT gear makers are already supporting higher inlet temperatures, some as high as 95-degrees F or more. Earlier this year, our CTO Chris Brown provided the short answer, “In general, there are cost savings and energy efficiency gains to be realized by increasing supply temperatures in data halls.” and then went on to add that, ‘BUT it is not as simple as it sounds, because alot of factors need to be considered before it’s a ‘slam-dunk'”.
So lets dive into these factors in some more detail:
Increasing the inlet supply temperature often results in increased air flow requirements. Depending on the design of the facility, this may over tax the fans and significant increase their energy consumption if the system is tightly designed around a given air flow. Remember that a non-trivial amount of the energy spent for cooling is consumed by the fan motors themselves, so running fans faster WILL reduce any savings seen with higher temperatures. Using an Energystar report on power consumption of fans, they state that “a reduction of 10% in fan speed reduces that fan’s use of electricity by approximately 25%. A 20% speed reduction yields electrical savings of roughly 45%.” so for this discussion we’d be doing the reverse… Increasing the fan speed with the resulting increase in power draw.
Manufacturers usually warranty their equipment when used within their specified operational conditions, including the range of temperatures at the inlet. And while some manufacturers may have increased those ranges to allow for warmer inlet temperatures, the data hall must be cooled to the LOWEST operating condition specified across all equipment installed in that hall. In other words, your thousand Dell servers may happily support inlets of 85 or 95-degrees, but your Cisco NEXUS sitting at the end of each row may require 80F or less. And the storage appliance may specifiy 78F. Each piece of gear has it’s own allowable environment to be warrantied. And there is no practicle way to separate the various types of gear across different ‘temperature zones’ in a commercial data center even if such zones were available, which they are not.
Most enterprise grade IT gear also includes in-chassis fans, which vary in speed as the inlet temperature goes up. Again, raising the hall temperatures and the inlet temperatures will likely cause all of the in-chassis fans to run much faster, and draw significantly more current. (Which ironicly is great for the old macro PUE calculation, but horrible when considering actual energy usage for cooling). And as Chris pointed out in his response, there is a secondary concern when using in-chassis fans more heavily: “doing so can significantly raise the sizing requirements on UPS systems, because all of that new in-chassis fan power is now considered ‘IT load’ from a design standpoint.” A huge factor to be sure.
The incentive may not exist. Most colocation providers write SLAs (service level agreements) specifying inlet temperatures UP to and including the ASHRAE “Recommended” 80-degrees. (Many still contractually guarantee an inlet of 74F). Chris advises that he has not run across any of those type of client agreements which offer different rates based on willingness to operate in other environmental conditions (i.e. ASHRAE A1 through A4 for instance). This is likely because providers with existing facilities do not have their data hall designed to allow for varying the supply temperature in such granular neighborhoods. Hot/Cold aisels may exist, but the inlet temperatures are not offered on a slide scale pricing schedule. And since not all tenants in any given hall will want an elevated inlet supply air temperature, those operators and their contracts do not provide any variants.
Failure rates of IT gear have not been studied sufficiently at the A3 and above higher temperature ranges. Intel has been studing this question for years and regularly concludes that servers running up through the A2 range (95 degrees F) seem to have typical failure rates. They also state that more study is required to make the same statements around A3 and A4. And then you have the network, storage and security gear from hundrerds of manufacturers that also have the same requirements.
And keep in mind that running a data center above ASHRAE’s “Recommended” levels might limit the number of hours the data center technicians could remain on the IT floor due to local regulatory, health and safety concerns. As cited by OSHA, working conditions between 68 F and 78 F are ideal for the workplace. A number of other health and safety journals (including Healthday) go into more detail about what would be expected if technicians are required to work hotter data centers. Their summary includes higher rates of accidents and more mis-judgments as the temperature increases above 73-degress, with significant concerns for employees working in the conditions identified by ASHRAE’s A3 and A4.
So can higher temperature halls be effective? Absolutely, if they were designed to be higher. If the mechanical design accounted for the hall to be much hotter (gasp: A3 or A4 ?) and if ALL of the IT gear that was to be installed in that hall was purchased and warrantied for that range, then absolutely this works. But few if any halls are built and/or operated that way today, because doing so would severely limit the choice of gear that could be included on the raised floor.
Directionally a great discussion… it’s just the retrofit of reality discussion that gets awkward…
https://journal.uptimeinstitute.com/wp-content/uploads/2020/10/thermo3.jpg7352420Mark Harrishttps://journal.uptimeinstitute.com/wp-content/uploads/2022/12/uptime-institute-logo-r_240x88_v2023-with-space.pngMark Harris2020-10-12 06:00:522020-09-21 15:37:57Increasing data center temperatures to reduce energy costs
Data centers are built and sited to withstand all that Mother Nature can throw at them — or at least, is likely to throw at them — during their lifecycle. This has long been a given, practiced and understood by designers, planners and regulators. But climate change is changing everything. Recent weather events have led some governmental agencies to ask how exposed data centers are from climate, such as high winds and lightning.
These are questions data centers throughout the world should consider, as storms and other natural disasters are increasingly frequent and severe. In the last few weeks alone, the world has experienced record-shattering derechos and tropical cyclones, as well as heatwaves and wildfires (both of which can alter wind patterns). Managers must review their data center’s resiliency and disaster recovery plans in the context of the amplifying effects of climate change. As we said in the Uptime Institute report A mission-critical industry unprepared for climate change, these reviews should be conducted regularly and plans updated; the climate is, regrettably, changing rapidly and proving volatile.
Recently, Uptime Institute was asked (via our colleagues at TechUK, a UK membership body for the IT industry) to give our views on three specific issues that might arise as a result of severe storms: staff access during storms, lightning strikes, and standards. Some thoughts on these are below.
The first relates to blocked transport routes — that is, how do data centers manage if staff are unable to get to work? The answer is, most data centers have a plan for this. This is an aspect of Uptime Institute’s Tier Standard: Operational Sustainability program, and it requires preparation and budget. When bad weather is forecast, many data centers either locate personnel at the facility or in hotels nearby. Because one shift cannot leave the site until they are relieved by the next shift, staff may work overtime waiting for the next crew to arrive. All these contingencies must be considered — possibly for extended periods of time. In extremis, of course, it is possible for staff to remain on site, with sleeping accommodation and food. But this is not a practice that is advised except in the most severe situations.
Another question asked after countermeasures against damage from lightning strikes and/or power surges. The answer first lies in local guidance and regulations. All locales are a bit different, but most have some life safety codes that address power surges and lightning. Lightning protection systems are foremost geared to building protection, but other codes cover ways to prevent local lightning strikes from impacting other systems, such as power and communication systems. Typically, data centers will not only meet these codes, but also exceed them in some ways.
There are ways to design the facility to help reduce the risk of the data center itself being affected. A first step is to incorporate a lightning protection system that uses grounded lightning rods or active systems to shunt ion buildup in the immediate atmosphere to ground. This stops the local buildup of a charge that will result in a lightning strike.
Another issue to anticipate is that a lightning strike can occur near the data center and could enter the facility via the power system. Electrical systems are often designed to transfer the energy from a lightning strike to a grounding device. The efficacy of this strategy depends on the proximity of a strike to the service entry. Outdoor communications cables (telecommunications or controls) usually have devices to block increased voltage from entering the facility.
In locations where lightning is prevalent, many facilities circumvent the problem: when a storm is forecast, they transfer to on-site power production and isolate from the outside power system altogether. This underlines the need for on-site power systems; because two utility feeders are installed in relatively close proximity to one another, a single strike could impact both systems. Dual utility feeds cannot provide the absolute isolation that on-site power production can.
A third question asked after standards that cover risks from lightning and/or high winds. Local building codes address this, but such codes are intended to help ensure human life is not put at risk during such events, not that a facility operates through it. Tier Standard: Operational Sustainability addresses this by requiring the site risks from natural events be addressed.
The challenges vary based on location and threat. In the US, Texas Tech University has done research on winds and tornadoes for over 20 years. They have an air cannon that shoots things like wooden boards at buildings to see how they perform. This research is used to help structural engineers learn how to harden structures against high wind and the debris it carries. The US Federal Emergency Management Agency has developed guidance on how to prepare for extreme weather events, as have other nations’ governments regarding threats their regions face.
There is one important thing to keep in mind: While people are quite capable of dealing with the typical or even slightly more severe than usual threats in their area, this does not mean a data center will necessarily be able to operate through it. Storms seem to be getting more severe; in August 2020, the US experienced the strongest hurricane (Hurricane Laura) ever to make landfall on the Texas Gulf Coast.
Even when storms diminish after landfall, they can do a lot of (sometimes unpredictable) damage. For example, just three years before Laura, Hurricane Harvey dumped so much rain that 500-year flood planes were inundated. This is not something most prepare for.
In such situations, even if the data center is operational, the telecommunications system may have facilities that are not. In these cases, a data center is ready to operate but can’t communicate with the outside world. And staff’s personal priorities also must be considered: few will report to work when their family’s safety is at risk — a lesson learned in several natural disasters.
Bottom Line: While we should take precautions to help ensure data centers are prepared for the foreseeable, we have to remember that just hardening a data center is not enough; Mother Nature will always show us where we stand in the grand scheme of things.
PUE: The golden metric is looking rusty
/in Executive, Operations/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]When the PUE (power usage effectiveness) metric was first discussed at a meeting of The Green Grid in Santa Clara, back in 2007, a microphone stand was placed in each aisle of the auditorium. The importance of the initiative was understood even then: experts, including the founders of Uptime Institute, formed lines to give their considered input. And if there was one point that came across, it was that the industry should not treat PUE as a comparative metric, a gold standard that every designer, operator and planning authority must chase down, for comparison, presentation and applause.
Thirteen years on, that is almost exactly what has happened. PUE, thanks to its simplicity and universal applicability, has become the critical benchmark for scoring data centers for efficiency and “greenness” (environmental responsibility). But it is often used uncritically.
This has had both positive and negative consequences. The positive side is that PUE values have been forced down across the world, as operators strive to reduce waste and operating costs. To meet their goals, suppliers have focused on improving electrical efficiency of equipment and have developed nonmechanical alternatives (such as free cooling). PUEs have fallen from an average of 2.5 in 2007 to around 1.6 today. In an Uptime Institute survey, around 95% of respondents said it is important that colocation companies have a low PUE.
So what is the negative side? Its simplicity. The very quality that led to PUE’s near universal adoption also may have led to its being over-used and misapplied — increasingly, with unintended consequences.
This observation is hardly new — indeed, this is exactly what those experts feared back in 2007 as they lined up to voice their concerns. And it is why PUE was developed into a standard, with rules on how it should be measured, and why Partial PUE variants were developed.
But the unintended consequences are becoming more serious and more concerning. The metric may be driving the wrong behaviors. For this reason, it is very possible that PUE values may begin to be de-emphasized in the near future and more operators may allow their number to rise a little. (In 2019, Uptime Institute did note a marginal rise in PUE values.) If this happens, some operators may find themselves in conflict with certain regulators and planners, for whom PUE has been an easily understood but simplistic way of enforcing environmental goals (policing this is another matter). These planning authorities, then, may also need to review their policies around driving down the PUE numbers. (California, Singapore, Amsterdam, take note.)
There are, we think, three strong and strengthening reasons why focusing on the PUE of the data center might be counter-productive:
There is, to be fair, a good counter-argument to these three points: that by searching for ways to lower the PUE regardless of these constraints, data center operators can find ways to meet all the goals. Liquid cooling, for example, can be highly resilient and uses little water. The argument here, then, is not for anyone to abandon PUE, or to stop watching and measuring it, but to apply it more cautiously than ever, and more flexibly.
Why data center operators are investing in more redundancy
/in Design, Executive, Operations/by Andy Lawrence, Executive Director of Research, Uptime Institute, [email protected]When Uptime Institute recently asked over 300 data center managers how the pandemic would change their operations, one answer stood out: Two-thirds expect to increase the resiliency of their core data center(s) in the years ahead. Many said they expected their costs to increase as a result.
The reasoning is clear: the pandemic — or any future one — can mean operating with fewer staff, and possibly with disrupted service and supply chains. Remote monitoring and preventive maintenance will help to reduce the likelihood of an incident, but machines will always fail. It makes sense to reduce the impact of failure by increasing system redundancy.
But even before the pandemic, there was a trend toward higher levels of redundancy. As shown in the figure below, roughly half of those participating in the Uptime Institute 2020 global survey of suppliers, designers and advisors reported their customers have increased redundancy levels in the last three to five years.
This trend may seem unsurprising to some, but it was not entirely predictable. The growth of cloud has been accompanied by the much greater use of multisite resiliency and regional availability zones. In theory, at least, these substantially reduce the impact of single-site facility outages, because traffic and workloads can be diverted elsewhere. Backed by this capability, some operators — Facebook is an example — have proceeded with lower levels of redundancy than was common in the past (thereby saving costs and energy).
The use of availability zones, however, has come with its own problems, with networking and software issues often causing service interruptions. And the loss of one data center immediately places capacity and traffic demand on others, heightening risks. For this reason, even the big cloud providers and internet application operators manage mostly concurrently maintainable facilities, and it is common for them to stipulate that colocation partners have N+2 level facilities.
With a variety of options, the overall shift to increased resiliency is still slow and quite nuanced, with designers mostly favoring either N+1 or N+2 configurations, according to site and business needs, and, often, according to the creativity of the designers. Overall, there is actually a marginal decrease in the number of data centers that are 2N, but a steady three-year shift from N+1 to N+2 — not only in power, but also in cooling (see figure below). There is also an increase in the use of active-active availability zones, as discussed in our recent report Uptime Institute global data center survey 2020.
Demand patterns and growing IT dependency partly account for these higher levels of redundancy/resiliency. The level of resiliency needed for each service or by each customer is dictated by business requirements, but this is not fixed in time. The growing criticality of many IT services highlights the importance of mitigating risk through increased resiliency. “Creeping criticality” — a situation in which infrastructure and processes have not been upgraded or updated to reflect the growing criticality of the applications or business processes they support — may require redundancy upgrades.
Uptime Institute expects operators to make more use of distributed resiliency in the future — especially as more workloads are designed using cloud or microservices architectures (workloads are more portable, and instances are more easily copied). But there is no sign that this is diminishing the need for site-level resiliency. The software running these distributed services is often opaque, complex and may be prone to errors of programming or configuration. Annual outage data shows these types of issues are proliferating. Further, any big component failures can cascade, making recovery difficult and expensive, with data and applications synchronized across multiple sites.
The trend for now is clear: more resiliency at every level is the least risky approach — even if it means some extra expense and duplication of effort.
Job Projections, 2019-2029, Macro shifts, Gig work and the Baby-Boomers
/in Executive/by Mark HarrisIn mid-September 2020, The US Bureau of Labor and Statistics published their updated 2019-2029 Employment Projection summary news release and the associated handbook which discusses various job roles and hiring and salary expectations over the next decade. It identified a number of easy to consume trends which caught my attention and perhaps worth your consideration.
In the report, they state that in 2019 we had 162.8 million workers in the USA which they project to grow by 6-8 million jobs (in total) over the next ten years, an annual growth of less than HALF of 1%, compared to the last decade’s growth of 1.3%. Not surprising (and due to the aging of America), 60% of those new jobs will be in the health and medical fields. You can browse through the interactive handbook and click on any number of filters and report types, but BE SURE to try the links in the “Browse Occupations” section on the bottom half of the first screen. Try “Most New Jobs” or “Highest Paid Jobs”. (Spoiler alerts: Highest-Paid Jobs has Psychiatrists listed at the top, and while Most-New Jobs is listed as “Home Health”, the second biggest increase in jobs is Fast Food workers!)
BLS talks about some notable related and supporting projections which can be summarized as follows:
The BLS offers their narrative: “The decline in labor force participation is due to the aging of the babyboom generation, a continuation of the declining trend in men’s participation, and a slight decline in women’s participation”. And throughout the report they cite the aging of the baby-boomers as a root cause.
BLS spends billions of dollars and countless resources gathering data and analyzing it and ultimately come to the same conclusion and projections that we all implicitely feel: Baby-Boomers are timing out, or as I like to say ‘Greying out’. Couple this workforce aging with the URGENT macro topics of:
and we have a crazy ride over the next 10 years to be sure. The world we knew has changed, and those changes are STRUCTURAL. We will never go back to pre-2020 life…. This is not a fad, nor a short-term manageable incident. The world has changed and the quicker each of us decides how they wish to participate in the new structure, the easier life will be for that person. And when enough people have gotten on board, life will become easier once again.
Note: While the report above is USA-centric, similiar patterns are already being seen worldwide.
Increasing data center temperatures to reduce energy costs
/in Design, Executive/by Mark HarrisPeriodically we get questions about data center temperatures and energy savings. The questions usually ask about possible energy savings in commercial data centers by raising the temperature of the data hall. Some form of this question surfaces regularly over time because it just seems to be a common sense opportunity. But there is alot to consider before running out and adjusting your “thermostat”.
The most recent question took the form of “I have followed ASHRAE for some years and have increased temperature thresholds in rooms over which I have full control. If there are cost and efficiency gains by accepting higher temperatures, is there talk of marketing this — i.e., segregated/containerized hotter rooms — to colo customers who believe they can take the heat to save on monthly recurring charges? If so, I have not seen it yet.”.
This really is a great question, because it means we are thinking about energy conservation and optimization. At first glance this seems like easy avenue to pursue. In fact many of the IT gear makers are already supporting higher inlet temperatures, some as high as 95-degrees F or more. Earlier this year, our CTO Chris Brown provided the short answer, “In general, there are cost savings and energy efficiency gains to be realized by increasing supply temperatures in data halls.” and then went on to add that, ‘BUT it is not as simple as it sounds, because alot of factors need to be considered before it’s a ‘slam-dunk'”.
So lets dive into these factors in some more detail:
So can higher temperature halls be effective? Absolutely, if they were designed to be higher. If the mechanical design accounted for the hall to be much hotter (gasp: A3 or A4 ?) and if ALL of the IT gear that was to be installed in that hall was purchased and warrantied for that range, then absolutely this works. But few if any halls are built and/or operated that way today, because doing so would severely limit the choice of gear that could be included on the raised floor.
Directionally a great discussion… it’s just the retrofit of reality discussion that gets awkward…
Infographic: Data Center Industry Staffing Shortage is Getting Worse
/in Executive, Operations/by Rhonda Ascierto, Vice President, Research, Uptime InstituteGender Diversity 2018-2020
Data Centers and Thunder, Lightning, Wind and Rain
/in Design, Executive, Operations/by Chris Brown, Chief Technical Officer, Uptime InstituteData centers are built and sited to withstand all that Mother Nature can throw at them — or at least, is likely to throw at them — during their lifecycle. This has long been a given, practiced and understood by designers, planners and regulators. But climate change is changing everything. Recent weather events have led some governmental agencies to ask how exposed data centers are from climate, such as high winds and lightning.
These are questions data centers throughout the world should consider, as storms and other natural disasters are increasingly frequent and severe. In the last few weeks alone, the world has experienced record-shattering derechos and tropical cyclones, as well as heatwaves and wildfires (both of which can alter wind patterns). Managers must review their data center’s resiliency and disaster recovery plans in the context of the amplifying effects of climate change. As we said in the Uptime Institute report A mission-critical industry unprepared for climate change, these reviews should be conducted regularly and plans updated; the climate is, regrettably, changing rapidly and proving volatile.
Recently, Uptime Institute was asked (via our colleagues at TechUK, a UK membership body for the IT industry) to give our views on three specific issues that might arise as a result of severe storms: staff access during storms, lightning strikes, and standards. Some thoughts on these are below.
The first relates to blocked transport routes — that is, how do data centers manage if staff are unable to get to work? The answer is, most data centers have a plan for this. This is an aspect of Uptime Institute’s Tier Standard: Operational Sustainability program, and it requires preparation and budget. When bad weather is forecast, many data centers either locate personnel at the facility or in hotels nearby. Because one shift cannot leave the site until they are relieved by the next shift, staff may work overtime waiting for the next crew to arrive. All these contingencies must be considered — possibly for extended periods of time. In extremis, of course, it is possible for staff to remain on site, with sleeping accommodation and food. But this is not a practice that is advised except in the most severe situations.
Another question asked after countermeasures against damage from lightning strikes and/or power surges. The answer first lies in local guidance and regulations. All locales are a bit different, but most have some life safety codes that address power surges and lightning. Lightning protection systems are foremost geared to building protection, but other codes cover ways to prevent local lightning strikes from impacting other systems, such as power and communication systems. Typically, data centers will not only meet these codes, but also exceed them in some ways.
There are ways to design the facility to help reduce the risk of the data center itself being affected. A first step is to incorporate a lightning protection system that uses grounded lightning rods or active systems to shunt ion buildup in the immediate atmosphere to ground. This stops the local buildup of a charge that will result in a lightning strike.
Another issue to anticipate is that a lightning strike can occur near the data center and could enter the facility via the power system. Electrical systems are often designed to transfer the energy from a lightning strike to a grounding device. The efficacy of this strategy depends on the proximity of a strike to the service entry. Outdoor communications cables (telecommunications or controls) usually have devices to block increased voltage from entering the facility.
In locations where lightning is prevalent, many facilities circumvent the problem: when a storm is forecast, they transfer to on-site power production and isolate from the outside power system altogether. This underlines the need for on-site power systems; because two utility feeders are installed in relatively close proximity to one another, a single strike could impact both systems. Dual utility feeds cannot provide the absolute isolation that on-site power production can.
A third question asked after standards that cover risks from lightning and/or high winds. Local building codes address this, but such codes are intended to help ensure human life is not put at risk during such events, not that a facility operates through it. Tier Standard: Operational Sustainability addresses this by requiring the site risks from natural events be addressed.
The challenges vary based on location and threat. In the US, Texas Tech University has done research on winds and tornadoes for over 20 years. They have an air cannon that shoots things like wooden boards at buildings to see how they perform. This research is used to help structural engineers learn how to harden structures against high wind and the debris it carries. The US Federal Emergency Management Agency has developed guidance on how to prepare for extreme weather events, as have other nations’ governments regarding threats their regions face.
There is one important thing to keep in mind: While people are quite capable of dealing with the typical or even slightly more severe than usual threats in their area, this does not mean a data center will necessarily be able to operate through it. Storms seem to be getting more severe; in August 2020, the US experienced the strongest hurricane (Hurricane Laura) ever to make landfall on the Texas Gulf Coast.
Even when storms diminish after landfall, they can do a lot of (sometimes unpredictable) damage. For example, just three years before Laura, Hurricane Harvey dumped so much rain that 500-year flood planes were inundated. This is not something most prepare for.
In such situations, even if the data center is operational, the telecommunications system may have facilities that are not. In these cases, a data center is ready to operate but can’t communicate with the outside world. And staff’s personal priorities also must be considered: few will report to work when their family’s safety is at risk — a lesson learned in several natural disasters.
Bottom Line: While we should take precautions to help ensure data centers are prepared for the foreseeable, we have to remember that just hardening a data center is not enough; Mother Nature will always show us where we stand in the grand scheme of things.