Article Featured in Data Centre Review
By: Gordon Johnson, Senior CFD Manager, Subzero Engineering
The current and growing industry focus on sustainability has highlighted the need to work towards energy efficient data center designs, which, in turn, should provide the perfect combination of sustainable operating conditions. The end result being truly optimized data center performance.
Computational Fluid Dynamics (CFD) plays a vital role when it comes to data center construction, retrofit, and helping improve the efficiency and performance of legacy data centers. It allows the owner or operator to simulate endless combinations of racks, cabinets, cabling, and mechanical and engineering (M&E) equipment for greater optimization.
The owners and operators of these facilities may not be aware that they are literally blowing a lot of (unnecessary) cold air into the aisles in order to keep the servers, switches, routers, and the storage units cool. What they will know, however, is that this wasted air is wasted money. Worse still, emissions will be increased, hot spots will remain an issue, and they will eventually experience equipment failures.
Many data center managers know that something has to change to overcome this unsustainable situation, but they may not have a plan or strategy. This is where CFD is critical.
CFD is essentially an MRI of the data center, and when it comes to informed decision-making, the mantra ‘if you can’t measure it, you can’t manage it’ still holds strong. However, in the case of CFD, ‘if you can’t measure it, you can’t improve it’ may be more appropriate.
What is a CFD analysis?
CFD software creates a 3D model of the physical attributes within a data center, as well as the location and performance characteristics of the cooling units, IT equipment, power systems, and other significant components of the data center.
This ‘data center MRI’ provides a complete, detailed thermal understanding of the data center, providing insight relating to the physical layout of the space and equipment; the power equipment thermal dissipation and rack power density; environmental conditions; cooling capacity and air distribution effectiveness. This is the baseline model or the “as is” snapshot of how the data center is currently operating.
From the information obtained, additional CFD modeling will recommend how to optimize the operating environment while reducing energy and maximizing efficiency.
Many key benefits of CFD analysis include assessment of the data center layout and its performance; validation of proposed design changes, considerations for efficiency and reliability; accurate prediction/simulation of the cooling design process; the ability to assess temperature and airflow modeling; and reconditions for a cost-effective and energy efficient design.
CFD provides clarity
A data center CFD analysis provides the visibility that is essential when it comes to understanding both your existing operating environment and any potential design changes. CFD can also be used to test out ‘what if’ scenarios when it comes to new data center construction or modernization projects.
In essence, a CFD provides detailed, data-driven analysis of the cold supply air and the hot exhaust air throughout the data center, and identifies the potential overheating of IT equipment “hot spots”. Such analysis highlights the way in which energy costs can be reduced by optimizing and maximizing data center cooling. CFD is a tool used to virtually test legacy (and new) data center environments for active temperature, pressure and airflow, and to optimize the mechanical system directly to server heat loads (kW).
Armed with data from the CFD analysis, a suitably experienced technology partner, such as Subzero Engineering, can advise customers on impactful data center design changes, upgrades, or tweaks to improve operating improvements – all of which will help optimize and reduce energy usage.
For example, CFD modeling can demonstrate what impact a cold aisle containment (CAC) system will have versus a hot aisle containment (HAC) system. It will advise which method will save more energy, full or partial containment. After containment is installed, it will show if computer room air conditioning (CRAC) units can be turned off and how low fan speeds can be reduced on computer room air handlers (CRAHs). Finally, it will also inform the user on how high the operating temperature can be increased, while following the recommended thermal guidelines from AHSRAE.
The end result is that CFD based design improvements will provide customers with significant PUE, WUE, KW, kWh, carbon, and water reductions.
A logical conclusion
One of the major benefits of using CFD modeling and analysis is that it can quickly demonstrate how a containment solution will immediately reduce a data center’s energy cost and lower its environmental footprint.
Regardless of which type of containment is used, large energy savings can be achieved by optimizing the airflow and raising the supply temperature, while simultaneously eliminating hotspots. Additionally, containment improves the benefits and economic feasibility of other efficiency measures, such as digital scroll compressors, electronic commutation (EC) and variable speed drives, data center infrastructure management (DCIM) software, and building controls.
Further, by separating the cold supply air from the hot exhaust air, containment saves energy, thus reducing operating costs. Yes, there is the initial investment cost of installing the containment solution, but the potential return on investment (ROI) of 10-14 months means that, very quickly, end-users will achieve payback. What’s more, from that point onwards, they’ll be saving energy, reducing operating expenditure (OpEX) money, and reducing carbon emissions to become more sustainable.
A savings summary
Since 2015, we’ve kept track of the various savings we have achieved for our customers through the careful, considered optimization of their data center environments. The numbers are impressive: our customers have saved just under 3.5 billion Kilowatt hours, reduced water consumption by one and a half billion gallons and have a total carbon reduction of over 3.2 million tons – the equivalent of planting 17 million trees.
Along the way, we’ve also been able to help customers achieve up to a 29% energy cost reduction, and been able to reduce PUE (Power Usage Effectiveness) by an average of 0.4 and WUE (Water Usage Effectiveness) by an average of 0.3.
Subzero Engineering believes that a data-driven approach to digital infrastructure is essential and thanks to the use of CFD, we can simultaneously help our customers increase data center density, operating efficiency, and environmental sustainability by implementing optimized containment solutions.
However you choose to address the challenge and opportunity of improving the efficiency of your data center, it’s essential that CFD is used as a key tool in this process. Without CFD, whether you choose to implement an optimization program yourself, or to engage with a technology partner, you’ll neither be able to understand how your data center is performing right now, nor what improvements can be made into the future.
Andy Connor – Channel Director, EMEA, Subzero Engineering
In 2020, the Uptime Institute’s, Andy Lawrence stated, “The average power usage effectiveness (PUE) ratio for a data center is 1.58, only marginally better than 7 years ago.”
This revelation may come as a welcome shock, and while it might be overstating the situation to characterise data center energy usage as the Internets ‘dirty secret’, there’s little doubt that the reality of the sectors carbon impact has been masked by the many headlines which focus solely on its sustainability successes.
Colt Data Centre Services, for example, recently announced that its operations across Europe are now fully powered by 100% renewable energy, while many members of the U.S. hyperscale community are publicly revealing their latest renewable energy projects and initiatives. Carbon offsetting is another idea quickly embraced by end-users, vendors and operators of all shapes and sizes, and while all of these activities, in part, contribute to data center sustainability improvements, they do not directly address the issue of data center power consumption.
Data Center Power Usage
Today there are many estimates as to the amount of power that data centers across the globe consume on an annual basis. Energy Innovation estimates that, in 2018, data centers likely consumed 205 terrawatt-hours (TWh), which equates to 1% of total global electricity. However, the authors of a paper published in ‘Global Energy Interconnection’ in June 2020 state that ‘data centers will become the world’s largest users of energy consumption, with the ratio rising from 3% in 2017 to 4.5% in 2025’.
The data differs again in the January 2020 Uptime Institute Journal, which reports EU data center energy consumption figures of 130 TWh in 2017, alongside Greenpeace’s 2018 Chinese data center figure of 160 TWh, which makes for a combined total of 290 TWh for China and Europe alone!
The fact is that as data center capacity increases, so will energy usage. And while we may not agree on the exact numbers, few would argue about the direction of travel. So what can we do to change the trajectory, and how can we begin to pinpoint consistency within sustainable strategies?
A Change In Demand
Demands for digital transformation are a key factor behind data center energy consumption, but rather than overload you with a tsunami of data, I would urge you to take a short time out to consider just how essential information technology has become to almost every factor of everyday life.
Take a typical workday, how do you communicate with colleagues, what does your role entail? Then think about your plans for the weekend (lockdown not withstanding), have you thought about the films you’ll watch, the apps you might use, or your personal connection to a data center?
Now try and imagine a future that also includes artificial intelligence (AI), augmented (AR) and virtual reality (VR), what impacts will this digital consumption have on the data center industry, and what does it mean for sustainability?
Those are big questions, many of which people outside the sector won’t have considered, but the answer may lie within a recent data center industry initiative to show us the way forward. Recently 25 companies and 17 associations across Europe joined together to sign the Climate Neutral Data Centre Pact, with the objective of making data centers climate neutral by 2030.
There’s clearly a growing momentum behind sustainability, but the industry needs to move beyond the environmental easy wins of the past few years and to start to address the factors that really address efficiency and PUE ratings – those which have hardly changed in seven years.
There’s also the question of is PUE truly enough to measure our carbon impact, so with this in mind, where can we begin?
Beginning With The Data
Back in 2005, Subzero Engineering started life as a Computational Fluid Dynamics (CFD) consultancy. At the time, a large percentage of the industry were using raised floors and experiencing issues with leakages. Yet, with a simple to use and accurate software solution we were able to show customers how they could analyse their data center infrastructure and take steps to both improve efficiency and reduce their environmental impact.
Fast-forward sixteen years and that approach has stayed with us. Today we’re an engineering-led solutions provider that helps world-leading businesses achieve a lower carbon footprint, greater efficiency, reduced operating costs and exceptional performance – and it all starts with the data.
For example, by showing customers the hot and cold air influences within their data center and helping them to analyse, optimise and retrofit their facilities, we believe we can help them find the perfect balance between sustainability and performance.
The proof is in the outcomes and today we keep a live record of the annual energy savings we’ve achieved for our customers. To-date they include:
- Total savings: $332M
- Total kW savings: 356kW
- Total kWh savings: 3BN kWh
- H2O savings (gallons): 1.5BN
- CO2 reductions (tons): +3M
However, while these data points show some of the gains that can be made by focusing on sustainability, two questions remain; how do organizations become more energy conscious, and what are the next steps they can take to become more sustainable?
Defining Next Steps
At Subzero Engineering we believe that beginning with a data-driven CFD report is the first step, and offers data center operators insight into how to drive efficiencies across all areas of their facility. This is not limited to airflow; it includes the Uninterruptible Power Supply (UPS), Computer Room Air Conditioners (CRAC), racks, IT and cooling fan speeds.
A CFD analysis also shows them how they can achieve a higher rack density, more computing power and help increase the operating temperature to gain both a higher performing and more streamlined, efficient data center.
This information is invaluable, offering both a starting point and a medium for creating a strategy that balances performance and efficiency. It also offers a means of truly understanding what kind of return on investment (ROI) they can expect from improving sustainability, especially in terms of reductions in energy and water usage, and lower carbon emissions.
Today energy efficiency and sustainability objectives have become key drivers for owners and operators. Subzero has always been a sustainability-engineering organization; it just so happens that in recent years ‘sustainability’ has become a key talking point for the industry.
Coming back to the data, a paper authored by Anders S.G. Andrae once presented three possible scenarios for data center electricity usage (TWh) by 2030. The best-case figure is 1,137; the expected figure is 2,967; and the worst case is 7,933. As an industry we cannot let the latter become a reality.
In the absence of the grown-up sustainability conversation that needs to happen soon, where more businesses and consumers become fully aware of the environmental consequences of their digital footprint, I believe more and more pressure will force our industry to perform better.
Sustainability, however, begins with data-driven action, and a free CFD analysis is a perfect place to start.
To learn more about modernizing or driving data center sustainability with the help of Subzero Engineering, connect with a local technical expert here.
This blog was first published on Intelligent CIO Europe in March 2021. To read the original online, click here.
Subzero understands how complicated it can be when trying to figure out how to save energy and money in your data center and we are here to help make it easier. Our Senior CFD Manager, Gordon Johnson, has created an easy-to-use calculator suite that can help you figure out some of those complicated scenarios and point you in the right direction.
The Subzero Calculator Suite consists of 6 calculators and can be downloaded in both US units and SI units.
1. ROI Calculator
Calculate the annual cost of operating your data center and estimate the yearly savings after installing containment and increasing the supply temperature from the CRACs. You can also estimate new PUE after containment and the ROI payback of a containment project based on the total cost.
2. VFD Calculator
Lowering the fan speeds on CRACs, especially after installing containment, saves energy and money. This calculator allows you to enter before and after fan speed, CRAC CFM, and fan motor power, and via the Fan Affinity Law provides the new fan motor power and the annual $ savings based on the new fan speed.
3. CRAC Annual Cost Calculator
Determine the total annual cost of running a CRAC unit by entering a few details.
4. CRAC Cooling Calculator
Determine the true kW of cooling from any CRAC or cooling unit based on airflow (CFM) and the Delta T (return air temperature – supply air temperature) across the CRAC.
5. Airflow CFM Calculator
Determine airflow in CFM needed to cool a rack based on the rack’s kW and the Delta T (temperature rise of air through the servers). Determine if you meet your design cooling capacity of supply airflow from the CRACs versus demand airflow from the IT equipment.
6. UPS kW Calculator
Determine the UPS Heat Load (kW) based on user inputs of UPS power rating, UPS % load, and UPS % efficiency.
Need more help?
Let one of our DCEP, CDCDP Certified Engineers help your data center reach its full potential.
SERVICES WE OFFER
Computational Fluid Dynamics – Subzero Engineering offers Computational Fluid Dynamics (CFD) services from accredited CDCDP, DCEP professionals, thus providing a comprehensive approach to modeling the airflow, temperature, and creating an accurate energy profile of a data center.
Through state-of-the-art software, we construct a 3D layout of your data center. This layout models the hot and cold airflow within your facility, as well as the impact of load distribution. This allows Subzero’s engineering team to develop a baseline from which improvements can be noted and potential savings calculated. The engineers that perform the CFD services for Subzero have CDCDP (Certified Data Center Design Professional) and DCEP (Data Center Energy Practitioner) accreditation.
Energy Assessments – With the ever-increasing rise in data center energy consumption, all in the industry as well as the United States Department of Energy (DOE) have been looking at ways, strategies and programs to reduce this consumption.
Subzero Engineering has sent a team through the DCEP training and has been helping customers realize their potential savings. Utilizing the DOE toolset DC Profiler, Utilizing the DOE toolset DC Profiler, Subzero’s DCEP Certified Engineers can help you with many important items.
We interviewed Gordon Johnson who is a certified data center design professional, Data Center Energy Practitioner (DCEP), CFD and electrical engineer regarding the use of CFD’s and containment.
Gordon, what is the principle way CFD’s are used with regard to containment?
We use CFD’s to determine two basic data sets. The first is the baseline, or the current airflow pattern. This initial CFD model shows supply intake temperatures to each cabinet. This model also determines the effectiveness of each AC unit as it relates to airflow volume, return air temperature, delta T, and supply air temperature.
The second model is the proposed design of the CFD engineer who uses the information from the base model to enact airflow management best practices to separate supply from return airflow. Typically several models are created in order to adjust airflow volume, set point temperatures, and adjust individual aisle supply volume.
Gordon, Are there situations in which the CFD engineer does not recommend containment?
Not really, because the entire basis of airflow management is the full separation of supply and return airflow. Anytime these two airflows mix there is a loss of energy and consistent supply temperature to the IT thermal load.
We have seen CFD’s used by manufactures to prove product effectiveness. What are some ways CFD’s are made to exaggerate product effectiveness?
Exaggerations usually stem from the principle known as GIGO, short for Garbage In, Garbage Out. This refers to the fact that computers operate by logical processes, and thus will unquestioningly process unintended, even nonsensical input data (garbage in) and produce undesired, often nonsensical output (garbage out).
Let me give you an example. Recently I recreated a CFD model that was used to explain the effectiveness of airflow deflectors. The purpose of the CFD was to show the energy savings difference between airflow deflectors and full containment. We found that certain key data points were inserted into the models that do not reflect industry standards. Key settings were adjusted to fully optimize energy savings without regard to potential changes to the environment. Any potentially adverse effects to the cooling system’s ability to maintain acceptable thermal parameters, due to environmental changes, are not revealed in the CFD model. Thus, the model was operating on a fine line that could not be adjusted without a significant impact on its ability to cool the IT load.
Can you give us any specifics?
The airflow volume was manually changed from 1 kW at 154 CFM to 1 kW at 120 CFM. Industry standard airflow is 154 CFM. The formula most commonly used is as such:
120 CFM airflow does not give the cooling system any margin for potential changes to the environment.
Another key area of unrealistic design is the placement of cabinet thermal load and high volume grates. The base model places high kW loads in specific, isolated areas surrounded by high volume grates. What then happens, if additional load is placed in areas of low volume airflow? Any changes to the rack kW in areas without high volume grates could not be accounted for. At the end of the day, any changes to the IT load would require an additional airflow management audit to determine what changes would affect the cooling solution. Thus, the proposed model is unrealistic because no data center would propose a cooling solution that would require regular modifications.
Are you recommending a CFD study every time you make changes to the data center thermal load?
No. a full separation supply and return airflow eliminates the guesswork with regards to the effect of air mixture. It also eliminates the need of specific high volume perforated tiles or grates to be placed in front of high kW loads. Instead, a CFD model would incorporate expected increases to the aisle thermal load. This falls in line with the “plus 1” kind of approach to cooling. Creating a positive pressure of supply air has many additional benefits, such as lowering IT equipment fan speed, and ensuring consistent supply temperature across the face of the IT intake.
Data centers should not be operated with little margin for changes or adjustments to the thermal load. That is why I always recommend a full containment solution with as close to 0% leakage as possible. This is always the most efficient way to run a data center, and always yields the best return on investment. The full containment solution, with no openings at the aisle-end doors or above the cabinets, will easily allow the contained cold aisles to operate with a slightly greater supply of air than is demanded. This in turn ensures that the cabinets in the fully contained aisle have a minimum temperature change from the bottom to the top of the rack, which allows the data center operator to easily choose predictable and reliable supply temperature set points for the cooling units. The result? Large energy savings, lower mean time between failures, and a more reliable data center.
What do you recommend as to the use of CFD studies and containment?
It’s important to create both an accurate baseline and a sustainable cooling solution design. This model will give data center operators a basis for an accurate representation of how things are being cooled. The proposed cooling solution can be used in numerous ways:
- Accurate energy savings
- Safe set point standards
- Future cabinet population predictions
- The ability to cool future kW increases
- Identify and eliminate potential hot spots
Subzero Engineering endorses accurate and realistic CFD modeling that considers real world situations in order to create real world solutions.