Visit us at Data Center World
March 14 – 18, 2016 • Las Vegas • Booth #601
The Subzero Arctic Enclosure was designed to support the dynamic needs of today’s data centers by supporting airflow management needs out of the box. This enclosure can support all types of data center demands from low density, to high density; data closets, to enterprise data centers.
Subzero now combines our cutting edge containment and cabinet solutions with power management. These items combined create the most powerful ‘plug and play’ solution in the industry. Over 200 configurations of the Polar PDUs are available to be custom configured.
Subzero’s cutting edge containment is custom built to meet our customer’s most daunting challenges. Hot Aisle Containment, Cold Aisle Containment, Isolated Equipment Containment, Doors, Roofs, Retractable Roofs, Floor Panels, Above Rack Panels… We have your data center covered.
While at the show, join us for the following presentations:
NFPA Compliant Containment
Wednesday, March 16 10:45 – 11:45
The Co-relationship of Containment and Computational Fluid Dynamics
Tuesday, March 15 9:30-10:30
Subzero Data Center cold and/or hot aisle containment is the best way to lower intake temperature to IT equipment.
What the Evidence Shows in Real Time
The IBM data center efficiency group in New York wanted the same proof. Gerry Weber an engineering consultant at IBM along with other monitoring technicians recorded a time-lapse video that shows the containment install along side of the temperature changes.
In the video, you can see the temperature dropped nearly 14 degrees in a 5 hour period! What the video does not display is that the temperature across the face of the IT intake did not vary more than one degree.
Subzero Engineering has similar data from numerous data centers with an average of a 10 degree in supply temperature. At the same time the intake Relative Humidity Levels were increased by over 20%.
What does this mean for data center operators?
- Consistent supply temperatures
- Increase use of rack space due to consistent supply temperatures at the top of the rack.
- Predictable supply temperatures make it easy to anticipate cooling solutions when an increase of thermal load or kW is introduced to the space.
- Maximize cooling efficiency by adopting ASHRAE increase in temperature.
- Convert cooling energy to IT equipment.
We interviewed Gordon Johnson who is a certified data center design professional, Data Center Energy Practitioner (DCEP), CFD and electrical engineer regarding the use of CFD’s and containment.
Gordon, what is the principle way CFD’s are used with regard to containment?
We use CFD’s to determine two basic data sets. The first is the baseline, or the current airflow pattern. This initial CFD model shows supply intake temperatures to each cabinet. This model also determines the effectiveness of each AC unit as it relates to airflow volume, return air temperature, delta T, and supply air temperature.
The second model is the proposed design of the CFD engineer who uses the information from the base model to enact airflow management best practices to separate supply from return airflow. Typically several models are created in order to adjust airflow volume, set point temperatures, and adjust individual aisle supply volume.
Gordon, Are there situations in which the CFD engineer does not recommend containment?
Not really, because the entire basis of airflow management is the full separation of supply and return airflow. Anytime these two airflows mix there is a loss of energy and consistent supply temperature to the IT thermal load.
We have seen CFD’s used by manufactures to prove product effectiveness. What are some ways CFD’s are made to exaggerate product effectiveness?
Exaggerations usually stem from the principle known as GIGO, short for Garbage In, Garbage Out. This refers to the fact that computers operate by logical processes, and thus will unquestioningly process unintended, even nonsensical input data (garbage in) and produce undesired, often nonsensical output (garbage out).
Let me give you an example. Recently I recreated a CFD model that was used to explain the effectiveness of airflow deflectors. The purpose of the CFD was to show the energy savings difference between airflow deflectors and full containment. We found that certain key data points were inserted into the models that do not reflect industry standards. Key settings were adjusted to fully optimize energy savings without regard to potential changes to the environment. Any potentially adverse effects to the cooling system’s ability to maintain acceptable thermal parameters, due to environmental changes, are not revealed in the CFD model. Thus, the model was operating on a fine line that could not be adjusted without a significant impact on its ability to cool the IT load.
Can you give us any specifics?
The airflow volume was manually changed from 1 kW at 154 CFM to 1 kW at 120 CFM. Industry standard airflow is 154 CFM. The formula most commonly used is as such:
120 CFM airflow does not give the cooling system any margin for potential changes to the environment.
Another key area of unrealistic design is the placement of cabinet thermal load and high volume grates. The base model places high kW loads in specific, isolated areas surrounded by high volume grates. What then happens, if additional load is placed in areas of low volume airflow? Any changes to the rack kW in areas without high volume grates could not be accounted for. At the end of the day, any changes to the IT load would require an additional airflow management audit to determine what changes would affect the cooling solution. Thus, the proposed model is unrealistic because no data center would propose a cooling solution that would require regular modifications.
Are you recommending a CFD study every time you make changes to the data center thermal load?
No. a full separation supply and return airflow eliminates the guesswork with regards to the effect of air mixture. It also eliminates the need of specific high volume perforated tiles or grates to be placed in front of high kW loads. Instead, a CFD model would incorporate expected increases to the aisle thermal load. This falls in line with the “plus 1” kind of approach to cooling. Creating a positive pressure of supply air has many additional benefits, such as lowering IT equipment fan speed, and ensuring consistent supply temperature across the face of the IT intake.
Data centers should not be operated with little margin for changes or adjustments to the thermal load. That is why I always recommend a full containment solution with as close to 0% leakage as possible. This is always the most efficient way to run a data center, and always yields the best return on investment. The full containment solution, with no openings at the aisle-end doors or above the cabinets, will easily allow the contained cold aisles to operate with a slightly greater supply of air than is demanded. This in turn ensures that the cabinets in the fully contained aisle have a minimum temperature change from the bottom to the top of the rack, which allows the data center operator to easily choose predictable and reliable supply temperature set points for the cooling units. The result? Large energy savings, lower mean time between failures, and a more reliable data center.
What do you recommend as to the use of CFD studies and containment?
It’s important to create both an accurate baseline and a sustainable cooling solution design. This model will give data center operators a basis for an accurate representation of how things are being cooled. The proposed cooling solution can be used in numerous ways:
- Accurate energy savings
- Safe set point standards
- Future cabinet population predictions
- The ability to cool future kW increases
- Identify and eliminate potential hot spots
Subzero Engineering endorses accurate and realistic CFD modeling that considers real world situations in order to create real world solutions.
And how this influences data center airflow management.
Sorry sports fans… this is not about your favorite team. Instead we are going to explore the fascinating world of mechanical fans.
How many times have you seen vender illustrations of fans pushing air in long blue lines from perforated raised floor tiles into the intake of a rack? The truth is that air does not move in such a way. Calculating the airflow induced by one particular fan at any given distance away from the fan, about any point of the fans face is a very involved set of calculations.
Traditional thermal designs for fans were originally measured as jet velocity of water jets. This presented a close estimate, but inaccurate data. A recent study in 2012 helped in creating very accurate formulas as to fan outlet velocity and distributions.
Fan Outlet Velocity Distributions and Calculations
Eli Gurevich, Michael Likov (Intel Corporation, Israel Design Center, Haifa, Israel)
David Greenblatt, Yevgeni Furman, Iliya Romm (Technion Institute of Technology, Haifa, Israel)
Generally, volumetric flow rate and distance traveled decreases when contained air enters ambient room air, and this is why mechanical air contractors use ductwork or a contained plenum to direct supply air to the thermal load. Increasing the velocity of air in order to reach the thermal load, instead of using a duct system, is considered inefficient.
It’s important to understand the relationship of mechanical air movement from fans and what actually happens to the airflow. The issue with fans is the manufacturer’s stated CFM capacity, and the distance of air movement that the fan is capable of will carry it. This value reflects what the fan is able to produce in a given test environment. Manufacturer stated air displacement (CFM) is based on what is called normal temperature and pressure conditions (NTP). The actual volume of air that a fan can displace varies due to two factors:
1) Air density (hot, low density or cold, high density)
2) Air pressure (positive or negative)
Thus it is important to determine the manufacturer’s test conditions for the fan, and then compare the data to the actual planned environment in which the fan will operate.
For example, when considering the installation of a fan in the subfloor to move subfloor air into the cold aisle, the first question that should be addressed is: “what is the temperature of the air and head pressure that the fan will operate in?”
Why? The temperature of the air will determine its density when confined to a constant volume. In most cases, the subfloor air is denser, which is good. Thus the more important question will be about the subfloor pressure. It is not unusual to have negative pressure areas in the subfloor due to high velocity air steams. The Bernoulli principle explains our concern, in that an increase of air speed will result in a decrease of air pressure. Additionally, when two air streams of high velocity air intersect from opposing directions, the result is often a subfloor vortex, resulting in the reversal of current.
So what’s the point? Imagine putting a raised floor fan system over an area with negative pressure. This would negatively affect the fan’s ideal operating conditions.
Consider this, what is the typical reason for using additional fans to move air into the cold aisle? Most likely the unassisted perforated tile or grate is not able to deliver sufficient airflow to the thermal load of the racks. What if this is based on inadequate subfloor pressure? If that is the case, adding a fan assisted raised floor panel will require taking into consideration the fan NTP. Also it will can drastically and unpredictably impact other areas of the data center as you “rob Peter to pay Paul” so to speak.
Consider the following subfloor airflow management strategies:
1) Eliminate high velocity air: This will ensure a more balanced delivery of air due to a nominalized subfloor pressure.
2) Cold Aisle Containment: Instead of designing rack cooling by placing an airflow-producing raised floor tile at the feet of each rack, why not create a cold aisle that is not dependent on perforated tile placement?
Cold aisle containment creates a small room of supply air that can be accessed by all IT equipment fans. Instead of managing each supply raised floor tile, the only requirement is ensuring positive air pressure in the aisle. Cold aisle containment systems provide several benefits: most contained cold aisles will only have a one-degree differential from the bottom to the top of the rack, and the cold aisle containment does not require high air velocity, which can create other airflow management problems, such as bypassing IT equipment intake.
Understanding the NTP conditions of IT equipment cooling fans is an important aspect of data center airflow management. For example, in order to properly adjust CRAC unit set points, it is important to know the temperature at which the supply air’s density will drop below each fan’s NTP conditions. It is possible to lower the supply temperature to a level at which an increase in fan speed would be required to make up for the less dense airflow, potentially offsetting any energy savings from a higher cooling set point.
Simply adding fans to cool IT equipment is not a quick fix; it is imperative to first understand why sufficient airflow is not available. It is important to understand the fan’s NTP in the proposed environment, and to see if you can supply IT equipment with consistent airflow by simply separating supply and return air through data center containment. Containment can prevent the unnecessary use of additional electricity that is required to operate fans, saving money and electricity in the long term.
Does hot air rise? The answer of course is “yes”.
Does hot air fall? The answer is yes again.
What about sideways? Yes!
Heat can move up, down, or sideways, depending on the situation. The idea that hot air has an inherent desire to flow up is a misconception that we in the data center airflow management business would like to see dissipate.
Temperature difference is the major factor with regards to the direction and rate of heat transfer. Because air tends to move towards thermal equilibrium, it is important to maintain physical separation of hot and cold air in data centers; the need for hot and cold air separation was the reason that the data center containment industry came into existence. The laws of thermodynamics state that air moves from areas of higher temperature towards areas of lower temperature. Air is a fluid that accounts for both density and buoyancy. When air is heated the molecules move around faster, which causes it to expand, and as it expands its density becomes lower. The warmer, lower density air will rise above the denser, cooler air.
Pressure is another determining factor when looking at air movement. The flow of air from areas of high pressure to areas of low pressure is an embodiment of Newton’s third law. Equilibrium is what also drives movement between areas of differing pressure, so uninhibited air will continuously move from high to low pressure until equilibrium is reached. This movement towards equilibrium is also known as expansion.
Principles of air movement:
1) Heat Transfer:
a. Conduction: Air flows from a higher temperature region to a lower temperature between mediums
that make physical contact.
b. Convection: Heat transfer due to the movement of a fluid; can be free/natural, or forced.
2) Air flows from a higher pressure to a lower pressure
What does this have to do with data center airflow management?
The data center containment industry has been inundated with graphs depicting airflow, most of which show large, sweeping lines indicating the flow of air. In most cases, the airflow depicted is a result of a mechanical device, usually a fan. The data presented by these graphs tends to lead one to believe that mechanically induced airflow will sufficiently separate hot exhaust air from cold intake air. In real-world scenarios, air curtains are inefficient and ineffective.
Modern mechanical air conditioning systems rely on four sided duct systems to deliver supply air to the source of the heat load, and the return is moved by the same means. This is the only way to ensure the separation of supply and return airflow. Systems administrators and building managers should be dubious of airflow management systems that require an increase in energy to accomplish air separation. Instead, it is best to apply the simplest principles of airflow when designing a system aimed at full separation of supply and return airflow.
If you would like to learn more about the flow of air, please see the following link:
Learn How Air Moves Through This Incredible Optical Device
New White Paper – Airflow Management’s Role in Data Center Cooling Capacity, by Larry Mainers
Airflow management (AFM) is changing the way data centers cool the IT thermal load. In the simplest terms AFM is the science of separating the cooling supply from the hot return airflow.
AFM’s impact on cooling capacity is huge. This is because the traditional cooling scenario without the full separation of supply and return airflow requires as much as four times the cooling capacity to satisfy the same thermal load. This creates the unnecessary need for cooling units due to airflow inefficiency.
Data center managers can easily determine the percentage of inefficiency by counting the tons of available cooling capacity and measuring it against the IT thermal load measured in kW.
To read more click here.
James Glanz of The New York Times and Ken Brill, an expert in the field, tour a data center and cover the topics of data center cooling, cold aisle containment, backup power management, and energy efficiency.
A Subzero cold aisle containment system is shown at the 2:52 minute mark in the video. This is a large data center that features 6 backup diesel generators that could power a small city of about 30,000 people.
Link to video: http://nyti.ms/OMaDnu
Cold aisle containment systems typically include aisle end doors and a roof system so as to contain the supply airflow to the intake of the IT and server equipment. Cold aisle containment ensures uniform and predictable air temperature at the IT intake, allowing equipment to be cooled to manufactures’ specifications.
Learn more about cold aisle containment here.