Rack Hygiene – Stop the Madness!

Rack Hygiene – Stop the Madness!

We’ve all heard of personal hygiene, and perhaps you’ve heard of mental hygiene, but what about rack hygiene? Please don’t look this up in a dictionary it will only confuse you. The word hygiene is now associated with data center cabinets or racks; which is a good thing. Why? Because the word hygiene makes people think of practices that maintain health and prevents disease. The word cleanliness also comes to mind. We all want clean bodies and minds. What about your data center racks?

Think about it this way… What happens to your mental state when you must make a cable change and the cable management system looks like a bowl of spaghetti? How’s your mental health now? Is it possible that your thoughts are moving to the dark side, unclean? Would you like to meet the guy in purchasing that saved the company $200.00 per cabinet who has no clue about the time lost in network cable mining?

Stop the madness! Rack hygiene is a must for every data center. Saving a few dollars on cabinets without cable management systems is nothing short of crazy.

The time wasted on unmanaged cable during the life of the cabinets will easily outweigh the additional costs for cable management.

What we’ve learned over the years is that asking technicians to mastermind a cable management program without the proper tools is like going into battle with a slingshot instead of a rifle.

My mom use to say in her lilting voice, “A place for everything and everything in its place.” Mom was borrowing an expression that came from the 1800s that has been attributed to many sources. My favorite is the quote from Masterman Ready, or the Wreck of the Pacific in 1842 that uses the expression in a nautical context “In a well-conducted man-of-war every thing is in its place, and there is a place for every thing”.

Boats don’t have much room, so its imperative to stow everything is such a way that it can be easily found and ready for use.

The same can be said for cabinets, there is no room for clutter. A properly organized cabinet goes a long way in new equipment deployment, as well as tracking down outages.

The point we want to make is that rack hygiene and cable management begins during the purchasing phase of the racks and the cabinets. Not all cable management systems are created equal, nor for the same purpose. Here are some important variables to consider:

• Vertical cable managers
• Horizontal cabling systems
• Backbone cable installations
• Copper
• Fiber
• Maintenance holes
• Bonding & grounding
• Support facilities such as raceways, cable trays holes coring, slot and sleeves
• Fastener types
• Wireless systems

Take the time to design a cabinet that makes cable hygiene easy. Without it your technician’s mental health will be anything but clean!

Cable MaintenanceCable ManagementCablesCabling Systemsdata centerdata center cabinetsdata center racksLarry MainersRack HygieneSubzero Engineering
The Correlation of Data Center Best Practices and Disaster Recovery

The Correlation of Data Center Best Practices and Disaster Recovery

A significant part of my career has been spent responding to, and assisting with data center disaster recovery. I have witnessed some rather spectacular disaster recovery situations, from the Bank of New York adjacent to the fallen World Trade Center towers, to the government offices that were affected by the Oklahoma City bombing and the river that flowed through the Estes Trucking data center in northern Virginia.

Through the years, I have noticed an interesting trend in the ability to recover high tech sites from disaster: there is a correlation between a data center’s attention to industry best practices and its ability to rebound from the disaster. A company that had instituted a cable management strategy had a greater ability to recover from a flood than a company who hadn’t prioritized cable management.   Similarly, the company that employed airflow management prior to an event was more prepared during recovery efforts than those that had no defined cooling strategy.

When disaster strikes a data center, one of the most profound effects on employees is the changed appearance of their site; nothing looks the same. Instead of clean walls, floors and ceilings, the facility is often extremely dirty, with missing tiles and computers covered in disaster residue. I have noticed that such stark changes can initially cause managers to delay key disaster recovery decisions, negatively affecting the recovery effort.

In contrast, those managers who are up to date on industry best practices are better able to react during disaster recovery. Such managers are better attuned to the condition of their data centers, and don’t require extra time to recreate the conditions of the data center prior to the disaster. Adhering to industry best practices reduces the reaction time to a disaster, making the recovery process faster more efficient.

When data center managers stay up to date with important best practices, it will benefit them in being prepared for the unexpected.

Larry Mainers / CEO / Subzero Engineering

data centerData Center DisasterDisaster RecoveryIndustry Best PracticesLarry MainersSubzero Engineering
The Role of CFDs in Containment

The Role of CFDs in Containment

Data center airflow management engineers have used Computational Fluid Dynamics (CFD) programs for years to determine the complicated movement of airflow in data centers. CFD models pinpoint areas where airflow can be improved in order to provide a consistent cooling solution and energy savings.

We interviewed Gordon Johnson who is a certified data center design professional, Data Center Energy Practitioner (DCEP), CFD and electrical engineer regarding the use of CFD’s and containment.

Gordon, what is the principle way CFD’s are used with regard to containment?

We use CFD’s to determine two basic data sets. The first is the baseline, or the current airflow pattern. This initial CFD model shows supply intake temperatures to each cabinet. This model also determines the effectiveness of each AC unit as it relates to airflow volume, return air temperature, delta T, and supply air temperature.

The second model is the proposed design of the CFD engineer who uses the information from the base model to enact airflow management best practices to separate supply from return airflow. Typically several models are created in order to adjust airflow volume, set point temperatures, and adjust individual aisle supply volume.

Gordon, Are there situations in which the CFD engineer does not recommend containment?

Not really, because the entire basis of airflow management is the full separation of supply and return airflow. Anytime these two airflows mix there is a loss of energy and consistent supply temperature to the IT thermal load.

We have seen CFD’s used by manufactures to prove product effectiveness. What are some ways CFD’s are made to exaggerate product effectiveness?

Exaggerations usually stem from the principle known as GIGO, short for Garbage In, Garbage Out. This refers to the fact that computers operate by logical processes, and thus will unquestioningly process unintended, even nonsensical input data (garbage in) and produce undesired, often nonsensical output (garbage out).

Let me give you an example. Recently I recreated a CFD model that was used to explain the effectiveness of airflow deflectors. The purpose of the CFD was to show the energy savings difference between airflow deflectors and full containment. We found that certain key data points were inserted into the models that do not reflect industry standards. Key settings were adjusted to fully optimize energy savings without regard to potential changes to the environment. Any potentially adverse effects to the cooling system’s ability to maintain acceptable thermal parameters, due to environmental changes, are not revealed in the CFD model. Thus, the model was operating on a fine line that could not be adjusted without a significant impact on its ability to cool the IT load.

Can you give us any specifics?

The airflow volume was manually changed from 1 kW at 154 CFM to 1 kW at 120 CFM. Industry standard airflow is 154 CFM. The formula most commonly used is as such:


120 CFM airflow does not give the cooling system any margin for potential changes to the environment.

Another key area of unrealistic design is the placement of cabinet thermal load and high volume grates. The base model places high kW loads in specific, isolated areas surrounded by high volume grates. What then happens, if additional load is placed in areas of low volume airflow? Any changes to the rack kW in areas without high volume grates could not be accounted for. At the end of the day, any changes to the IT load would require an additional airflow management audit to determine what changes would affect the cooling solution. Thus, the proposed model is unrealistic because no data center would propose a cooling solution that would require regular modifications.

Are you recommending a CFD study every time you make changes to the data center thermal load?

No. a full separation supply and return airflow eliminates the guesswork with regards to the effect of air mixture. It also eliminates the need of specific high volume perforated tiles or grates to be placed in front of high kW loads. Instead, a CFD model would incorporate expected increases to the aisle thermal load. This falls in line with the “plus 1” kind of approach to cooling. Creating a positive pressure of supply air has many additional benefits, such as lowering IT equipment fan speed, and ensuring consistent supply temperature across the face of the IT intake.

Data centers should not be operated with little margin for changes or adjustments to the thermal load. That is why I always recommend a full containment solution with as close to 0% leakage as possible.  This is always the most efficient way to run a data center, and always yields the best return on investment. The full containment solution, with no openings at the aisle-end doors or above the cabinets, will easily allow the contained cold aisles to operate with a slightly greater supply of air than is demanded.  This in turn ensures that the cabinets in the fully contained aisle have a minimum temperature change from the bottom to the top of the rack, which allows the data center operator to easily choose predictable and reliable supply temperature set points for the cooling units.  The result?  Large energy savings, lower mean time between failures, and a more reliable data center.

What do you recommend as to the use of CFD studies and containment?

It’s important to create both an accurate baseline and a sustainable cooling solution design. This model will give data center operators a basis for an accurate representation of how things are being cooled. The proposed cooling solution can be used in numerous ways:

  • Accurate energy savings
  • Safe set point standards
  • Future cabinet population predictions
  • The ability to cool future kW increases
  • Identify and eliminate potential hot spots

Subzero Engineering endorses accurate and realistic CFD modeling that considers real world situations in order to create real world solutions.

Airflow ManagementCFDCold Aisle ContainmentComputational Fluid DynamicsContainmentdata centerData Center Containmentdata center coolingGordon JohnsonHot Aisle ContainmentLarry MainersSubzero Engineering
The Truth About Fans in the Data Center

The Truth About Fans in the Data Center

And how this influences data center airflow management.

Sorry sports fans… this is not about your favorite team. Instead we are going to explore the fascinating world of mechanical fans.

How many times have you seen vender illustrations of fans pushing air in long blue lines from perforated raised floor tiles into the intake of a rack? The truth is that air does not move in such a way.  Calculating the airflow induced by one particular fan at any given distance away from the fan, about any point of the fans face is a very involved set of calculations.

Traditional thermal designs for fans were originally measured as jet velocity of water jets. This presented a close estimate, but inaccurate data. A recent study in 2012 helped in creating very accurate formulas as to fan outlet velocity and distributions.

Fan Outlet Velocity Distributions and Calculations
Eli Gurevich, Michael Likov  (Intel Corporation, Israel Design Center, Haifa, Israel)
David Greenblatt, Yevgeni Furman, Iliya Romm (Technion Institute of Technology, Haifa, Israel)

Generally, volumetric flow rate and distance traveled decreases when contained air enters ambient room air, and this is why mechanical air contractors use ductwork or a contained plenum to direct supply air to the thermal load. Increasing the velocity of air in order to reach the thermal load, instead of using a duct system, is considered inefficient.

It’s important to understand the relationship of mechanical air movement from fans and what actually happens to the airflow. The issue with fans is the manufacturer’s stated CFM capacity, and the distance of air movement that the fan is capable of will carry it. This value reflects what the fan is able to produce in a given test environment. Manufacturer stated air displacement (CFM) is based on what is called normal temperature and pressure conditions (NTP).  The actual volume of air that a fan can displace varies due to two factors:

1) Air density (hot, low density or cold, high density)
2) Air pressure (positive or negative)

Thus it is important to determine the manufacturer’s test conditions for the fan, and then compare the data to the actual planned environment in which the fan will operate.

For example, when considering the installation of a fan in the subfloor to move subfloor air into the cold aisle, the first question that should be addressed is: “what is the temperature of the air and head pressure that the fan will operate in?”

Why? The temperature of the air will determine its density when confined to a constant volume. In most cases, the subfloor air is denser, which is good.  Thus the more important question will be about the subfloor pressure. It is not unusual to have negative pressure areas in the subfloor due to high velocity air steams. The Bernoulli principle explains our concern, in that an increase of air speed will result in a decrease of air pressure. Additionally, when two air streams of high velocity air intersect from opposing directions, the result is often a subfloor vortex, resulting in the reversal of current.

So what’s the point? Imagine putting a raised floor fan system over an area with negative pressure. This would negatively affect the fan’s ideal operating conditions.

Consider this, what is the typical reason for using additional fans to move air into the cold aisle? Most likely the unassisted perforated tile or grate is not able to deliver sufficient airflow to the thermal load of the racks. What if this is based on inadequate subfloor pressure? If that is the case, adding a fan assisted raised floor panel will require taking into consideration the fan NTP. Also it will can drastically and unpredictably impact other areas of the data center as you “rob Peter to pay Paul” so to speak.

Consider the following subfloor airflow management strategies:

1) Eliminate high velocity air: This will ensure a more balanced delivery of air due to a nominalized subfloor pressure.
2) Cold Aisle Containment: Instead of designing rack cooling by placing an airflow-producing raised floor tile at the feet of each rack, why not create a cold aisle that is not dependent on perforated tile placement?

Cold aisle containment creates a small room of supply air that can be accessed by all IT equipment fans. Instead of managing each supply raised floor tile, the only requirement is ensuring positive air pressure in the aisle. Cold aisle containment systems provide several benefits: most contained cold aisles will only have a one-degree differential from the bottom to the top of the rack, and the cold aisle containment does not require high air velocity, which can create other airflow management problems, such as bypassing IT equipment intake.

Understanding the NTP conditions of IT equipment cooling fans is an important aspect of data center airflow management. For example, in order to properly adjust CRAC unit set points, it is important to know the temperature at which the supply air’s density will drop below each fan’s NTP conditions.  It is possible to lower the supply temperature to a level at which an increase in fan speed would be required to make up for the less dense airflow, potentially offsetting any energy savings from a higher cooling set point.

Simply adding fans to cool IT equipment is not a quick fix; it is imperative to first understand why sufficient airflow is not available. It is important to understand the fan’s NTP in the proposed environment, and to see if you can supply IT equipment with consistent airflow by simply separating supply and return air through data center containment. Containment can prevent the unnecessary use of additional electricity that is required to operate fans, saving money and electricity in the long term.

AirflowAirflow ManagementContainmentdata centerData Center Containmentdata center coolingData Center FansLarry MainersSubzero Engineering
The Truth Behind Data Center Airflow Management: It’s Complicated

The Truth Behind Data Center Airflow Management: It’s Complicated

Does hot air rise? The answer of course is “yes”.

Does hot air fall? The answer is yes again.

What about sideways? Yes!

Heat can move up, down, or sideways, depending on the situation. The idea that hot air has an inherent desire to flow up is a misconception that we in the data center airflow management business would like to see dissipate.

Temperature difference is the major factor with regards to the direction and rate of heat transfer. Because air tends to move towards thermal equilibrium, it is important to maintain physical separation of hot and cold air in data centers; the need for hot and cold air separation was the reason that the data center containment industry came into existence. The laws of thermodynamics state that air moves from areas of higher temperature towards areas of lower temperature. Air is a fluid that accounts for both density and buoyancy. When air is heated the molecules move around faster, which causes it to expand, and as it expands its density becomes lower. The warmer, lower density air will rise above the denser, cooler air.

Pressure is another determining factor when looking at air movement. The flow of air from areas of high pressure to areas of low pressure is an embodiment of Newton’s third law. Equilibrium is what also drives movement between areas of differing pressure, so uninhibited air will continuously move from high to low pressure until equilibrium is reached. This movement towards equilibrium is also known as expansion.

Principles of air movement:
1) Heat Transfer:
a. Conduction: Air flows from a higher temperature region to a lower temperature between mediums
that make physical contact.
b. Convection: Heat transfer due to the movement of a fluid; can be free/natural, or forced.
2) Air flows from a higher pressure to a lower pressure

What does this have to do with data center airflow management?

The data center containment industry has been inundated with graphs depicting airflow, most of which show large, sweeping lines indicating the flow of air. In most cases, the airflow depicted is a result of a mechanical device, usually a fan. The data presented by these graphs tends to lead one to believe that mechanically induced airflow will sufficiently separate hot exhaust air from cold intake air. In real-world scenarios, air curtains are inefficient and ineffective.

Modern mechanical air conditioning systems rely on four sided duct systems to deliver supply air to the source of the heat load, and the return is moved by the same means. This is the only way to ensure the separation of supply and return airflow. Systems administrators and building managers should be dubious of airflow management systems that require an increase in energy to accomplish air separation. Instead, it is best to apply the simplest principles of airflow when designing a system aimed at full separation of supply and return airflow.

If you would like to learn more about the flow of air, please see the following link:

Learn How Air Moves Through This Incredible Optical Device

AirflowAirflow ManagementContainmentdata centerData Center Containmentdata center coolingLarry MainersSubzero Engineering
Extending the Capacity of the Data Center Using Hot or Cold Aisle Containment

Extending the Capacity of the Data Center Using Hot or Cold Aisle Containment

What correlation does consistent supply air across the face of the rack have to do with increased data center capacity?

Hot or Cold Aisle Containment can significantly increase the capacity of a data center when all U’s are fully populated due to consistent intake temperatures across the rack face.

Additionally, when cooling energy can be converted to IT equipment this too can elongate the life of a data center that is running out of power.

Problem Statement – Air Stratification
Most data centers without containment have air stratification. Air stratification occurs when supply and return air mix. This creates several temperature layers along the intake face of the rack. It is not uncommon for the temperature at the bottom of the rack to be 8 to 10 degrees colder than the top. As a result, many data centers have implemented policies that do not allow the top 6 to 8 U’s to be populated. This can decrease the data centers IT equipment capacity by 16%. Capacity from a space perspective is one thing, but when the unpopulated U’s are potentially high density systems the lost space is amplified.

Click here to read the latest Subzero Engineering White Paper “Extending the Capacity of a Data Center”.


Airflow ManagementCold Aisle Containmentdata centerData Center CapacityData Center ContainmentHot Aisle ContainmentLarry MainersSubzero Engineering
Check out our new fully NFPA compliant retractable roof system!

Check out our new fully NFPA compliant retractable roof system!

This game changing roof system ensures that the containment roof obstruction is fully removed electronically when a smoke detector is alarmed. Additional benefits include the ability to wirelessly open the roof for maintenance above the containment, modular design allowing for increase or decrease of size, ease of deployment, and the sleek look of the roof is easy on the eyes. This is the ultimate containment roof system.

Polar Cap Retractable Roof Available Fall of 2014!

The patented Subzero Engineering Polar Cap is the first fully NFPA compliant containment roof system that attaches to the top of the racks and forms a ceiling that prevents hot and cold air from mixing.

Most data center containment systems rely on the heat generated from a fire related incident to release the containment system, as it can pose an obstacle to the fire suppression agent. The NFPA has determined that it is important to have a faster response time and more importantly, a testable system.

The updated Subzero Polar Cap retractable roof system is now a fully electric roof system that retracts into a metal housing when the fire suppression system is alarmed. Having a pre-action system that reacts to a smoke detector will ensure that the containment roof is fully retracted long before the fire suppression system is discharged. Additionally, the roof material is made with the highest fire resistant standard of ASTM E-84 Class A rating.

The Polar Cap can also be opened and closed when maintenance is required above the containment space.

The new roof system is fully customizable in both length (up to 30’) and width (up to 5’). The aluminum profile is less than 6” high and thus it presents no problem with obstructions above the cabinets.

ASTM E-84Cold Aisle ContainmentContainmentcontainment roofData Center Containmentdata center containment roofelectric roofLarry Mainersnfpanfpa compliantRetractable Roofroof systemSubzero Engineering
Airflow Management’s Role in Data Center Cooling Capacity

Airflow Management’s Role in Data Center Cooling Capacity

New White Paper – Airflow Management’s Role in Data Center Cooling Capacity, by Larry Mainers

Airflow management (AFM) is changing the way data centers cool the IT thermal load. In the simplest terms AFM is the science of separating the cooling supply from the hot return airflow.

AFM’s impact on cooling capacity is huge. This is because the traditional cooling scenario without the full separation of supply and return airflow requires as much as four times the cooling capacity to satisfy the same thermal load. This creates the unnecessary need for cooling units due to airflow inefficiency.

Data center managers can easily determine the percentage of inefficiency by counting the tons of available cooling capacity and measuring it against the IT thermal load measured in kW.

To read more click here.

Data Center Containmentdata center coolingLarry MainersSubzero EngineeringUNIFLOW
Subzero is pleased to announce that the chairperson of the Data Center World Conference suggests our talk on Energy Rebates as one of the top 13 sessions to attend.

Subzero is pleased to announce that the chairperson of the Data Center World Conference suggests our talk on Energy Rebates as one of the top 13 sessions to attend.

Larry Mainers, CEO of Subzero Engineering will be giving a session explaining “How to Apply and Receive Utility Rebates with Environmental Monitoring Systems”. This case study will show how a California data center received a large rebate from PG&E after DCEP trained experts implemented a wireless monitoring system that provided key before and after data needed to verify the energy savings to the utility company.

Please join us and see how Subzero’s DCEP trained engineers can assist you through the entire energy rebate process.

Wednesday, April 30, 2014
10:30 – 11:30 am
FAC 3.1
How to Apply and Receive Utility Rebates with Environmental Monitoring Systems
Presented by: Larry Mainers, CEO, Subzero Engineering

Data Center ContainmentDCEPenergy rebatesEnvironmental MonitoringLarry MainersSubzero Engineeringutility rebates
You can’t change the wind, but you can adjust the sails.

You can’t change the wind, but you can adjust the sails.

Business is a lot like sailing; both require people (crew), equipment, a starting point, a destination, and of course, the voyage. Like business, sailing can be a lot of fun: sun, sea, sails and wind… not too shabby. And just like life, sailing has its good and bad days. On good days the wind is in your favor, on bad days… not so much. Thus, the analogy goes. Based on this, it would seem that our ability to adjust the sails on the contrary days is the difference between success and failure.

Sadly, it’s not always as simple as adjusting the sails. For example, try adjusting your sails in a hurricane.

Positive thinking alone does not equate to success, but it can be a powerful ally. The facts are that many positive, bright, and talented people venture out into the world of business only to be brushed back by the strong winds of adversity. What then defines the companies that consistently prove successful where others have not?

The answer can be found with the captain and crew.

When you align yourself with talented, positive, and bright people the winds of adversity are mitigated. Surrounding yourself with great people creates dynamics that cannot be easily defeated. No wind can change the direction of an army of dedicated people.

Our world has a way of glorifying the individual, but most of the time that individual shines because of the team they have. For instance it’s always the star quarterback that is interviewed after the game. But if you privately ask the quarterback, he will tell you that without the great people surrounding him he could not advance the football even one yard.

I have had the privilege of owning and operating several successful companies. In each case what made the company successful were the people.

I am proud to be at the helm of Subzero Engineering. Our crew is filled with talented, bright, and passionate people who work as a team. Combined, we are a force to be reckoned with. Instead of one person with a vision, we are many with a singular focus.

There is an old proverb that says “And if somebody could overpower one alone, two together could make a stand against him. And a threefold cord cannot quickly be torn in two.” It is essential in sailing to use lines or ropes that use multiple strands, that create incredible strength. A great business joins the talents and drive of many people into one singular goal, one cord. Additionally, great people make the joys and challenges of business a real pleasurable experience. For me –  this makes the voyage, not the destination, a far better experience.

My motto is simple; if you surround yourself with great people who share your vision, you will find that when the wind of adversity blows you can hold the course by merely adjusting the sails.

Larry Mainers / CEO / Subzero Engineering

Larry Mainers