More Cisco Press book chapters from new and classic Cisco Press books.
Rate your favorite Cisco Press books.
This chapter outlines the benefits and drawbacks of running power, data connections, and cooling into the Data Center by way of the ceiling versus installing a raised floor system and routing it underneath. The chapter details the components involved and offers sample illustrations for both overhead and under-floor systems as well as provides common problems to watch for and avoid.
Overhead Installation
In an overhead installation, structured cabling and electrical conduits are typically routed above the Data Center's false or suspended ceiling and then terminated directly above the room's server rows. Cooling is channeled above the false ceiling as well, then directed to the server environment below by way of adjustable vents.
The primary advantage of locating infrastructure overhead is that it enables you to forego a raised floor system for your server environment. Overhead installation is therefore less expensive, occupies less floor space, and fits more conveniently in shorter building spaces. Cable trays, ladder racks, and raceways are less expensive than a complete raised floor system, contributing to the cost savings. Less floor space is needed for an overhead installation because there is no entrance ramp, which—depending upon the height of the floor and how steeply the ramp is angled—can have a footprint of more than 200 square feet (18.6 square meters). This also provides some cost savings. Because likely there already has to be a pocket of space above the Data Center's suspended ceiling to accommodate wiring for the overhead lights and piping for sprinklers, any electrical and data cabling needed for the Data Center can be included in the overhead area as well. Elimination of a raised floor also does away with space required for the subfloor.
Be aware that installing infrastructure overhead and forgoing a raised floor does present a greater challenge for circulating air within the Data Center. Raised floor server environments are typically cooled by drawing warm air away through overhead ducted returns while blowing cold air under the floor. This process leverages the natural behavior of hot air to rise. It is significantly more difficult to cool a server environment by pushing cold air downward. Other effects of routing infrastructure overhead are mixed. Patching power cords and data cabling atop cabinet locations rather than below reduces the chances of someone snagging or damaging the cords as he or she walk by, with or without a pallet jack of equipment in tow. Data Center users must climb a stepladder to plug in to the room's infrastructure, however, which is potentially unstable and arguably more likely to trigger an accidental disconnect.
There is also a possibility of a power cable or patch cord coming unplugged due to gravity and the weight of the cord. If you choose to use an overhead system, mitigate this risk by using twist-lock power receptacles on your electrical outlets and corresponding plugs on your server cabinet power strips. Twist-lock electrical components require you to insert and then rotate an incoming power plug clockwise to lock it into position within the receptacle. Once in place the plug can't be pulled directly outward. It must first be rotated counter-clockwise. Likewise, make sure that the patch cords used in your server environment have unbroken tabs and, preferably, some form of support ladder to lessen the strain on their connectors. The patch cords should click firmly into place when inserted in an infrastructure port or server. This makes accidental disconnects less likely to occur.
Under-Floor Installation
In an under-floor installation, a system of horizontal and vertical bars is mounted on the Data Center's true floor, creating an elevated grid in which flat panels are placed. This creates a raised floor surface, under which structured cabling, electrical conduits, and cooled air may be routed. Sprinkler piping and leak detection infrastructure might be located here as well.
Structured cabling and electrical infrastructure typically terminates in the subfloor below each server cabinet location in multimedia data boxes and power receptacles that are either free-standing or incorporated into raceways under the Data Center server rows. Alternatively, infrastructure can be routed through pre-made openings in the raised floor and terminate into patch panels and power receptacles that are installed within the Data Center cabinets. Cooling is directed into the server environment above by the placement of solid and perforated floor tiles.
Most Data Centers are built with a raised floor system. Despite the additional cost, a raised floor provides several benefits for a server environment:
The raised floor creates a dedicated space to channel cooled air through. By strategically placing perforated or grated floor tiles, you focus and direct this cooling wherever it is needed in a server environment. There is no equivalent mechanism for controlling airflow in an overhead system. While it is possible to install multiple air vents overhead and adjust how far they are open or shut, they can't direct air in the same pinpoint manner that floor tiles can.
Routing infrastructure under a raised floor keeps hundreds or thousands of associated patch cords and power cables out of sight, which makes them less susceptible to damage or being unplugged accidentally. The absence of cabling or raceways in the Data Center's common area also gives the room a more professional and less cluttered appearance.
Although infrastructure is out of sight, it remains easily accessible under the floor, much more so than when located above Data Center server cabinets.
Note - The needs of your particular Data Center are going to determine whether it is worth the price and physical space necessary for installation of a raised floor system. When I design a server environment, I let its size drive whether to run infrastructure overhead or under floor.
If the Data Center is small—In round numbers, generally less than 1000 square feet (100 square meters)—or is being constructed in a building that has limited space from floor to ceiling, I design it with overhead infrastructure. The ramp needed for a raised floor and the height it occupies simply take up too much space in such a small room.
If the Data Center is larger—I design it with a raised floor and place its various infrastructure down below. I find the raised floor system essential for controlling airflow and neatly routing infrastructure. I've also learned that it is much simpler to lift a floor tile and plug in to or disconnect from infrastructure components than to climb a stepladder and stretch above a server cabinet to make connections.
Separation of Power and Data
No matter where you choose to route structured data cabling and electrical conduits in your Data Center, provide physical separation between the two. Electrical wiring can generate enough electromagnetic interference (EMI) to distort the information transmitted along nearby data cabling. Mixing power and data in a common bundle or within a single raceway can also be problematic if either infrastructure needs servicing. A vendor working on one faces the risk of disturbing or damaging the other.
Exactly how much separation to have between data cabling and electrical conduits, or between data cabling and any power source for that matter, is a gray area. No firm industry standards exist, although many cabling vendors or Data Center design consultants can recommend minimum distances.
Three factors contribute to how powerfully electromagnetic interference affects data cabling and therefore the amount of physical separation needed—the strength of the interference, whether the data cabling is unshielded or shielded, and whether cabling is encased in a raceway (typically metal or plastic).
Tables 5-1 and 5-2 show suggested minimum separation distances between data cabling and power sources. The distances, which are rounded off rather than straight conversions between measuring systems, are recommended by The Siemon Company.
Table 5-1 Recommended Minimum Separation for Unshielded Twisted Pair (UTP) Cabling
Power Level | Spaces | Pathways |
Less than 3 kva | 2 in. (50 mm.) | 2 in. (50 mm.) |
3–6 kva | 10 ft. (3 m.) | 5 ft. (1.5 m.) |
6 kva or more | 20 ft. (6 m.) | 10 ft. (3 m.) |
Table 5-2 Recommended Minimum Separation for Shielded Twisted Pair (STP) Cabling
Power Level | Spaces | Pathways |
Less than 3 kva | 0 in. (0 mm.) | 0 in. (0 mm.) |
3–6 kva | 2 ft. (.6 m.) | 2 ft. (.6 m.) |
6 kva or more | 3 ft. (1 m.) | 3 ft. (1 m.) |
While the characteristics of structured cabling are discussed in detail in Chapter 7, "Designing a Scalable Network Infrastructure," it is important to understand the difference between UTP and STP cabling.
Unshielded twisted pair cabling contains multiple pairs of twisted copper conductors, which are gathered in a single sheath (typically four pairs for data communication). Each wire is surrounded by plastic insulation. Shielded twisted pair cabling is similarly constructed, but also contains a metallic foil or braid around each pair of wires. As the STP cable's name suggests—and as the preceding tables bear out—this metal shields the cable from electromagnetic interference.
Despite the greater interference protection of STP cabling, most Data Centers use UTP cabling because it is less expensive and easier to install—two significant issues because server environments usually involve large quantities of structured cabling. If you are particularly concerned about the nearness of your data cabling and electrical conduits, you may wonder whether to run one form of infrastructure overhead and the other below a raised floor—all data connections above each server cabinet location while all electrical conduits within the subfloor, or vice versa. It is certainly possible to do so, but such arrangements essentially provide the disadvantages of each system—the clutter and difficulty of reaching overhead infrastructure along with the cost and additional height requirements of a raised floor—without any particular advantage. Moreover, it is unnecessary. Merely running electrical conduits and structured data cabling several inches (centimeters) apart provides sufficient separation to prevent signal interference.
For an overhead installation, you can run the two types of infrastructure in separate channels and then terminate them in parallel raceways over server cabinets to permit easy access to both. Keep data cabling not just away from the electrical conduits but also from the Data Center's overhead lights, which are sources of interference as well.
Note - Fluorescent lights in particular are known to generate electromagnetic interference. Because copper cabling is vulnerable to such interference, keep it at least 5 inches (12.7 centimeters) away from fluorescent light fixtures. More information about structured cabling and interference is provided in Chapter 7.
For an under-floor installation, you don't even need the channels, although many companies opt to use some sort of tray system to route infrastructure—just place the electrical conduits and structured cabling so they don't follow identical paths. You can locate data cabling directly below server cabinet locations and place electrical conduits one floor tile space behind, for instance. Orient whatever multimedia boxes and power receptacles they terminate within so that they face toward one another. As with the overhead scenario, data and power are separate from one another but still within reach.
Plenum and Non-Plenum Spaces
In most regions, building codes permit only specially rated cables to be used in the plenum—those pockets of space used for air distribution in your server environment. These are normally the cavities above the false ceiling or below the raised floor. The codes are intended to reduce the chances or effects of a fire, because once a blaze reaches these spaces, there is little to stop its smoke and flames from spreading rapidly. Because plenum space is more vulnerable to fire, stricter standards are applied to its cabling.
While these regulations don't directly affect your decision of whether to run infrastructure above the ceiling or below the floor, be aware of what types of cables you must use when routing them through different Data Center spaces. Three types of cabling are typically used in server environments:
Non-plenum/riser-rated—These cables are used in general purpose cabling installations. They are very flexible and the least expensive of the three types. In a server environment, non-plenum cables are usually enabled for use in the main area of the room—below the ceiling or above the raised floor. Non-plenum cables are often made of polyvinyl chloride, or PVC, and if they catch fire produce toxic smoke.
Plenum—As indicated by their name, these cables are intended for use in air distribution spaces. Plenum cables emit less toxic smoke when burned than non-plenum cables. They are preferred for use in the United States, but prohibited in many European countries.
Low smoke/zero halogen—These are preferred for use in Europe, but restricted for use in the United States. They produce little toxic smoke in a fire.