The data center from hell, Part 2

Opinion
Sep 25, 20084 mins

* The inside view

Buitron: One circuit breaker was in a garage bay where company trucks parked. Anyone from the street could walk in at any time and throw the switch on the breaker box, cutting off power instantly to all of the company’s servers.

In the previous column, security specialist Jan Buitron, a graduate student in the Master of Science in Information Assurance program at Norwich University, began a report on a horribly insecure facility at which she worked some years ago. Today she goes from the outside to the squishy inside of the house of horrors.

* * *

Facility design (outside the building)

Since the IT department was in operation from 6 a.m. to 7 p.m. every day, the exterior of the building should have been well lit for personnel safety, but it wasn’t. The exterior entrance door to the showroom floor had no floodlight; in the evenings it got very dark. The main entrance door to the IT department had two automatic floodlights pointed at it, but visibility overall was poor. The door was set back in a recessed area on the side of the main building, and visibility of the area was reduced.

The automatic floodlights were supposed to switch on when it got dark. As it turned out, they worked very well in the summer, but in cold weather in winter, there were nights when the light never switched on (or it would switch on after I was leaving the facility). I mentioned the poor lighting and lack of attention to personnel safety several times to my management, but my concerns were never addressed.

Facility design (inside the building)

The core processing for the whole company was housed in a separate area in the same building as manufacturing. One piece of good planning in place was that manufacturing and the data center were on separate circuits with separate power feeds.

The incoming power for manufacturing was in a locked room near the manufacturing area. The incoming T-1 1ine was also in the same room.

The circuit breaker boxes were in two different exposed areas. One was in a garage bay where company trucks parked. Anyone from the street could walk in at any time and throw the switch on the breaker box, cutting off power instantly to all of the company’s servers.

The breaker box for the server room was just inside a main exterior door on the other side of the building. Although the breaker box on the wall was kept locked, anyone could walk into the hallway where the breaker box was and pick the lock on the box. After gaining access to the breaker box, it was easy to flip off the switch in the breaker box and bring down all of the company’s servers at once. The data center was equipped with a motion-detection system armed each night by the last employee to leave. The circuit breaker was not in the area covered by the motion detectors.

The IT department housing the server room and computer support area was isolated from the rest of the facility by a locked door. Only selected individuals got keys to the IT department. One day, when my colleagues were showing me the ropes, they made it a special point to tell me to make sure to take my department key with me when I stepped out of the IT department for a break so I could get back in the locked door.

However, if I forgot the key, I could CLIMB UP AND OVER THE TOP OF THE DOOR AND LIFT THE CEILING TILES TO GET OVER THE DOOR. Then they noted that one colleague had had to climb over several times after hours because he was really bad about forgetting the key.

Interference

Information technology departments should be concerned about radio-frequency interference from sources in the immediate area. In our case, there was a wireless telephone distributor located next door to the data processing facility. The wireless company had a wireless transmission tower in its parking lot. In addition, the company assembled cell transmission boxes within 100 feet of the data center where our company’s core processing took place.

* * *

In the last of this three-part series, Jan reflects on the lessons for all of us from this facilities-security debacle.