WLAN managers see cavalry coming

Faced with deploying up to 1,000 wireless access points across 160 buildings on two separate campuses, McGill University network chief Gary Bernstein immediately recognized the management challenge.

Faced with deploying up to 1,000 wireless access points across 160 buildings on two separate campuses, McGill University network chief Gary Bernstein immediately recognized the management challenge:

“With a thousand access points, you can’t use sneaker power to manage [them],” says Bernstein, whose Montreal educational institution is among the trailblazers rolling out large-scale wireless LANs (WLAN) (see part 1 of our series).

Many of these pioneers by necessity have built their own management tools and figured out management techniques by trial and error. However, today they can take advantage of a growing number of third-party management products. They also are starting to evaluate a new approach to large-scale WLAN deployment: the so-called wireless switch, the first of which are starting to ship.

But none of these tools can or will replace the need to first think big about WLAN management. Experts say to control a big WLAN and minimize operational problems, you have to think through the issues from top to bottom.

Enterprise WLANs are almost invisible to a traditional network management system, even as they add hundreds or thousands of end devices that need to be managed. As Bernstein notes, hiring a platoon of support technicians to hike around a sprawling deployment and fiddle with access points is not practical.

Nearly all of the biggest WLAN sites are using a blend of homegrown tools and third-party applications.

Many WLAN hardware vendors create SNMP management information bases (MIB) on their access points. MIBs are chunks of code that use SNMP to pass data about the device’s behavior and health to network management applications, where the data can be analyzed. MIBs written by vendors can pass back sketchy information about highly detailed data. And getting to the data, getting it out and getting it stored is a pain.

“Today, I have to go out and poll 560 access points,” says Brad Noblet, director of technical services at Dartmouth College in Hanover, N.H. “That’s time-consuming and bandwidth-consuming.”

Cisco’s internal IT group, overseeing about 3,000 access points in the company’s global wireless network, has used some Cisco management tools, but they rely mainly on a set of applications they wrote themselves.

“I will use those tools if they offer me something [I need],” says David Castaneda, member of the technical staff with Cisco’s Infrastructure IT group. “If they don’t, I will build what I need.”

What they built was their own wireless network provisioning tools, which typically run at night under the direction of Cisco-written scripts. Triggered by the scripts, the programs update the software on every access point in the network. That update is simplified because Cisco decided that the exact same software load, or image, would run on each device. The payoff is a network that almost seems to run itself, according to Castaneda.

“Our wireless LAN is very non-labor intensive,” he says. “We wanted an ‘install-and-forget’ scenario, and that is what we built.”

Many routine chores on distributed access points still have to be done one at a time. Network managers have turned to do-it-yourself automation to make this feasible for networks such as the one emerging at McGill. One example is changing the service set identifier (SSID)on each access point. The SSID is attached to wireless packets and acts as a kind of password to join a specific WLAN.

“If you want to change the SSID on all your access points, typically you still have to do this manually,” says Pascal Beauregard, project manager for McGill’s WLAN.

McGill created a set of Perl scripts that runs nightly to apply changes to the SSID and to collect device data using SNMP.

Designing for simplified management was a key element in Microsoft’s installation of a huge WLAN at its Redmond, Wash., campus. From the outset, the company’s internal IT group made sure it had remote control of the console port on each of the 2,500 access points deployed there (See Part 1). Then, operations staff built three databases with information on device addresses, radio channel assignments, locations and settings, and a bundle of scripts.

Installing an access point is now so simple it’s handled by a building’s facilities engineering staff, instead of the IT group. After the device is installed, a network administrator clicks on a script, which pulls out the needed data and configures the device.

“The script brings all this together and configures one access point or a whole subnet of access points with one button-click,” says Don Berry, senior network engineer with Microsoft’s Operations and Technology Group.

A growing number of third-party applications are taking a similar approach.

St. Vincent’s Hospital in Birmingham, Ala., manages 170 access points with Mobile Manager from WaveLink Wireless and Cisco WLAN management utilities. Until recently, such tools typically have focused on remotely managing individual access points.

By contrast, WaveLink lets St. Vincent’s corral access points into groups based on criteria such as location, business department or function. Users assigned to a department, such as the outpatient clinics, inherit the access rights of that department. Network managers also can send software upgrades or configuration changes by group, instead of individually.

Crowded market

Traffic and signal monitors from companies such as AirMagnet, Sniffer Technologies and WildPackets are used for sweeping radio channels to identify unauthorized access points and determine signal strength

Wireless security gateway companies, such as Bluesocket, Colubris, Fortress Technologies, ReefEdge and Vernier Networks, are adding a growing array of device management features to extend centralized control over access points.

WLAN switch start-ups, such as Airespace, Aruba Wireless Networks and Vivato, are have either begun shipping products or are about to. These boxes have Layer 2 and Layer 3 switch features to aggregate access points into manageable groups. There are companion access points, and software tools for monitoring radio signals and sometimes automatically adjust the signals.

Traditional network management applications, such as HP OpenView and Computer Associates’ Unicenter, have new features, often as options, designed for managing wireless network devices and radio signals.

Customers can simplify managing large-scale WLANs by making smart design and architectural decisions at the outset, experts say. A common practice is to mandate a single software bundle, called the system image, for each access point. All access points come with software. 

The single-image mandate is designed to make sure each device has the same software version, and the same standard set of configurations and settings. This makes access points easier to troubleshoot and, if necessary, replace. Remote-control capabilities and power over Ethernet can make on-site visits by support staff rare. A self-service wireless Web site can let users handle a range of questions and minor problems on their own.

Microsoft’s help desk handles all client-related issues, such as new setup and public-key infrastructure problems. Problems that are infrastructure-related or affect a group of users are passed to the Global Network Operations Center. Microsoft reports there are about 700 help desk calls each month from wireless users.

The top two issues for the network operations center staff are “hung” or unresponsive access points, requiring a reboot or power off/on cycle. The replacement rate for the 3,700-odd access points so far works out to about one device every other week, or about 25 per year.

The wireless team at McGill says client-related issues are their main operational thorn. McGill had decided to use a VPN to encrypt wireless traffic, without the overhead of digital certificates and a private-key infrastructure. But VPNs require client code running on a laptop or PDA. McGill’s wireless team found that the way the Windows operating system and the VPN interact created problems in the early stages. Installing and configuring wireless network interface cards, especially when several client operating systems have to be supported, is another source of problems.

Managing the spectrum

One of the most difficult challenges is managing the radio spectrum, because most network management tools assume the connection medium is a wire, not a radio transmission. The result is somewhat analogous to trying to adjust a satellite TV dish by yourself.

“We’d get reports [from users] that radio coverage wasn’t great,” Noblet says, recalling the early stage of Dartmouth’s wireless rollout. “We’d go out [to the access points], jigger things around and ask users, ‘Is it working now?'”

Today, Dartmouth’s network staff makes use of radio monitoring software from AirMagnet, and freeware applications such as NetStumbler, to get a more precise picture of coverage patterns and throughput. Other signal and traffic management tools include applications such as Wireless Valley Communications’ LANPlanner, for predicting radio coverage patterns; and from Newbury Networks’ WiFi Watchdog, for detecting, monitoring and pinpointing all 802.11 devices.

Radio management for big WLANs today really begins at the design stage, where you can address throughput requirements, channel assignments for each access points and interference. A good design can minimize radio management problems, such as those caused by channel interference.

Both 802.11b and 802.11g networks run in the 2.4-GHz radio band, which has only three non-overlapping channels. Each access point is assigned one of these channels, and clients use that channel to connect to a given access point. Those three channels limit how many 2.4-GHz devices can be located close together.

“This is a [network] design problem,” says Gary Braver, principal consultant with FastLane Networks, a WLAN integrator. “If you have 50 users on one [802.11b] access point, you end up giving them about 100K bit/sec of sustained throughput per user.”

Microsoft’s IT group struggled with this initially to find a design that balanced the number of access points and performance for the buildings on the campus.

Another complicating factor is that the WLAN radio picture is changing constantly. Cisco’s Infrastructure IP group has a process of continually scanning the airwaves at all WLAN sites and scanning the nets to match all media access control addresses that show up with a database of official MAC addresses. This scanning unmasks unauthorized access points, but it also gives network operators a picture of the radio architecture.

“Because your radio architecture works one month, there’s no guarantee it will work the next month,” says Oisin Mac Alasdair, technical project manager with Cisco Infrastructure IT.