WLANs scale, just not easily

Bill Gates decided in 1999 that it was time for Microsoft to get serious about wireless LANs . At a public CEO summit, he announced that the software maker would have a wireless network blanketing its Redmond, Wash., campus within 12 months.

That announcement sent the company’s Operations and Technology Group scrambling.

Today, there are about 2,500 Cisco Aironet access points  at headquarters and another 1,200 worldwide, making Microsoft’s WLAN rollout one of the largest.

But networks of this size are still uncommon, and for good reason: They’re a lot of work.

That’s the verdict from network professionals who’ve built them. If you’re thinking about rolling out a big WLAN, they say, be prepared for a project that will rival, if not surpass, in complexity and detail any LAN you’ve built.

Cut corners and you’ll end up with a WLAN that could have spotty throughput, gaping security holes and a cumbersome end-user experience. It also could be a nightmare to manage.

Planning is key

If not carefully planned, WLANs become disproportionately more complicated when deployed as an enterprise network compared with a department network. 

Procedures and practices that work fine for 10 access points fall apart when the project is 10 times larger. “That just doesn’t scale” is a constant refrain from the network builders interviewed for this story.

Microsoft began to get a grip on scale with its pilot WLAN. During the four-month project that included 600 users in two six-story buildings, the company uncovered several problems, from end-user fears of radio-wave irradiation to potentially huge delays and cost caused by having to run power cables to each access point.

Another problem was network operations and maintenance.

“We found that about [every] 40 access points generated about one service call per day,” says Don Berry, senior network engineer of the Operations and Technology Group at Microsoft. “This was not scalable to what we envisioned.”

“Working out an end-to-end support model before you begin is very important,” he says. That means asking, and answering, questions such as: How will we service the unit, and who will replace it?

Thinking big led the Microsoft engineers to come up with innovative solutions. One was creating, in effect, its own power-over-Ethernet technology, before any commercial implementations were available. That saved $600,000 and shortened the install cycle by eight weeks, Berry says.

But more importantly, it lets network managers switch power on and off remotely, causing a reboot or reset of the access point without having to send a technician to the device.

At the same time, Microsoft linked each access point console port to a terminal server in a cable room. “We now have remote access to all console ports,” Berry says. That means that a local building facilities technician, instead of a network administrator from the data center, can install or replace an access point. “Then, the standard access-point configuration is brought up remotely from our operations center,” he says.

Designing for size

To translate big thoughts into reality, these users say, requires a wireless architecture – a set of design rules that allow flexibility in deployment while ensuring that the result will be manageable, secure and scalable.

Microsoft, to a large degree, created its architecture as it went along, partly through the lessons learned from its pilot project. But the perspective from the beginning was that all the WLAN decisions had to work across thousands of access points.

Cisco, now with 3,000 wireless access points at 300 sites in 100 countries, began with the fixed idea of a single global WLAN deployment.

“We created a single support model, a single client Web site with all our generic wireless LAN information, and standards for security and installation,” says David Castaneda, member of technical staff, with Cisco’s Infrastructure IT group.

Based on tests and pilots, the Cisco design team laid out an architecture that still guides all of the company’s WLAN deployments. The architectural rules included:

• Use only 802.11b 11M bit/sec gear.
• Connect no more than 25 clients connecting to any one access point.
• Authenticate via a global system based on the 802.1x standard for port-based authentication and RADIUS servers.
• Deploy only one virtual WLAN per building, so employees can move anywhere in the building and maintain their session.

One of the most important features of the Cisco architecture is the processes and procedures, embodied in documents and flow charts that communicate these rules throughout a global company and to the systems integrators with whom it partners for deployments outside the U.S. 

“The question we asked is, ‘How do we apply this across hundreds of [Cisco] sites?'” says Oisin Mac Alasdair, technical project manager with Cisco Infrastructure IT.

A global project management team worked closely with “theater” project managers in various large regions, holding weekly conferences. The details of local deployments were left to local offices as long as they followed the blueprint. Local vendors and suppliers followed the same blueprint for equipment and installation. And all of them had to meet a detailed, post-installation testing before the project team signed off on the network.

Now a global operational management team focuses on improving the stability of the WLAN. A group responsible for global network infrastructure will oversee future changes, such as an upgrade to 54M bit/sec 802.11g  radios.

Wireless security

The security problems of WLANs can be summed up simply: It’s like putting an unattended Ethernet jack on the sidewalk outside your office.

In large-scale WLANs, you need multilayered security and a mindset that never takes anything for granted, says Tim Stettheimer, CIO for St. Vincent’s Hospital. The 338-bed hospital has about 170 access points covering its Birmingham, Ala., campus (read a St. Vincent’s case study ). Among other practices at St. Vincent’s are:

• Polling all access points every few minutes to catch configuration changes.

• Registering media access control addresses to control network access.

• Scanning radio waves constantly for intruders or unauthorized access points.

• Confirming wireless users via the existing RADIUS authentication system.

Flexibility has been an invaluable asset for the network professionals’ project at University of Maryland Medical Center in Baltimore. The hospital group has deployed Enterasys Networks access points and network interface cards (NIC), partly to match up with its Enterasys-based wireline network. The initial focus is on “care units” such as the shock-trauma unit, cardiology and new patient-care wings.

“We didn’t think of going outside the care units into the hallways,” says Michael Minear, CIO for the healthcare facility. “And they [doctors and nurses] wanted us to do that.”

Maryland Medical accommodated the unexpected expansion into more open areas, including elevators, in part because of a flexible security framework: wireless VPN with 128-bit encryption keys, firewalls and wireless protocol sniffers; and security software that encrypts downloaded data on PDAs and tablet PCs.

To VPN or not to VPN

One of the most commonly cited wireless security solutions is using a VPN, which typically involves a firewall and a client software application. In effect, you treat your WLAN as if it were the Internet: an untrusted and potentially hostile network.

But two of the biggest sites don’t use VPNs for their WLANs.

At Microsoft, a user powers up the laptop and the wireless NIC associates with an uncontrolled port on an access point. This port admits only RADIUS traffic, which is directed to a RADIUS authentication server. The server connects to a domain controller, which issues the appropriate digital certificates that are part of Microsoft’s public-key infrastructure (PKI). Then, the access point opens the controlled port to give the authenticated client network access.

“PKI is a significant investment,” Berry says. “It’s certainly not for everyone.”

Cisco took another route. The wireless architects concluded that they had to build a global authentication system – from scratch – to replace three separate databases based on Microsoft Windows NT as the domain authentication systems.

The new “triple-A” system (authentication, authorization and auditing) is based on Microsoft Active Directory and 13 Cisco Access Control Servers, which support the so-far ironclad Advanced Encryption System. One systems administrator oversees the system, which serves about 27,000 employees worldwide.

Like Microsoft, Cisco was an early champion and adopter of the IEEE 802.1x authentication standard. Cisco also authored the Lightweight Extensible Access Protocol (LEAP) and co-authored the Protected Extensible Authentication Protocol (PEAP), which is gaining ground as the protocol of choice in the enterprise.

Large-scale VPNs

By contrast, McGill University in Montreal wanted to avoid the complexities of PKI for wireless security. McGill has 200 access points in some of the 120 buildings that form its downtown campus, with about 100 concurrent users. The plan is to deploy 1,000 at all locations. VPNs offered the kind of strong encryption McGill wanted, without the administrative overhead of digital certificates. 

At the same time, it was vital in a network this size that wireless users be able to log on easily, and have the VPN connection just work. For this, McGill turned to Colubris Networks, and its CN3500 Access Controller, coupled with Colubris’ VPN servers and access points. “The CN3500 blocks access to the backbone until the authentication by our existing RADIUS server is done,” says Gary Bernstein, McGill’s director of networks and communications services.

As part of this watchdog function, the controller creates a simple way to load the VPN client application on student laptops. When new students arrive and register, they get their network username and password. When they fire up the Web browser on their wireless laptop for the first time, one of 13 CN3500s intercepts the request for a Web page, and via Secure Sockets Layer displays a link. Clicking on the link triggers a 10-second download of the client code, called NetConnect, which installs almost automatically.

“It’s a very neat way to do a client distribution without actually [physically] distributing clients. It’s a bootstrap operation,” Bernstein says. “Our goal was to minimize [the need for calls to] the help desk.”

As soon as the students authenticate, they enter username and password in a screen form, click, and the controller sets up the encrypted VPN tunnel.

The next stage, says Francois Robitaille, McGill’s manager of network infrastructure, will begin when Colubris blends the VPN software into the controller, and modifies NetConnect so that users will only have to enter their username and password once. Then the controller can launch the VPN session on behalf of users running Windows.

As these accounts make clear, thinking big, and designing big, are critical if you want a WLAN to grow beyond a score or so of access points.

In Part 2, we explore management and operations issues for big WLANS.