SMP boosts scalability for net elements

High-end network elements are starting to have trouble delivering the sheer processing power demanded by the bandwidth explosion.

For instance, in core routers the control-plane CPU must now maintain a routing table of 500,000 or more entries, using compute-intensive protocols such as Open Shortest Path First. On top of that, the CPU must perform administrative and management functions, process SNMP packets, download new routes to each line card and support operator consoles.

Problem is, many control-plane CPUs have hit the wall.

Can newer, faster CPUs address the issue? Not likely, because computational demands are accelerating faster than CPU performance. Consequently, many systems designers are attempting to distribute the workload across multiple CPUs, using symmetric multiprocessing (SMP).

SMP lets multiple CPUs share the same board, memory, I/O and operating system. Nonetheless, each CPU in an SMP system can act independently. While one CPU handles a database lookup, others can update the database and perform other tasks.

SMP costs relatively little. When scaling from one CPU to two, only one processor board is needed. Processing power is doubled without paying for additional support chips and without taking up an additional slot in the chassis.

Using a microkernel architecture for the operating system fits nicely with SMP. Compared with a monolithic operating system kernel, a microkernel is extremely small, because it contains only essential services, such as process scheduling. All other services run as separate, memory-protected programs. As a result, the kernel modifications for SMP are also small: just a few kilobytes of code. Only the microkernel is modified. All other services and applications can gain the performance advantages of SMP without code changes. For instance, a router vendor can develop an application once, then deploy it across a product line that includes single-processor and SMP systems. Scalability is built in.

Availability also gets a boost. Because drivers, protocols and applications all run as memory-protected programs, they can't corrupt the kernel or each other. If any software process misbehaves, it can be automatically restarted while all other services continue to run.

Despite its benefits, SMP can only scale so far. Bottlenecks can occur when several CPUs on a board share a single memory bus. Rather than put too many CPUs on the same SMP board, designers of high-end network elements can distribute applications across a networked cluster of SMP boards, where each board has its own memory array, I/O and operating system.

This approach has its challenges. For instance, network managers have to add network-specific code to applications so they can "talk" to each other across the networked boards. Also, because drivers and protocols in most operating systems are bound to the kernel, moving any of them from one board to another means you have to create and carefully test a new kernel image for each board.

A microkernel operating system sidesteps these problems in two ways. First, because applications, protocols and drivers are all decoupled from the operating system kernel, they can be moved freely from one CPU board to another; without the need for kernel reconfiguration. Second, applications in a microkernel operating system typically communicate via message passing, which eliminates the need for network-specific code.

The bottom line is that SMP hardware and microkernel operating system architecture allow vendors to add processing power - and scalability - to their network elements without adding software development or compromising system density.

Leroux is a technology analyst at QNX Software Systems. He can be reached at paull@qnx.com.