How we did it

We tested in Placerville, Calif., and Tullahoma, Tenn., with identical videoconferencing systems from each manufacturer.

We examined all aspects of the systems, including options; connecting cables to appropriate displays and networks; and through various user interfaces, entering identical data in every system pair.

We first tested performance over ISDN. We dialed to a like end point at two data rates: 256K and 384K bit/sec - two and three Basic Rate Interfaces (BRI), respectively. If we ran into problems at either of these data rates, we stepped down to 128K bit/sec (a single BRI).

We then tested each system's ability to negotiate calls with competing manufacturers' systems. We scored each system's success/failure under interoperability, and noted call setup and tear-down behaviors. In this fashion we established H.320 interoperability.

We evaluated the quality of audio and video at 384K bit/sec on ISDN by keeping the call up for at least 10 minutes. We varied the camera field of view to several subjects, including talking heads and high-motion scenes. We noted image clarity, spectral qualities, frame rate, type and frequency of artifacts, audio clarity, lip synchronization and echo. For high-motion scenes we also scored time to recovery from rapid changes in content.

We then simulated a corporate IP LAN/WAN environment, testing performance on shared and switched Ethernet LANs, and introduced routers in the test bed to simulate two WAN hops. We used a traffic generator to simulate different traffic types over the test beds. We also manipulated the router prioritization settings to evaluate their effects on video quality.

We systematically placed calls between systems on the LAN to test cross-vendor H.323 interoperability. To compare the effects of switched and shared networks on videoconferencing systems we populated two test beds with eight identical end points. The first test bed used a shared hub from Webramp. The second used a Nortel Networks BayStack 450 nonblocking switch. We initiated and completed calls and evaluated each product's ability to retrieve a DHCP IP address.

For the LAN/WAN test we put a VCON Escort desktop videoconferencing system and an additional client PC without videoconferencing capabilities on a separate LAN connected to a second BayStack 450 switch. We connected the two at 100M bit/sec via Nortel Passport BLN routers.

We simulated 1M, 2M and 5M bit/sec frame relay connections over the serial interfaces to evaluate the performance. Using Ganymede Software Chariot 3.1, we simulated FTP and UDP streams on the LAN and WAN. At the same time, the videoconferencing end points were placed in calls with the Escort.

For performance testing we assigned each system a unique IP address and turned off DHCP. We evaluated performance of the end points on the IP network test bed using the same criteria established for ISDN-based calls above.