How the Clear Choice Test of VoIP quality over an SSL VPN was conducted.
We used hardware and software from GL Communications and Shunra Software to test VoIP quality over an SSL VPN.
We started by using GL Communications Voice Quality Testing (VQT) tool suite to evaluate the quality of VoIP calls. On one side of the VoIP call was a Grandstream SIP phone. The GL Communications Universal Telephony Adapter (UTA) hardware connected to the Session Initiation Protocol phone as a handset, intercepting audio flowing from both directions.
On the other side of the VoIP call was a Dell laptop running the open source X-Lite and SJPhone SIP soft-phone software packages. We used a Griffin Technologies’ iMic for a line-level audio in and audio out from the laptop (because the Dell laptop only has mic-level audio inputs) and connected this to the GL Communications UTA.
This setup allowed the GL Communications VQT software to play audio files into the “hard” SIP phone and receive them over the VoIP connection on the soft phone, and vice versa. GL Communications VQT software generates the ITU-specified metrics to evaluate the quality of a voice call. We used the ITU G.711 u-law codec for all calls. The G.711 u-law codec is the most popular in the United States and requires 64Kbps of bandwidth for the data path (plus additional overhead when packed into real-time streaming protocol, User Datagram Protocol and IP. See “Minding your Ps and Qs?”)
We picked a set of six, 8-second audio samples from the GL Communications library and sent them through the VoIP call path in both directions to generate a baseline set of call-quality values. We repeated this test 10 times to validate that our statistics were not straying more than 1% from test to test.
Next, we took Shunra’s Virtual Enterprise network simulator and used it to introduce latency, bandwidth limitations and loss into our VoIP network. We gave the Virtual Enterprise hardware four network profiles to reproduce. The first set was “no latency, no loss” to see how the SSL VPN would behave all on its own compared with a no latency, no loss Ethernet network without SSL VPN.
The next two sets of data were set up to represent typical broadband connections a user would experience traveling from location to location. Both were limited to 0.5Mbps bandwidth. One set represented “good broadband” and was based on 800 separate data sets we collected while traveling during the latter half of 2005 from T-Mobile hot spot and wired broadband facilities around the United States. T-Mobile’s quality is pretty good, and we got a fairly consistent nearly-0% packet loss (even over wireless) with 90 millisec Round Trip latency from the broadband network back to our DS3-connected data center in Tucson. For statistics fans, on T-Mobile’s network we also measured mean round-trip packet-to-packet delay variance, also called jitter, at 20 millisec. When we simulated the “good broadband” network in the Shunra Virtual Enterprise, we used these measured values.
The third set represented “average broadband” and was based on an additional 2,300 data sets we collected at other locations while traveling, including multiple SprintPCS Wi-Fi hot spots along with several Ethernet-connected hotels in the United States, Asia, Europe and the Caribbean. In those cases, we had a higher latency and packet loss, averaging about 2% (wireless and wired) with a 120 millisec round-trip latency back to Tucson. Jitter was also measured at 40 millisec.
Finally, we created a fourth set of data representing what we thought would be the worst-case connection: a loss rate of 2%, 120 millisec round-trip latency, 40 millisec jitter, and a bandwidth cap of 100Kbps.
For each of the SSL VPN devices, we set up the Shunra Virtual Enterprise between the soft-phone client and the SSL VPN device, which was connected via Ethernet to the hard phone and rest of our VoIP network.
Then we ran the GL Communications test (send and receive six voice samples in each direction) 12 times for each of the three settings of the Virtual Enterprise, discarding the lowest and highest scores, and averaging the others to give a single set of numbers for each SSL VPN device, for each configuration.
Network Testing Scenarios for VOIP over SSL VPN test We tested using four scenarios, with the Good and Bad scenarios representing the majority of broadband service that travelers would expect to get worldwide. Home users on good quality ISPs would generally expect networks matching the Good profile as well. | ||||||||
|
