Performance Evaluation with TTCP and iperf

This section will briefly describe how to evaluate the throughput performance of the owl device driver and the HD Wireless Wi-Fi device.

The throughput performance evaluation is performed by sending TCP data between the Wi-Fi device (DUT) and a PC. The DUT and PC should be connected to the same access point. To avoid any limitations introduced by the access point and the PC, the PC should have a wired connection to the access point. For maximum performance, make sure that the selected channel is not used by any other equipment and that no other client is connected to the AP.

Using the TCP protocol to test throughput performance certainly introduces quite some protocol related overhead such as extra headers, retransmissions and acknowledgements. Therefore the actual throughput results will be highly dependent on the platform dynamic responsiveness (e.g. to handle incoming TCP acknowledgements) since TCP packets will be sent in both directions to successfully complete a single direction data transfer. The actual raw data throughput can be considered significantly better; e.g. using the UDP protocol would give a higher throughput of actual payload data.

The throughput measurement setup and execution will be performed in the following steps:

All the steps and expected output is provided in detail below.

Obtain and configure the software tools

In this case, to transmit and receive TCP data, the standard tool TTCP has been chosen. It can be obtained from e.g. http://www.netcore.fi/pekkas/linux/ipv6/ttcp.c.

To compile ttcp for the host platform to which the Wi-Fi device is connected (DUT), use the cross compiler for the platform. The cross compiler used here is the same as the one used to compile the owl device driver. Make sure to replace /path/to/gcc with the proper cross compiler, e.g. armlinux-
gcc if that is the name of the GCC binary.

$ /path/to/gcc ttcp.c –o ttcp

Make sure to copy the ttcp binary to the host platform. ttcp is also needed on the PC. On a Linux PC, the same TTCP source code can used:

$ gcc ttcp.c –o ttcp

If a windows PC is used, pcattcp should be used and can be downloaded from http://www.pcausa.com/Utilities/pcattcp.htm.

During test execution, the TTCP tool will be invoked on both the DUT and PC, one acting as the transmitter and the other one as the receiver. The transmitter will send a fixed (but configurable) amount of data to the receiver using the TCP protocol. The throughput will then be calculated based on the amount of payload data transferred in during the elapsed time.

Establish a link between the DUT and the PC

Make sure that the PC gets an IP address from the access point (or use a static IP) when the network cable is connected. In Windows, this should happen automatically as long as DHCP is enabled in the network configuration for the ethernet port. In Linux the DHCP client might have to be started manually, depending on the distribution and configuration. Now connect the DUT to the same access point.

It should now be possible to ping the PC from the DUT through the access point by using the ping command in the DUT console (make sure that the actual IP address of the PC is replaced in the example below)

$ ping <ip>
...

Test TX throughput

First start ttcp on the PC in receive mode. In windows use the pcattcp tool:

PC> pcattcp -r -s

In Linux, use the ttcp tool:

PC> ttcp -r -s

Then the transmitter should be started on the DUT by using the ttcp command on the DUT:

$ ttcp -t –s –n128000 <ip address of PC>

This will start the transfer of 128 Mb payload data. When the transfer is completed, throughput information will be printed on both the DUT console and the PC console. Se below for example output shown on DUT followed by output on PC.

$ ttcp -t –s –n128000 <ip address of PC>
ttcp-t: nbuf=128000, buflen=1024, port=2000
ttcp-t: socket
ttcp-t: connect
ttcp-t: 0.1user 3.8sys 0:52real 7% 0i+0d 0maxrss 0+2pf 6185+37534csw
ttcp-t: 131072000 bytes processed
ttcp-t: 3.94 CPU sec = 32487.3 KB/cpu sec, 259898 Kbits/cpu sec
ttcp-t: 52.0811 real sec = 2457.71 KB/real sec, 19661.6Kbits/sec

PC> ttcp -r -s
ttcp-r: nbuf=1024, buflen=1024, port=2000
ttcp-r: socket
ttcp-r: accept
ttcp-r: 0.0user 0.5sys 0:52real 1% 0i+0d 392maxrss 0+0pf 91562+27csw
ttcp-r: 131072000 bytes processed
ttcp-r: 0.564035 CPU sec = 226936 KB/cpu sec, 1.81549e+06 Kbits/cpu sec
ttcp-r: 52.0956 real sec = 2457.02 KB/real sec, 19656.2 Kbits/sec

Test RX throughput

First start ttcp on the DUT in receive mode:

$ ttcp –r -s

Then the transmitter should be started on the PC. In windows, use the pcattcp tool:

PC> pcattcp -t -s –n128000 <ip address of DUT>

In linux, use the ttcp tool:

PC> ttcp -t -s –n128000 <ip address of DUT>

This will start the transfer of 128 Mb payload data. When the transfer is completed, throughput information will be printed on both the DUT console and the PC console. Se below for example output shown on DUT followed by output on PC.

$ ttcp –r –s
ttcp-r: nbuf=1024, buflen=1024, port=2000
ttcp-r: socket
ttcp-r: accept
ttcp-r: 0.4user 13.1sys 0:58real 23% 0i+0d 0maxrss 0+1pf 85295+399984csw
ttcp-r: 131072000 bytes processed
ttcp-r: 13.6 CPU sec = 9411.76 KB/cpu sec, 75294.1 Kbits/cpu sec
ttcp-r: 58.4456 real sec = 2190.07 KB/real sec, 17520.6 Kbits/sec

PC> ttcp -t -s –n128000 <ip address of DUT>
ttcp-t: nbuf=128000, buflen=1024, port=2000
ttcp-t: socket
ttcp-t: connect
ttcp-t: 0.0user 0.1sys 0:58real 0% 0i+0d 392maxrss 0+1pf 7118+26csw
ttcp-t: 131072000 bytes processed
ttcp-t: 0.156009 CPU sec = 820465 KB/cpu sec, 6.56372e+06 Kbits/cpu sec
ttcp-t: 58.4284 real sec = 2190.71 KB/real sec, 17525.7 Kbits/sec

Using the iperf tool to measure performance

Instead of using ttcp, the iperf tool can be used to measure performance. Iperf can be used in the same setup, where one end acts as a server (receiving data) and the other end acts as a client (transmitting data).

For instance, to measure the performance when the DUT is sending TCP data to the PC, the following commands below can be used.

First start iperf in server mode on the PC:

PC> iperf -s

Then start the client on the DUT:

$ iperf -c <ip address of PC>

When the transfer is completed, throughput information will be printed on both the DUT console
and the PC console. Se below for example output shown on DUT followed by output on PC.

$ iperf -c 192.168.2.100
------------------------------------------------------------
Client connecting to 192.168.2.100, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.2.132 port 38443 connected with 192.168.2.100 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 22.6 MBytes 18.9 Mbits/sec

PC> iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 5] local 192.168.2.100 port 5001 connected with 192.168.2.132 port 38443
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 22.6 MBytes 18.9 Mbits/sec