IPERF and TCP window size

IPERF and TCP window size


The TCP window size can affect the results of an IPERF test and mislead a user on a network's performance capabilities.

Iperf appears to use different TCP window sizes depending on the version and OS of the build. The actual implementation of the TCP window for a given OS is beyond the scope of this article, however, it is possible to give Iperf hints about what window size to use/request. I say 'use/request' because it it is not clear to me how one verifies the tcp window size actually in use.

In the following example, 'blue' has a 5ms round trip time to iperf.wiscnet.net.

[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 8KB
------------------------------------------------------------
Client connecting to iperf.wiscnet.net, TCP port 5001
TCP window size: 8.0 KByte
------------------------------------------------------------
[  3] local 216.56.13.114 port 37348 connected with 205.213.110.244 port 5001
[  3]  0.0- 1.0 sec  2.37 MBytes  19.9 Mbits/sec
[  3]  1.0- 2.0 sec  2.30 MBytes  19.3 Mbits/sec
[  3]  2.0- 3.0 sec  2.27 MBytes  19.1 Mbits/sec
[  3]  3.0- 4.0 sec  2.24 MBytes  18.8 Mbits/sec
[  3]  4.0- 5.0 sec  2.31 MBytes  19.4 Mbits/sec
[  3]  5.0- 6.0 sec  2.31 MBytes  19.4 Mbits/sec
[  3]  6.0- 7.0 sec  2.29 MBytes  19.2 Mbits/sec
[  3]  7.0- 8.0 sec  2.26 MBytes  18.9 Mbits/sec
[  3]  8.0- 9.0 sec  2.20 MBytes  18.5 Mbits/sec
[  3]  9.0-10.0 sec  2.23 MBytes  18.7 Mbits/sec
[  3]  0.0-10.0 sec  22.8 MBytes  19.1 Mbits/sec
Here is the same test with a 64KB window
[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 64KB
------------------------------------------------------------
Client connecting to iperf.wiscnet.net, TCP port 5001
TCP window size:   128 KByte (WARNING: requested 64.0 KByte)
------------------------------------------------------------
[  3] local 216.56.13.114 port 37353 connected with 205.213.110.244 port 5001
[  3]  0.0- 1.0 sec  17.8 MBytes    149 Mbits/sec
[  3]  1.0- 2.0 sec  17.8 MBytes    149 Mbits/sec
[  3]  2.0- 3.0 sec  18.1 MBytes    152 Mbits/sec
[  3]  3.0- 4.0 sec  18.3 MBytes    153 Mbits/sec
[  3]  4.0- 5.0 sec  18.5 MBytes    155 Mbits/sec
[  3]  5.0- 6.0 sec  18.5 MBytes    155 Mbits/sec
[  3]  6.0- 7.0 sec  18.3 MBytes    154 Mbits/sec
[  3]  7.0- 8.0 sec  18.6 MBytes    156 Mbits/sec
[  3]  8.0- 9.0 sec  18.2 MBytes    153 Mbits/sec
[  3]  9.0-10.0 sec  18.3 MBytes    154 Mbits/sec
[  3]  0.0-10.0 sec    182 MBytes    153 Mbits/sec
8kb * 8 = 64kb. 20 Mbps * 8 = 160 Mbps. This is not a coincidence.

A single iperf flow with a 8KB buffer size is not representative of a production enviornment, which would likely have many active hosts in a single LAN.

If you are trying to optimize TCP throughput for a single flow, increasing packet payload size and TCP windows are your best bets. If you can decrease the round trip time, that is also valuable.


Other considerations:

On 4/18/2013 a commenter chimed in with the following:
 
> I have done testing with iperf and although the behavior changes with 
> the -w flag, i.e. I saw an improvement in bandwidth utilization, packet 
> captures show that the hosts are using TCP windowing well outside of the 
> requested size. I suspect the -w flag may be controlling an internal 
> buffer on the application, but I saw no changes to the TCP windowing. 
> Testing was conducted on Windows 7 professional and Red Hat EL Server 5.9.

On 2/10/2015 a commenter chimed in on the following:

> If you modify the window size with the -w switch, you MUST modify the window
> size on both the client and server sides. Interestingly, you do see a significant
> boost in throughput by modifying it on the client side only. This must relate to an
> increased buffer as suggested.

> I have confirmed via Wireshark that the window size remains at the 64KB default
> on Server 2012 if you change the window size to any other value on the client
> side without changing it on the server side. I have then confirmed that changing
> the value on the server side simultaneously results in an increased window size
> as displayed in Wireshark. Between two Hyper-V virtual machines on the same
> server with 10Gb virtual ethernet connections, a window size of 1MB is very
> unstable, rising significantly above and below the specified window size. Perhaps
> an indication of reaching the maximum throughput of the VM's.



Keywords:IPERF TCP window size bandwidth throughput   Doc ID:41705
Owner:Michael H.Group:University of Wisconsin System Network
Created:2014-07-08 11:29 CDTUpdated:2017-04-13 09:36 CDT
Sites:University of Wisconsin System Network
Feedback:  8   0