IPERF and TCP window size
The TCP window size can affect the results of an IPERF test and mislead a user on a network's performance capabilities.
Iperf appears to use different TCP window sizes depending on the version and OS of the build. The actual implementation of the TCP window for a given OS is beyond the scope of this article, however, it is possible to give Iperf hints about what window size to use/request. I say 'use/request' because it it is not clear to me how one verifies the tcp window size actually in use.
In the following example, 'blue' has a 5ms round trip time to iperf.wiscnet.net.
[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 8KB ------------------------------------------------------------ Client connecting to iperf.wiscnet.net, TCP port 5001 TCP window size: 8.0 KByte ------------------------------------------------------------ [ 3] local 216.56.13.114 port 37348 connected with 205.213.110.244 port 5001 [ 3] 0.0- 1.0 sec 2.37 MBytes 19.9 Mbits/sec [ 3] 1.0- 2.0 sec 2.30 MBytes 19.3 Mbits/sec [ 3] 2.0- 3.0 sec 2.27 MBytes 19.1 Mbits/sec [ 3] 3.0- 4.0 sec 2.24 MBytes 18.8 Mbits/sec [ 3] 4.0- 5.0 sec 2.31 MBytes 19.4 Mbits/sec [ 3] 5.0- 6.0 sec 2.31 MBytes 19.4 Mbits/sec [ 3] 6.0- 7.0 sec 2.29 MBytes 19.2 Mbits/sec [ 3] 7.0- 8.0 sec 2.26 MBytes 18.9 Mbits/sec [ 3] 8.0- 9.0 sec 2.20 MBytes 18.5 Mbits/sec [ 3] 9.0-10.0 sec 2.23 MBytes 18.7 Mbits/sec [ 3] 0.0-10.0 sec 22.8 MBytes 19.1 Mbits/secHere is the same test with a 64KB window
[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 64KB ------------------------------------------------------------ Client connecting to iperf.wiscnet.net, TCP port 5001 TCP window size: 128 KByte (WARNING: requested 64.0 KByte) ------------------------------------------------------------ [ 3] local 216.56.13.114 port 37353 connected with 205.213.110.244 port 5001 [ 3] 0.0- 1.0 sec 17.8 MBytes 149 Mbits/sec [ 3] 1.0- 2.0 sec 17.8 MBytes 149 Mbits/sec [ 3] 2.0- 3.0 sec 18.1 MBytes 152 Mbits/sec [ 3] 3.0- 4.0 sec 18.3 MBytes 153 Mbits/sec [ 3] 4.0- 5.0 sec 18.5 MBytes 155 Mbits/sec [ 3] 5.0- 6.0 sec 18.5 MBytes 155 Mbits/sec [ 3] 6.0- 7.0 sec 18.3 MBytes 154 Mbits/sec [ 3] 7.0- 8.0 sec 18.6 MBytes 156 Mbits/sec [ 3] 8.0- 9.0 sec 18.2 MBytes 153 Mbits/sec [ 3] 9.0-10.0 sec 18.3 MBytes 154 Mbits/sec [ 3] 0.0-10.0 sec 182 MBytes 153 Mbits/sec8kb * 8 = 64kb. 20 Mbps * 8 = 160 Mbps. This is not a coincidence.
A single iperf flow with a 8KB buffer size is not representative of a production enviornment, which would likely have many active hosts in a single LAN.
If you are trying to optimize TCP throughput for a single flow, increasing packet payload size and TCP windows are your best bets. If you can decrease the round trip time, that is also valuable.
Other considerations:
On 2/10/2015 a commenter chimed in on the following:
> If you modify the window size with the -w switch, you MUST modify the window
> size on both the client and server sides. Interestingly, you do see a significant
> boost in throughput by modifying it on the client side only. This must relate to an
> increased buffer as suggested.
> I have confirmed via Wireshark that the window size remains at the 64KB default
> on Server 2012 if you change the window size to any other value on the client
> side without changing it on the server side. I have then confirmed that changing
> the value on the server side simultaneously results in an increased window size
> as displayed in Wireshark. Between two Hyper-V virtual machines on the same
> server with 10Gb virtual ethernet connections, a window size of 1MB is very
> unstable, rising significantly above and below the specified window size. Perhaps
> an indication of reaching the maximum throughput of the VM's.