Bandwidth test between two computers, with QoS parameter set. Gigabit cards are connected with a crossover cable. DSCP values are assigned separately for client and server traffic.

To test available bandwidth on the circuit we need two computers, running IPERF for Windows on both ends. One computer acts a a server, listening for incoming packets; another one works as a client, connecting to the server and generating test traffic.

To measure maximum bandwidth, it's better to use UDP instead of TCP, to avoid TCP's intelligence with windowing, back off, and retransmissions.  IPERF UDP client just blasts the maximum traffic you order, and the circuit either carries or drops it. IPERF UDP server reports the total bandwidth of the traffic received, lost packets, and jitter.

It's better to begin testing for maximum throughput from very low target bandwidth (-b key), to ensure you get zero packet loss. Then you gradually increase target bandwidth, watching packet loss and CPU usage. On powerful computers with high quality gigabit network cards, ~900Mbits/sec throughput is possible!

There is a -w (window) parameter, which is in reality just the buffer size. Increasing it does not increase the amount of generated traffic. What actually increases it is the -l (packet length/size) parameter, which can be up to 64kbytes (the maximum size of a UDP packet.) In reality, numbers higher than 65500 just hang the program. Of course, 64k packets don't get transmitted on your network directly; if it's ethernet, they'll be fragmented to 1470 bytes.

To ensure that the circuit doesn't override the DSCP value you assigned, run a packet sniffer like Wireshark. It seriously affects performance, so you should first check that DSCP values are okay under light load, and then run full-load tests.

Start IPERF in server mode, say on IP address 10.0.0.1:

iperf.exe -s -u -p 5123 -i 5 -l 65500 -q 48 where:

-s	means server mode
-u	use UDP
-p 5123	listen on port 5123 instead of default 5001 (any large port number can be used, if it's open on firewalls)
-i 5	print a report every 5 seconds (more frequent ones aren't very representative)
-l 65500	use close to maximum UDP packet size (65536) to try to maximize throughput
-q 48	assign DSCP = 48 to server-to-client packets

IPERF server starts and waits for incoming traffic.

Then start IPERF in client mode:

iperf.exe -c 10.0.0.1 -u -p 5123 -b 5000m -i 5 -t 20 -l 65500 -q 46 where:

-c 10.0.0.1	means client mode, connecting to server 10.0.0.1
-u	use UDP
-p 5123	transmit to port 5123 instead of default 5001, it should match server port above
-b 5000m	target bandwidth to attempt is 5Gbit/sec (start low, say 100m, then increase until you get packet loss)
-i 5	print a report every 5 sec (match server setting, to avoid confusion)
-t 20	run for 20 seconds (with reports every 5 seconds)
-l 65500	use close to maximum UDP packet size (65536) to try to maximize throughput
-q 46	assign DSCP = 46 to client-to-server packets

and watch IPERF output together with Windows Task Manager's "Network" tab. The bandwidth used should match. By default, Task Manager summarizes sent and received traffic together, but can be configured to show them separately.

When your circuit is close to saturation, it is possible for some communication between server and client to get lost. When this happens, sometimes no final server report gets printed on client. That's normal behaviour, and is difficult to avoid since inter-IPERF communication occurs over data path being stress tested. One way around lost reports, is to run a Windows Terminal Session to the remote computer. It persists through heavy transient network loads, and will show you IPERF results on the remote machine. (That's how the movies on the home page were shot.)