####1. Description of setup#1 ---> 2520-debug_user
userspace: qemu-kvm-0.12.1.2-2.355.el6.x86_64
host kernel: 2.6.32-358.el6.x86_64
guest kernel: 62.64.104.5700
####2. Description of setup#2 ---> 2522-debug_user
userspace: qemu-kvm-0.12.1.2-2.355.el6.x86_64
host kernel: 2.6.32-358.el6.x86_64
guest kernel: 62.64.104.5700
The tests are * 60* seconds sessions of 'Netperf'. 'throughput' was taken from netperf's report.
Other measurements were taken on the host.
How to read the results:
- The Throughput is measured in Mbit/sec.
- Every Avg line represents the average value based on *5* repetitions of the same test,
and the following SD line represents the Standard Deviation between the *5* repetitions.
- The Standard deviation is displayed as a percentage of the average.
- The significance of the differences between the two averages is calculated using unpaired T-test that
takes into account the SD of the averages.
- The paired t-test is computed for the averages of same category.
== Regression Testing: Win2012.64.netperf_win.netperf_exe.best_registry_setting ==
Category:TCP_STREAM (RX) |
| # | Tile | size | sessions | throughput | CPU | thr_per_CPU |
| 1 | Avg | 64 | 1 | 1042.11 | 26.00 | 40.08 |
| | %SD | 1% | 2% | 3% |
| 2 | Avg | 1055.19 | 20.16 | 52.34 |
| | %SD | 2% | 12% | 13% |
| - | %Diff between Avg | +1.255% | -22.462% | +30.589% |
| - | Significance | +0.639 | -0.999 | +0.997 |
| 1 | Avg | 64 | 2 | 2323.99 | 27.26 | 85.25 |
| | %SD | 8% | 12% | 19% |
| 2 | Avg | 2028.23 | 28.61 | 70.89 |
| | %SD | 2% | 4% | 6% |
| - | %Diff between Avg | -12.726% | +4.952% | -16.845% |
| - | Significance | -0.992 | +0.582 | -0.932 |
| 1 | Avg | 64 | 4 | 2754.12 | 25.32 | 108.77 |
| | %SD | 4% | 5% | 2% |
| 2 | Avg | 4275.67 | 31.85 | 134.24 |
| | %SD | 3% | 4% | 6% |
| - | %Diff between Avg | +55.246% | +25.790% | +23.416% |
| - | Significance | +1.000 | +1.000 | +1.000 |
| 1 | Avg | 64 | 8 | 2735.46 | 26.19 | 104.45 |
| | %SD | 4% | 2% | 3% |
| 2 | Avg | 4563.32 | 31.95 | 142.83 |
| | %SD | 5% | 4% | 8% |
| - | %Diff between Avg | +66.821% | +21.993% | +36.745% |
| - | Significance | +1.000 | +1.000 | +1.000 |
| 1 | Avg | 256 | 1 | 2379.95 | 24.88 | 95.66 |
| | %SD | 9% | 14% | 6% |
| 2 | Avg | 4138.38 | 14.49 | 285.60 |
| | %SD | 1% | 18% | 20% |
| - | %Diff between Avg | +73.885% | -41.760% | +198.557% |
| - | Significance | +1.000 | -0.999 | +1.000 |
| 1 | Avg | 256 | 2 | 2899.72 | 27.47 | 105.56 |
| | %SD | 8% | 9% | 15% |
| 2 | Avg | 7306.39 | 27.80 | 262.82 |
| | %SD | 6% | 13% | 11% |
| - | %Diff between Avg | +151.969% | +1.201% | +148.977% |
| - | Significance | +1.000 | +0.128 | +1.000 |
| 1 | Avg | 256 | 4 | 2704.37 | 26.60 | 101.67 |
| | %SD | 10% | 10% | 17% |
| 2 | Avg | 8925.65 | 31.89 | 279.89 |
| | %SD | 4% | 4% | 5% |
| - | %Diff between Avg | +230.045% | +19.887% | +175.293% |
| - | Significance | +1.000 | +0.995 | +1.000 |
| 1 | Avg | 256 | 8 | 2639.93 | 29.76 | 88.71 |
| | %SD | 6% | 10% | 17% |
| 2 | Avg | 9272.87 | 30.41 | 304.93 |
| | %SD | 2% | 5% | 5% |
| - | %Diff between Avg | +251.254% | +2.184% | +243.738% |
| - | Significance | +1.000 | +0.317 | +1.000 |
| 1 | Avg | 1024 | 1 | 2617.37 | 28.04 | 93.34 |
| | %SD | 3% | 0.69% | 3% |
| 2 | Avg | 4030.55 | 17.85 | 225.80 |
| | %SD | 3% | 21% | 24% |
| - | %Diff between Avg | +53.992% | -36.341% | +141.911% |
| - | Significance | +1.000 | -1.000 | +1.000 |
| 1 | Avg | 1024 | 2 | 2750.56 | 27.10 | 101.50 |
| | %SD | 5% | 9% | 10% |
| 2 | Avg | 6485.01 | 27.00 | 240.19 |
| | %SD | 9% | 8% | 10% |
| - | %Diff between Avg | +135.771% | -0.369% | +136.640% |
| - | Significance | +1.000 | -0.049 | +1.000 |
| 1 | Avg | 1024 | 4 | 2744.78 | 26.33 | 104.25 |
| | %SD | 3% | 7% | 9% |
| 2 | Avg | 8855.04 | 32.31 | 274.06 |
| | %SD | 4% | 6% | 6% |
| - | %Diff between Avg | +222.614% | +22.712% | +162.887% |
| - | Significance | +1.000 | +0.998 | +1.000 |
| 1 | Avg | 1024 | 8 | 2625.22 | 27.28 | 96.23 |
| | %SD | 10% | 5% | 14% |
| 2 | Avg | 9574.42 | 30.92 | 309.65 |
| | %SD | 4% | 9% | 9% |
| - | %Diff between Avg | +264.709% | +13.343% | +221.781% |
| - | Significance | +1.000 | +0.970 | +1.000 |
| 1 | Avg | 4096 | 1 | 2386.97 | 25.43 | 93.86 |
| | %SD | 10% | 16% | 7% |
| 2 | Avg | 4484.65 | 17.41 | 257.59 |
| | %SD | 16% | 12% | 7% |
| - | %Diff between Avg | +87.880% | -31.538% | +174.441% |
| - | Significance | +1.000 | -0.996 | +1.000 |
| 1 | Avg | 4096 | 2 | 2848.11 | 25.96 | 109.71 |
| | %SD | 6% | 9% | 7% |
| 2 | Avg | 6931.08 | 28.63 | 242.09 |
| | %SD | 6% | 14% | 13% |
| - | %Diff between Avg | +143.357% | +10.285% | +120.664% |
| - | Significance | +1.000 | +0.779 | +1.000 |
| 1 | Avg | 4096 | 4 | 2804.41 | 25.74 | 108.95 |
| | %SD | 5% | 6% | 7% |
| 2 | Avg | 8742.29 | 31.37 | 278.68 |
| | %SD | 5% | 8% | 8% |
| - | %Diff between Avg | +211.734% | +21.873% | +155.787% |
| - | Significance | +1.000 | +0.997 | +1.000 |
| 1 | Avg | 4096 | 8 | 2538.49 | 27.61 | 91.94 |
| | %SD | 5% | 4% | 8% |
| 2 | Avg | 9189.93 | 29.57 | 310.79 |
| | %SD | 3% | 3% | 1% |
| - | %Diff between Avg | +262.023% | +7.099% | +238.036% |
| - | Significance | +1.000 | +0.984 | +1.000 |
| 1 | Avg | 16384 | 1 | 2551.17 | 26.99 | 94.52 |
| | %SD | 5% | 4% | 7% |
| 2 | Avg | 4150.25 | 16.38 | 253.37 |
| | %SD | 1% | 12% | 12% |
| - | %Diff between Avg | +62.680% | -39.311% | +168.060% |
| - | Significance | +1.000 | -1.000 | +1.000 |
| 1 | Avg | 16384 | 2 | 2678.05 | 24.59 | 108.91 |
| | %SD | 8% | 10% | 5% |
| 2 | Avg | 6978.34 | 28.41 | 245.63 |
| | %SD | 9% | 9% | 8% |
| - | %Diff between Avg | +160.575% | +15.535% | +125.535% |
| - | Significance | +1.000 | +0.957 | +1.000 |
| 1 | Avg | 16384 | 4 | 2757.87 | 26.25 | 105.06 |
| | %SD | 12% | 5% | 11% |
| 2 | Avg | 8686.12 | 32.07 | 270.85 |
| | %SD | 3% | 4% | 7% |
| - | %Diff between Avg | +214.958% | +22.171% | +157.805% |
| - | Significance | +1.000 | +1.000 | +1.000 |
| 1 | Avg | 16384 | 8 | 2582.94 | 26.91 | 95.98 |
| | %SD | 8% | 3% | 10% |
| 2 | Avg | 9305.11 | 31.17 | 298.53 |
| | %SD | 2% | 5% | 5% |
| - | %Diff between Avg | +260.253% | +15.831% | +211.034% |
| - | Significance | +1.000 | +1.000 | +1.000 |
| 1 | Avg | 65535 | 1 | 2614.05 | 27.19 | 96.14 |
| | %SD | 6% | 3% | 7% |
| 2 | Avg | 4082.23 | 17.07 | 239.15 |
| | %SD | 2% | 20% | 18% |
| - | %Diff between Avg | +56.165% | -37.220% | +148.752% |
| - | Significance | +1.000 | -1.000 | +1.000 |
| 1 | Avg | 65535 | 2 | 2809.00 | 26.05 | 107.83 |
| | %SD | 8% | 8% | 4% |
| 2 | Avg | 6839.74 | 28.47 | 240.24 |
| | %SD | 8% | 7% | 7% |
| - | %Diff between Avg | +143.494% | +9.290% | +122.795% |
| - | Significance | +1.000 | +0.909 | +1.000 |
| 1 | Avg | 65535 | 4 | 2803.29 | 25.34 | 110.63 |
| | %SD | 7% | 6% | 3% |
| 2 | Avg | 8753.70 | 31.52 | 277.72 |
| | %SD | 5% | 7% | 5% |
| - | %Diff between Avg | +212.265% | +24.388% | +151.035% |
| - | Significance | +1.000 | +0.999 | +1.000 |
| 1 | Avg | 65535 | 8 | 2665.02 | 26.94 | 98.92 |
| | %SD | 8% | 3% | 10% |
| 2 | Avg | 9338.49 | 29.98 | 311.49 |
| | %SD | 4% | 5% | 6% |
| - | %Diff between Avg | +250.410% | +11.284% | +214.891% |
| - | Significance | +1.000 | +0.994 | +1.000 |
| - | Total Significance | | | +1.000 | +0.263 | +1.000 |