Why the Speed flex result is capping to 1 Gbps when testing between the AP and the SZ\vSZ?

Summary

The Speed flex result is capping to 1 Gbps when testing between the AP and the SZ\vSZ.

Question

Why the Speed flex result is capping to 1 Gbps when testing between the AP and the SZ\vSZ?

Customer Environment

• SZ124 <<<<>>>> Switch1 <<<<>>> Switch 2<<<<>>>> R730 and R850 • SZ124 connected via Fibre port negotiating at 10 Gbps lan speed. • R730 and R850 connected to 2.5 Gbps eth port. Negotiating at 2.5 Gbps speed. • Switch Model : ICX7150-48ZP • Controller version : 6.1.0.0.935 • AP Version : 6.1.0.0.1595 • Fibre model : Type : 10GE SR 300m (SFP+) Vendor: RUCKUS Version: A Part# : 57-0000075-01 Serial#: AAF211080000WKD Note: The issue has been tested on 5.2.x, 6.0 and 6.1

Symptoms

When I do the Speed Flex test from the GUI between the AP and controller, the speed result is always less than 1 Gbps. And with higher packet drops.

Troubleshooting Steps

  1. When I do the Speed Flex test from the GUI between the AP and controller, the speed result is always less than 1 Gbps. And with higher packet drops.
User-added image
2. Same has been tested with ZAP and iPerf test as well.

3. Below is the iPerf output from the AP to controller, the result is capping to 1 Gbps. 
From AP:

# ./iperf -c 10.177.85.57  -i 1 -u -p 18301 -b 300M -P 4
------------------------------------------------------------
Client connecting to 10.177.85.57, UDP port 18301
Sending 1470 byte datagrams, IPG target: 37.38 us (kalman adjust)
UDP buffer size:  220 KByte (default)
------------------------------------------------------------
[  5] local 10.177.85.23 port 33674 connected with 10.177.85.57 port 18301
[  6] local 10.177.85.23 port 49220 connected with 10.177.85.57 port 18301
[  4] local 10.177.85.23 port 43796 connected with 10.177.85.57 port 18301
[  7] local 10.177.85.23 port 50016 connected with 10.177.85.57 port 18301
[ ID] Interval       Transfer     Bandwidth
[SUM]  0.0- 1.0 sec   150 MBytes  1.26 Gbits/sec

<>
[SUM]  1.0- 2.0 sec   150 MBytes  1.26 Gbits/sec

[SUM]  2.0- 3.0 sec   150 MBytes  1.26 Gbits/sec
….
[SUM]  3.0- 4.0 sec   150 MBytes  1.26 Gbits/sec
[SUM]  4.0- 5.0 sec   148 MBytes  1.24 Gbits/sec
[SUM]  5.0- 6.0 sec   152 MBytes  1.28 Gbits/sec
[SUM]  6.0- 7.0 sec   150 MBytes  1.26 Gbits/sec
[SUM]  7.0- 8.0 sec   150 MBytes  1.26 Gbits/sec
[  5]  8.0- 9.0 sec  37.5 MBytes   315 Mbits/sec
[  6]  8.0- 9.0 sec  37.5 MBytes   315 Mbits/sec
[  4]  8.0- 9.0 sec  37.5 MBytes   315 Mbits/sec
[  7]  8.0- 9.0 sec  37.5 MBytes   315 Mbits/sec
[SUM]  8.0- 9.0 sec   150 MBytes  1.26 Gbits/sec
[  6]  0.0-10.0 sec   373 MBytes   313 Mbits/sec
[  6] Sent 266247 datagrams
[  7]  0.0-10.0 sec   374 MBytes   313 Mbits/sec
[  7] Sent 266482 datagrams
[  5]  0.0-10.0 sec   373 MBytes   313 Mbits/sec
[  5] Sent 266234 datagrams
[  4]  0.0-10.0 sec   373 MBytes   313 Mbits/sec
[  4] Sent 266338 datagrams
[SUM]  0.0-10.0 sec  1.46 GBytes  1.25 Gbits/sec
#


Corresponding SZ iperf result:

[  6] local 10.177.85.57 port 18301 connected with 10.177.85.23 port 33674
[  3] local 10.177.85.57 port 18301 connected with 10.177.85.23 port 43796
[  4] local 10.177.85.57 port 18301 connected with 10.177.85.23 port 49220
[  5] local 10.177.85.57 port 18301 connected with 10.177.85.23 port 50016
[  6]  0.0- 1.0 sec  31.2 MBytes   261 Mbits/sec   0.035 ms 4525/26758 (17%)
[  6] 0.00-1.00 sec  2 datagrams received out-of-order
[  3]  0.0- 1.0 sec  30.1 MBytes   253 Mbits/sec   0.012 ms 5267/26744 (20%)
[  4]  0.0- 1.0 sec  31.5 MBytes   265 Mbits/sec   0.041 ms 4255/26755 (16%)
[  5]  0.0- 1.0 sec  29.7 MBytes   249 Mbits/sec   0.041 ms 5557/26726 (21%)
[SUM]  0.0- 1.0 sec   122 MBytes  1.03 Gbits/sec   0.041 ms 19604/106983 (18%)
[SUM] 0.00-1.00 sec  2 datagrams received out-of-order
[  6]  1.0- 2.0 sec  28.6 MBytes   240 Mbits/sec   0.026 ms 6330/26754 (24%)
[  3]  1.0- 2.0 sec  29.1 MBytes   245 Mbits/sec   0.029 ms 5973/26764 (22%)
[  4]  1.0- 2.0 sec  28.7 MBytes   241 Mbits/sec   0.020 ms 6286/26742 (24%)
[  5]  1.0- 2.0 sec  28.6 MBytes   240 Mbits/sec   0.034 ms 6354/26780 (24%)
[SUM]  1.0- 2.0 sec   115 MBytes   965 Mbits/sec   0.034 ms 24943/107040 (23%)
[  6]  2.0- 3.0 sec  30.0 MBytes   252 Mbits/sec   0.025 ms 5352/26742 (20%)
[  3]  2.0- 3.0 sec  28.3 MBytes   237 Mbits/sec   0.021 ms 6572/26737 (25%)
[  4]  2.0- 3.0 sec  29.7 MBytes   249 Mbits/sec   0.015 ms 5566/26749 (21%)
[  5]  2.0- 3.0 sec  27.9 MBytes   234 Mbits/sec   0.041 ms 6854/26750 (26%)
[SUM]  2.0- 3.0 sec   116 MBytes   972 Mbits/sec   0.041 ms 24344/106978 (23%)
[  6]  3.0- 4.0 sec  28.5 MBytes   239 Mbits/sec   0.056 ms 6389/26740 (24%)
[  3]  3.0- 4.0 sec  28.4 MBytes   239 Mbits/sec   0.081 ms 6460/26745 (24%)
[  4]  3.0- 4.0 sec  28.7 MBytes   241 Mbits/sec   0.059 ms 6250/26747 (23%)
[  5]  3.0- 4.0 sec  29.5 MBytes   247 Mbits/sec   0.061 ms 5726/26742 (21%)
[SUM]  3.0- 4.0 sec   115 MBytes   966 Mbits/sec   0.081 ms 24825/106974 (23%)
[  4]  4.0- 5.0 sec  29.1 MBytes   244 Mbits/sec   0.013 ms 5527/26254 (21%)
[  6]  4.0- 5.0 sec  29.2 MBytes   245 Mbits/sec   0.008 ms 5448/26290 (21%)
[  3]  4.0- 5.0 sec  29.9 MBytes   251 Mbits/sec   0.052 ms 5071/26374 (19%)
[  5]  4.0- 5.0 sec  28.5 MBytes   239 Mbits/sec   0.051 ms 6078/26430 (23%)
[SUM]  4.0- 5.0 sec   117 MBytes   979 Mbits/sec   0.052 ms 22124/105348 (21%)
[  6]  5.0- 6.0 sec  29.4 MBytes   247 Mbits/sec   0.027 ms 6186/27176 (23%)
[  3]  5.0- 6.0 sec  28.2 MBytes   237 Mbits/sec   0.025 ms 6943/27081 (26%)
[  4]  5.0- 6.0 sec  28.5 MBytes   239 Mbits/sec   0.028 ms 6904/27233 (25%)
[  5]  5.0- 6.0 sec  26.6 MBytes   223 Mbits/sec   0.021 ms 8030/27035 (30%)
[SUM]  5.0- 6.0 sec   113 MBytes   946 Mbits/sec   0.028 ms 28063/108525 (26%)
[  6]  6.0- 7.0 sec  29.2 MBytes   245 Mbits/sec   0.009 ms 5914/26776 (22%)
[  3]  6.0- 7.0 sec  29.5 MBytes   247 Mbits/sec   0.042 ms 5738/26777 (21%)
[  4]  6.0- 7.0 sec  30.0 MBytes   252 Mbits/sec   0.054 ms 5321/26730 (20%)
[  5]  6.0- 7.0 sec  29.8 MBytes   250 Mbits/sec   0.016 ms 5554/26783 (21%)
[SUM]  6.0- 7.0 sec   119 MBytes   994 Mbits/sec   0.054 ms 22527/107066 (21%)
[  6]  7.0- 8.0 sec  28.0 MBytes   235 Mbits/sec   0.106 ms 6762/26770 (25%)
[  3]  7.0- 8.0 sec  28.8 MBytes   242 Mbits/sec   0.066 ms 6225/26779 (23%)
[  4]  7.0- 8.0 sec  28.7 MBytes   241 Mbits/sec   0.072 ms 6314/26799 (24%)
[  5]  7.0- 8.0 sec  29.8 MBytes   250 Mbits/sec   0.045 ms 5468/26751 (20%)
[SUM]  7.0- 8.0 sec   115 MBytes   968 Mbits/sec   0.106 ms 24769/107099 (23%)
[  6]  8.0- 9.0 sec  28.2 MBytes   237 Mbits/sec   0.027 ms 6619/26747 (25%)
[  3]  8.0- 9.0 sec  28.1 MBytes   236 Mbits/sec   0.026 ms 6691/26755 (25%)
[  4]  8.0- 9.0 sec  30.4 MBytes   255 Mbits/sec   0.031 ms 5054/26740 (19%)
[  5]  8.0- 9.0 sec  28.4 MBytes   238 Mbits/sec   0.042 ms 6513/26748 (24%)
[SUM]  8.0- 9.0 sec   115 MBytes   966 Mbits/sec   0.042 ms 24877/106990 (23%)
[  4]  0.0-10.0 sec   293 MBytes   246 Mbits/sec   0.011 ms 56945/266247 (21%)
[  5]  0.0-10.0 sec   288 MBytes   241 Mbits/sec   0.021 ms 61338/266482 (23%)
[  6]  9.0-10.0 sec  28.7 MBytes   240 Mbits/sec  15.683 ms 5045/25482 (20%)
[  6]  0.0-10.3 sec   291 MBytes   238 Mbits/sec  15.683 ms 58570/266235 (22%)
[  6] 0.00-10.25 sec  2 datagrams received out-of-order
[  3]  9.0-10.0 sec  29.1 MBytes   244 Mbits/sec  15.679 ms 4831/25583 (19%)

[  3]  0.0-10.3 sec   290 MBytes   237 Mbits/sec  15.679 ms 59771/266339 (22%)
[SUM]  0.0-10.3 sec  1.13 GBytes   951 Mbits/sec  15.683 ms 236624/1065303 (22%)
[SUM] 0.00-10.25 sec  2 datagrams received out-of-order

 

Resolution

The Speedflex result was not exceeding more than 1 Gbps between the AP and controller, even though the Controller and AP’s ethernet speed capabilities was higher than 1 Gbps. This is due to the limitation on the controller side.

Speedflex for AP\Controller performance was designed based on the older models which was capable of 1 Gbps.
UI traffic is treated as data traffic and it goes through the controller’s ETH interface\Dataplane interface and this is not capped. However, when doing the Speeedflex test, it is not treated as data traffic and it goes through the controller Linux KNI interface and this interface is capped to 1 Gbps. This has been also tested with other tools like iPerf and ZAP test.

10 G port on SZ  is designed for UE traffic, not for management/control traffic.
The packet targeted to AP/DP is very limited, including image upgrade / configuration update and some connection to external server, so the traffic is always less than 1 Gbps. As there was no requirement to support over 1 G bandwidth for management/control traffic, the capping has not been improved from 1 Gbps. It can be increased but will take time since this has to go through the development process.

The Speedflex is still considered to be used for AP\Controller speed performance test, however we need to set the right expectation with customer when using the AP models with higher Eth capabilities like R730\R850.


Check the ER-11218 for complete discussion and test results.

We have also updated the guides with respect to higher AP models.

Previous details on the guide:
User-added image

Updated info on the guides:
User-added image

Article Number:
000012313

Updated:
July 14, 2022 11:28 PM (almost 2 years ago)

Tags:
Performance, System Network Management, Troubleshooting, SpeedFlex, virtual SmartCell Gateway, SZ100

Votes:
0

This article is:
helpful
not helpful

Working...Please wait

This is here to prevent you from accidentally submitting twice.

The page will automatically refresh.