APs unable to fail to join vSZ from remote location over WAN
Summary
Why APs unable to fail to join vSZ from remote locationQuestion
Why APs unable to fail to join vSZ from remote location over WAN?Customer Environment
vSZ,SZ,SCG, AP, Aruba Switch, FirewallRoot Cause
The problem was isolated to ArubaMobility Switch Denying FTP traffic on this site.Troubleshooting Steps
Customer had a newly installed setup of vSZ and APs deployed at different locations. Few sites were working ok, however from this new site, none of the AP join vSZ. We confirmed that there is no local firewall in this site. To isolate the problem, we connected one AP in the customers network and in parallel we collected tcpdump on the vSZ. We found that traffic from AP’s public IP was hitting vSZ and we send a response to it. From AP TCPdump we saw LWAPP packets to the vSZ's IP, but no response back.We suspected that the LWAPP packets are being dropped since, bringing up an AP from a different network works and if it is connected to problematic network, it connects fine to vSZ, the whole point of suspicion was the stage of LWAPP2SCG conversion. To rule this out, we upgraded another AP to 3.0.4 version and it came up fine on vSZ. This concreted our belief that there is some firewall rule/restriction on the network which drops LWAPP packets at the AP end. End customer however need proof to prove that.
Further LWAPP2SCG log analysis, we found that vSZ did receive the LWAPP packets and responded back, however firewall seems to be dropping it:
2016-08-03T10:22:13+00:00 RuckusDev lwapp2scg[38177]: [38177:lwappHandler:185] Info: Packet is coming ::ffff:80.78.71.220 10001 89f2c1b6 1~<<<<<<<<<<<
2016-08-03T10:22:13+00:00 RuckusDev lwapp2scg[38177]: [38177:sendLWAPPPacket:65] Info: Discovery Response sent ~<<<<<<<<<<
2016-08-03T10:22:15+00:00 RuckusDev lwapp2scg[38177]: [38177:signal_handler:342] Info: SIGTERM! ...<<<<<<<<<<
2016-08-03T10:22:15+00:00 RuckusDev lwapp2scg[58239]: [58239:loadSettings:260] Info: Work with FTP aware firewall!<<<<<<<<<<<<<
2016-08-03T10:22:15+00:00 RuckusDev lwapp2scg[58239]: [58239:loadSettings:318] Info: 'deny' <<<<<<<<<<<
Logs from AP shows a Discovery Request but never gets a response back:
Oct 21 00:59:11 RuckusAP local2.info syslog: Send Discovery Request message to 40.122.171.143(0), seq_num = 2
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type Discovery type(58) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type WTP Descriptor(3) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type WTP Radio Information(4) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type WTP Static IP Address Info(82) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type WTP Board Data(50) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type Vendor Specific(104) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.debug syslog: LWAPP_MSG_DISC_REQ radio id=0, max_vaps=8, mac=2c:c5:d3:0c:c6:08 country code=21843, environment=32
Oct 21 00:59:11 RuckusAP local2.debug syslog: LWAPP_MSG_DISC_REQ radio id=1, max_vaps=8, mac=2c:c5:d3:0c:c6:0c country code=21843, environment=32
Oct 21 00:59:11 RuckusAP local2.notice syslog: --- add message element 37 (model num ext), length 18 : r500
Oct 21 00:59:11 RuckusAP local2.debug syslog: apgroup name is
Oct 21 00:59:11 RuckusAP local2.info syslog: *** Add ME *** type WTP Static IPv6 Address Info(250) to Discovery Request
Oct 21 00:59:11 RuckusAP local2.info syslog: Start Discovery timer, timeout 5 seconds
We asked access to the intermediate switch noticed that the return traffic from vSZ at Aruba Switch had No Sync flag for LWAPP.
(ArubaS1500-24P) #show datapath session table 40.122.171.143
Datapath Session Table Entries
------------------------------
Flags: F - fast age, S - src NAT, N - dest NAT
D - deny, R - redirect, Y - no syn
H - high prio, P - set prio, T - set ToS
C - client, M - mirror, V - VOIP
Q - Real-Time Quality analysis
I - Deep inspect, U - Locally destined
E - Media Deep Inspect, G - media signal
u - User Index
Source IP/ Destination IP Prot SPort DPort Cntr Prio ToS Age Destination TAge UsrIdx UsrVer Flags
Destination MAC
-------------- -------------- ---- ----- ----- ---- ---- --- --- ----------- ---- ------ ------ -----
40.122.171.143 172.16.6.57 17 12223 12223 0/0 0 0 0 0/0/8 d 0 0 FNY <<<<<<<<
192.168.99.251 40.122.171.143 17 12223 12223 0/0 0 0 0 0/0/8 d 0 0 FSC
Further, customer bypassed their original router and connected Ruckus AP to a Linksys Router through Aruba1500 (Linksys-Aruba1500-AP). This allowed AP to pass the LWAPP however it was failing to download the firmware. While looking into the datapath session for the client IP, we found that switch was Denying the FTP traffic.
(ArubaS1500-24P) #show datapath session table 192.168.99.254
Datapath Session Table Entries
------------------------------
Flags: F - fast age, S - src NAT, N - dest NAT
D - deny, R - redirect, Y - no syn
H - high prio, P - set prio, T - set ToS
C - client, M - mirror, V - VOIP
Q - Real-Time Quality analysis
I - Deep inspect, U - Locally destined
E - Media Deep Inspect, G - media signal
u - User Index
Source IP/ Destination IP Prot SPort DPort Cntr Prio ToS Age Destination TAge UsrIdx UsrVer Flags Destination MAC
-------------- -------------- ---- ----- ----- ---- ---- --- --- ----------- ---- ------ ------ -----
192.168.99.254 40.122.171.143 6 51130 21 0/0 0 0 0 0/0/8 25 0 0 SDCI <<<<<<<<<<<<<<
Resolution
We ruled out that it could be the global firewall configuration or an uplink ACL on the switch denying FTP ingress/egress.To isolate the switch issue, we bypass the switch and connected AP directly to the Linksys Router. Moment we connected the
AP it joined the vSZ in few minutes. Subsequently, all AP’s joined to the vSZ from this site.
“For more information on this topic—including video tutorials—visit the Ruckus Support How-To Hub at https://support.ruckuswireless.com/how-to-hub.”
Article Number:
000005989
Updated:
June 07, 2021 10:47 PM (over 3 years ago)
Tags:
Configuration, Troubleshooting, SmartCell Gateway
Votes:
1
This article is:
helpful
not helpful