Guide to Troubleshoot Unexpected Reboots-Reload on RUCKUS ICX Switches

Summary

Comprehensive Troubleshooting Guide for Unexpected Reboots on ICX Switches: Includes Necessary Steps, Troubleshooting Tips, and Root Cause Identification for Standalone Units and Stacks.

Question

How to troubleshoot Unexpected Reboots on RUCKUS ICX switches?

Customer Environment

Standalone RUCKUS ICX Switch Stack RUCKUS ICX Switches

Symptoms

Unexpected ICX Switch Reboot
Frequent or Recurring Reboots on ICX RUCKUS Switch
Network Load-Induced Reboots
Stack Communication Issues leading on ICX random reboots
Software Crash Reboots on ICX Switches
Power Supply-Induced Reboots on ICX Switches
Overheating-Triggered ICX Switch Reboots
Memory Leak Causing Random Reboots on ICX Switches
Watchdog Timeout Reboots on ICX Switches
Faulty Stack Cable Causing Random ICX Switch Reboots
Hardware Failure-Induced ICX Switch Reboots
Traffic Storm Causing ICX Switch to Randomly Reboot
Inconsistent Power Source Resulting in ICX Reboots

Root Cause

N/A

Troubleshooting Steps

Random Reboots: Troubleshooting Guide for Standalone Units and Stacks

This (KB) guide is intended to help you troubleshoot random reboots on a standalone unit or stack. Please follow the steps below to gather relevant information and investigate potential causes.

Troubleshooting Steps:

Check the Reload Reason

To start investigating the cause of the reboot, check the reload reason by running the following command:

Show version

This will provide information on the reason for the reload. Reference the "Reason for for reload" section in the output for insights. Link to reference : https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-900FDF07-14E7-43D8-8416-F2426B3BBB47.html

If applicable, use the rconsole command to establish a remote console session with the stack member that rebooted. Link to reference : https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-749C8D70-FBAA-4538-875F-5998A800B022.html

Review Logs for Analysis

Review both local and external logs for any error messages or unusual activity. External logs, such as those from a syslog server or SmartZone (SZ), may provide additional insights.
Tip: Look for recurring error messages or warnings that appear before the reboots. These can provide clues about the root cause.

Inspect Hardware Components

Ensure that hardware components like fans and power supply units (PSUs) are functioning correctly and verify that the device is within normal temperature operating ranges. Use the following command:

Show chassis

Link to reference: https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-AC22F512-2359-4C20-B41C-F47797917936.html

Verify CPU and Memory Usage

Check the CPU and memory usage to ensure the device is not overloaded. Run the following commands:

show cpu
show memory

Run the show cpu command about five times at intervals of 30-50 seconds to check for stability.
Additionally, check if any interface is experiencing high utilization by using:

show interfaces | include line|util

Check for Patterns in Logs

Use the show logging command to examine the logs for any patterns that might indicate a correlation with random reboots, such as Layer 2 or Layer 3 loops.

Show logging

Link to reference: https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-D1307AB0-38DE-49B7-9C4D-A40E47174F71.html

Use the show mac-address and show arp commands to monitor the MAC address table and ARP entries. Look for any exponential increase or unusual changes, as this may indicate a Layer 2 loop.

show mac-address
Show arp

Comparing logs from different time periods can help identify recurring issues or changes in network behavior.

Inspect Stack Member Communication

If the rebooted unit is a stack member, check for CRC errors, input errors, or other communication issues that might cause instability within the stack. Related article : https://support.ruckuswireless.com/articles/000006902
Run the following commands to inspect the specific stack interfaces:

show interface ethernet x/x/x

Look for any input/output errors or CRC errors. Link to reference https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-54E45EEA-5E28-49B9-B4C7-DCA7811947F6.html
If CRC errors are increasing, clear the interface statistics to see if the counters keep increasing:

clear statistics ethernet x/x/x

Verify if the stacking media is valid and supported by the unit by using the following commands:

show media ethernet x/x/x
show media validation ethernet x/x/x

Tip: Regularly inspect and maintain stack cables and connections to prevent physical layer 1 issues.

Check for Crash Information

Typically, when a random reboot occurs due to a software crash or process issue, the reboot details are stored for future investigation. To check if this information is available, run: show files /cores/
If the output displays core dump files, ensure that the timestamp corresponds to the time of the reboot.

show files /cores/

Device# show files /cores/
F 99215437 core_1153_FastIron-1_2024-07-23_00-45-56.gz

Important Note: Collect the output as soon as possible and for every occurrence.

If relevant core dump files are present, download the support save and share it with the TAC team if you have an open case for further investigation. Link to reference : https://community.ruckuswireless.com/t5/RUCKUS-Self-Help/Methods-for-Collecting-Support-Save-from-ICX-Switches/m-p/71219

Configure Log Persistence

To retain system log messages after a soft reboot, configure log persistence on the device. Use the following command:

logging persistence

Link to reference: https://docs.commscope.com/bundle/fastiron-10010-commandref/page/GUID-C2C46849-8528-40E4-9B75-904A7FFD2C1D.html

Check for SNMP Tools or Scripts

If there is a recent SNMP tool or script running, or a newly added tool, stop the process and verify if it’s the cause of the random reboots.
Ensure that any SNMP tools or scripts are properly configured and not overloading the device with requests.

Additional Monitoring and Recording of Console Sessions

1. Console the Affected Unit:
Connect to the console of the affected unit and leave the console session open.

Enable the following commands :

debug destination console
logging console

2. Record the Console Session:
Use a terminal emulator that supports session recording to capture the console output. This will help document any messages or errors that appear during the reboot process.

Link as reference: https://my.kualo.com/knowledgebase/?kbcat=0&article=888

3. Monitor for Reboot Signatures:
If the unit reboots again, the reason for the reboot is likely to show up in the console session. The reboot signature will be displayed during the reboot process.

4. Share the Recorded Session:
Save the recorded console session file and share it with the TAC team for further investigation. This file will provide valuable insights into the cause of the random reboots.

Resolution

Reefer to the troubleshooting section.

Article Number:
000014619

Updated:
May 26, 2025 12:48 PM (25 days ago)

Tags:
Troubleshooting, Ruckus ICX Switches

Votes:
0

This article is:
helpful
not helpful