Introduction

This document describes how to streamline the upgrade process for Cisco Catalyst 3850 Series Switches and offers disaster recovery techniques in the case of a software or boot failure. The upgrade process is simple, but it can seem difficult if you do not have experience with Cisco IOS^® XE software upgrades.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• TFTP

• FTP

Components Used

The information in this document is based on the Cisco Catalyst 3850 Series Switch that runs Cisco IOS-XE Versions 03.03.00 and later. The examples in this document use a stacked solution; however, the same commands can be run on a standalone switch.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Note: In order to download Cisco IOS-XE images from the Cisco website, you must have a valid Cisco Connection Online (CCO) account with entitled credentials. Also, Cisco does not offer a free TFTP/FTP solution, so you must install and configure the TFTP/FTP before you begin.

Install Versus Bundle Mode

When you deploy the Cisco Catalyst 3850 Series Switch, you have two modes of operation to choose from: INSTALL and BUNDLE. There are minor differences between the two modes. Review the configuration guide for more details.

Cisco recommends that the INSTALL mode is used during operation because it allows for a more complete set of features and requires fewer resources upon boot. This document provides a brief overview of each mode for reference.

Note: Software defect may result in switch inoperability due to flash memory corruption when upgrading from a pre-3.6.3 or pre-3.7.2 software release to 3.6.3/3.7.2 in install mode. This issue is resolved in 3.6.4 or 3.7.3 or later software therefore the issue is not seen when upgrading to these software versions. As a result of this software defect, bundle mode is recommended for upgrades involving the effected software versions

Install Mode

This is the default mode for the switch. The INSTALL mode uses a package-provisioning file named packages.conf in order to boot the switch. In addition, there are a number of .pkg files in the flash. Cisco recommends that you do not alter these files unless directed by a Cisco Technical Assistance Center (TAC) engineer. 

Bundle Mode

If you are comfortable with the use of traditional monolithic IOS images in order to boot the switch, then the BUNDLE mode should be familiar. The BUNDLE mode consumes more memory than the INSTALL mode because the packages are extracted from the Bundle and copied to the RAM.

Verify the Mode

In order to verify the mode, enter the show version command:

3850-stack#

show version

Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.00SE RELEASE SOFTWARE (fc1)Switch Ports Model           SW Version     SW Image

Mode

  ------------------           ----------     ----------            ----       1 32    WS-C3850-24P    03.03.00SE     cat3k_caa-universalk9

INSTALL

*    2 56    WS-C3850-48T    03.03.00SE     cat3k_caa-universalk9

INSTALL

Upgrade

In order to begin the upgrade process, you must download the IOS-XE .bin file from the Cisco web site and place it in the flash of your active switch. The process that is used in order to copy the file to the switch is not covered in this document.

When you copy the .bin file to a single switch, the install process replicates the file to the other switches in the stack. Once the file is present, enter this command:

3850-stack#

software install file flash:cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin switch 1-2

Note: There are many options available after each command; however, in this example a basic upgrade is run.

Once you begin the upgrade process, the switch pushes the .bin file to the stack member peers.

Preparing install operation ...[2]: Copying software from active switch 1 to switch 2

After all of the members receive the .bin file, it is automatically expanded to the flash.

[1 2]: Starting install operation[1 2]: Expanding bundle flash: cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin[1 2]: Copying package files[1 2]: Package files copied[1 2]: Finished expanding bundle flash: cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin[1 2]: Verifying and copying expanded package files to flash:[1 2]: Verified and copied expanded package files to flash:[1 2]: Starting compatibility checks[1 2]: Finished compatibility checks[1 2]: Starting application pre-installation processing[1 2]: Finished application pre-installation processing

Next, the switch lists a summary of the files that are marked for both removal and addition to the packages.conf pointer file.

[1]: Old files list:    Removed cat3k_caa-base.SPA.03.03.00SE.pkg    Removed cat3k_caa-drivers.SPA.03.03.00SE.pkg    Removed cat3k_caa-infra.SPA.03.03.00SE.pkg    Removed cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg    Removed cat3k_caa-platform.SPA.03.03.00SE.pkg    Removed cat3k_caa-wcm.SPA.10.1.100.0.pkg[2]: Old files list:    Removed cat3k_caa-base.SPA.03.03.00SE.pkg    Removed cat3k_caa-drivers.SPA.03.03.00SE.pkg    Removed cat3k_caa-infra.SPA.03.03.00SE.pkg    Removed cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg    Removed cat3k_caa-platform.SPA.03.03.00SE.pkg    Removed cat3k_caa-wcm.SPA.10.1.100.0.pkg[1]: New files list:    Added cat3k_caa-base.SPA.03.03.01SE.pkg    Added cat3k_caa-drivers.SPA.03.03.01SE.pkg    Added cat3k_caa-infra.SPA.03.03.01SE.pkg    Added cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg    Added cat3k_caa-platform.SPA.03.03.01SE.pkg    Added cat3k_caa-wcm.SPA.10.1.110.0.pkg[2]: New files list:    Added cat3k_caa-base.SPA.03.03.01SE.pkg    Added cat3k_caa-drivers.SPA.03.03.01SE.pkg    Added cat3k_caa-infra.SPA.03.03.01SE.pkg    Added cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg    Added cat3k_caa-platform.SPA.03.03.01SE.pkg    Added cat3k_caa-wcm.SPA.10.1.110.0.pkg

Lastly, the packges.conf file is updated and committed.

[1 2]: Creating pending provisioning file[1 2]: Finished installing software.  New software will load on reboot.[1 2]: Committing provisioning file[1 2]: Do you want to proceed with reload? [yes/no]: yes

Verify that the update process is properly completed upon reload.

3850-stack#

show ver | i INSTALL

     1 32    WS-C3850-24P     03.03.01SE      cat3k_caa-universalk9 INSTALL*    2 56    WS-C3850-48T     03.03.01SE      cat3k_caa-universalk9 INSTALL

Flash Cleanup

Residual files remain in the flash from previous versions. If you want to clean up the residual files, you can enter the software clean command instead of a manual deletion of the files. This purges the files that the switch no longer needs in order to operate from the flash of each stack member. 

Note: This command also deletes the .bin file that is used in order to install the new IOS software. It is important to remember that once it is extracted, you no longer need it.

The next two sections provide examples of how the flash appears before and after the software clean command is used.

Before Flash Cleanup

3850-stack#

show flash

-#- --length-- ---------date/time----------- -----------------------path-----------------------  2    2097152 Feb 16 2014 11:38:46.0 +00:00 nvram_config  4  257016048 Jan 28 2014 17:22:12.0 +00:00 cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin  5       4096 Jan 28 2014 17:25:50.0 +00:00 mnt  6       4096 Jan 28 2014 17:25:50.0 +00:00 mnt/images  7       4096 Jan 28 2014 17:25:52.0 +00:00 mnt/images/ap.bak  8         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2.md5  9   11591680 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2 10         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1.md5 11   10444800 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1 12         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2.md5 13   13568000 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2 14         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140.md5 15   10291200 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140 16         11 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/version.info 17       1214 Jan 28 2014 17:25:10.0 +00:00 packages.conf.00- 18   79112096 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-base.SPA.03.03.00SE.pkg 19    6474428 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-drivers.SPA.03.03.00SE.pkg 20   34501468 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-infra.SPA.03.03.00SE.pkg 21       1248 Feb 16 2014 11:27:51.0 +00:00 packages.conf 22   34763952 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg 23        796 Feb 19 2014 11:43:13.0 +00:00 vlan.dat 24   24992476 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-platform.SPA.03.03.00SE.pkg 25   77167308 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-wcm.SPA.10.1.100.0.pkg 26       1224 Jan 28 2014 16:39:58.0 +00:00 packages.conf.01- 27       6571 Dec 20 2013 08:56:32.0 +00:00 BLANK_CONFIG.cfg 28  257193048 Feb 16 2014 11:19:44.0 +00:00 cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin 30   79113792 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-base.SPA.03.03.01SE.pkg 31   74409080 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-base.SPA.03.02.01.SE.pkg 32    2775728 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-drivers.SPA.03.02.01.SE.pkg 33    6476476 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-drivers.SPA.03.03.01SE.pkg 34   32478052 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-infra.SPA.03.02.01.SE.pkg 35   30389028 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg 36   18313952 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-platform.SPA.03.02.01.SE.pkg 37   63402700 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-wcm.SPA.10.0.101.0.pkg 38   34503664 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-infra.SPA.03.03.01SE.pkg 39   34788684 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg 40   25009040 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-platform.SPA.03.03.01SE.pkg 41   77296448 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-wcm.SPA.10.1.110.0.pkg

237428736 bytes available (1302147072 bytes used)

After Flash Cleanup

3850-stack#

software clean

Preparing clean operation ...[1 2]: Cleaning up unnecessary package files[1 2]: No path specified, will use booted path flash:packages.conf[1 2]: Cleaning flash:[1]: Preparing packages list to delete ...     In use files, will not delete:       cat3k_caa-base.SPA.03.03.01SE.pkg       cat3k_caa-drivers.SPA.03.03.01SE.pkg       cat3k_caa-infra.SPA.03.03.01SE.pkg       cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg       cat3k_caa-platform.SPA.03.03.01SE.pkg       cat3k_caa-wcm.SPA.10.1.110.0.pkg       packages.conf[2]: Preparing packages list to delete ...     In use files, will not delete:       cat3k_caa-base.SPA.03.03.01SE.pkg       cat3k_caa-drivers.SPA.03.03.01SE.pkg       cat3k_caa-infra.SPA.03.03.01SE.pkg       cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg       cat3k_caa-platform.SPA.03.03.01SE.pkg       cat3k_caa-wcm.SPA.10.1.110.0.pkg       packages.conf[1]: Files that will be deleted:    cat3k_caa-base.SPA.03.02.01.SE.pkg    cat3k_caa-base.SPA.03.03.00SE.pkg    cat3k_caa-drivers.SPA.03.02.01.SE.pkg    cat3k_caa-drivers.SPA.03.03.00SE.pkg    cat3k_caa-infra.SPA.03.02.01.SE.pkg    cat3k_caa-infra.SPA.03.03.00SE.pkg    cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg    cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg    cat3k_caa-platform.SPA.03.02.01.SE.pkg    cat3k_caa-platform.SPA.03.03.00SE.pkg    cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin    cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin    cat3k_caa-wcm.SPA.10.0.101.0.pkg    cat3k_caa-wcm.SPA.10.1.100.0.pkg    packages.conf.00-    packages.conf.01-[2]: Files that will be deleted:    cat3k_caa-base.SPA.03.02.01.SE.pkg    cat3k_caa-base.SPA.03.03.00SE.pkg    cat3k_caa-drivers.SPA.03.02.01.SE.pkg    cat3k_caa-drivers.SPA.03.03.00SE.pkg    cat3k_caa-infra.SPA.03.02.01.SE.pkg    cat3k_caa-infra.SPA.03.03.00SE.pkg    cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg    cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg    cat3k_caa-platform.SPA.03.02.01.SE.pkg    cat3k_caa-platform.SPA.03.03.00SE.pkg    cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin    cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin    cat3k_caa-wcm.SPA.10.0.101.0.pkg    cat3k_caa-wcm.SPA.10.1.100.0.pkg    packages.conf.00-    packages.conf.01-[1 2]: Do you want to proceed with the deletion? [yes/no]:

yes

[1 2]: Clean up completed

Here is the output from the show flash command after the flash cleanup:

3850-stack#

show flash

-#- --length-- ---------date/time----------- -------------------path--------------------  2    2097152 Feb 16 2014 11:38:46.0 +00:00 nvram_config  4       4096 Jan 28 2014 17:25:50.0 +00:00 mnt  5       4096 Jan 28 2014 17:25:50.0 +00:00 mnt/images  6       4096 Jan 28 2014 17:25:52.0 +00:00 mnt/images/ap.bak  7         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2.md5  8   11591680 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2  9         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1.md5 10   10444800 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1 11         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2.md5 12   13568000 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2 13         40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140.md5 14   10291200 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140 15         11 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/version.info 16       1248 Feb 16 2014 11:27:51.0 +00:00 packages.conf 17        796 Feb 19 2014 11:43:13.0 +00:00 vlan.dat 18       6571 Dec 20 2013 08:56:32.0 +00:00 BLANK_CONFIG.cfg 20   79113792 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-base.SPA.03.03.01SE.pkg 21    6476476 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-drivers.SPA.03.03.01SE.pkg 22   34503664 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-infra.SPA.03.03.01SE.pkg 23   34788684 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg 24   25009040 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-platform.SPA.03.03.01SE.pkg 25   77296448 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-wcm.SPA.10.1.110.0.pkg

1231515648 bytes available (308060160 bytes used)

Auto-Upgrade Feature for Catalyst 3850 Series Switches

There are many scenarios where you must introduce a new switch into a current stack of Catalyst 3850 Series Switches, such as when a new switch is purchased in order to expand the number of usable ports in the stack. In order to successfully add a new switch to a stack, you must ensure that the same software version is run on the new switch. Prior to IOS-XE Version 3.3.1, the only way to ensure that the versions match is to stage the new switch prior to introduction into the stack.

The Catalyst 3850 Series Switches include a feature called Auto-Upgrade that might be familiar if your migration occurs from the Catalyst 3750 Series platform. The goal of this feature is to ensure that a newly added switch is automatically provisioned by the stack members with the correct IOS-XE version. However, one caveat to note is that prior to Version 3.3.1, this feature does not work (reference Cisco bug ID ). Therefore, you must ensure that your current stack runs at least Version 3.3.1 in INSTALL mode.

Note: Auto-Upgrade is disabled by default and is not available in BUNDLE mode.

In order to use the Auto-Upgrade feature, add the software auto-upgrade enable command into the configuration of the current stack. This ensures that any newly added stack members are automatically upgraded.

Configure

Once the switch is stacked and boots up, you see a clear indication that there is a mismatch in versions, and the new member does not fully join the stack. If you watch the SYSLOG as the switch attempts to join, you notice that the Auto-Advise feature alerts you that the newly added switch runs a different software version and mode.

Note: For this example, the new switch runs IOS-XE Version 3.2.2 in BUNDLE mode.

%STACKMGR-1-STACK_LINK_CHANGE: STANDBY: 1 stack-mgr: Stack port 2 on switch 1 is up  (3850-Stack-1)%STACKMGR-1-STACK_LINK_CHANGE: 2 stack-mgr: Stack port 1 on switch 2 is up%STACKMGR-6-SWITCH_ADDED: 2 stack-mgr: Switch 3 has been added to the stack.%STACKMGR-6-SWITCH_ADDED: STANDBY:1 stack-mgr: Switch 3 has been added to the stack.  (3850-Stack-1)%INSTALLER-6-AUTO_ADVISE_SW_INITIATED: 2 installer: Auto advise initiated for switch 3%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: Switch 3 running bundled software has been added%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: to the stack that is running installed software.%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: The 'software auto-upgrade' command can be used to%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: convert switch 3 to the installed running mode by%INSTALLER-6-AUTO_ADVISE_SW: 2 installer: installing its running software.

Once the newly joined member is fully booted, you see that a mismatch is detected:

3850-Stack#

show switch

Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac AddressMac persistency wait time: Indefinite                                             H/W   CurrentSwitch#   Role    Mac Address     Priority Version  State------------------------------------------------------------*1       Active   0c27.24cf.ab80     14      B0      Ready              2       Standby  f41f.c238.a800     13      B0      Ready              3       Member   b4e9.b0d3.6600     12      0       V-Mismatch

Enable Auto-Upgrade Feature

In Global Configuration mode, enter the software auto-upgrade enable command. This enables the feature for any new switches that join the stack.

3850-Stack(config)

#

software auto-upgrade enable

3850-Stack(config)

#

end

You must reload the newly added switch only; a full stack reload is not necessary. In this case, the newly added switch is switch 3, so the reload slot 3 command is entered. 

Tip: The slot mentioned in these commands designates the switch in the stack (slot 1 = switch 1).

3850-Stack#

reload slot 3

Proceed with reload?

[confirm]

%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Reload Slot Command%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack.%STACKMGR-1-STACK_LINK_CHANGE: STANDBY: 2 stack-mgr:  Stack port 1 on switch 2 is down  (3850-Stack-2)Starting SWITCH-DELETE sequence, switch 3SWITCH-DELETE sequence complete, switch 3%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack.  (3850-Stack-2)Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2)SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)

Wait a few minutes while the switch reloads in the background. Then, you see this:

%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is up3850-Stack#%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is up  (3850-Stack-2)3850-Stack#%STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack.%STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack.  (3850-Stack-2)

You now see that the conversion from BUNDLE to INSTALL mode occurs, followed by a reload:

%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Converting switch 3 to installed mode by%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: installing its running software%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Setting the boot var on switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing the running software on switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to boot in installed mode%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been r3850-Stack#emoved from the stack.%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is down  (3850-Stack-2)Starting SWITCH-DELETE sequence, switch 3SWITCH-DELETE sequence complete, switch 3%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack.  (3850-Stack-2)3850-Stack#Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2)SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)

After the reboot, the upgrade continues:

%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Searching stack for software to upgrade switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Found donor switch 1 to auto upgrade switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Upgrading switch 3 with software from switch 1%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing software on switch 3%INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to complete the auto upgrade%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down%STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack.%STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port3850-Stack#t 1 on switch 2 is down  (3850-Stack-2)Starting SWITCH-DELETE sequence, switch 3SWITCH-DELETE sequence complete, switch 3%STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack.  (3850-Stack-2)

Another reload is performed automatically. Once the switch boots up, it successfully joins the stack with the correct IOS-XE version and software mode.

%STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack.%STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack.  (3850-Stack-2)%STACKMGR-6-SWITCH_READY: STANDBY:2 stack-mgr: Switch 3 is ready.  (3850-Stack-2)%STACKMGR-6-SWITCH_READY: 1 stack-mgr:  Switch 3 is ready.Starting SWITCH-ADD sequence, switch 3%NGWC_USB_CONSOLE-6-CONFIG_ENABLE: Switch 3: Console media-type changed to defaultStarting SWITCH-ADD sequence, switch 3 (3850-Stack-2)SWITCH-ADD sequence complete, switch 3 (3850-Stack-2)SWITCH-ADD sequence complete, switch 3

Verify

Use the show switch and show version commands in order to verify that the upgrade process is completed properly:

3850-Stack#

show switch

Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac AddressMac persistency wait time: Indefinite                                             H/W   CurrentSwitch#   Role    Mac Address     Priority Version  State------------------------------------------------------------*1       Active   0c27.24cf.ab80     14     B0      Ready              2       Standby  f41f.c238.a800     13     B0      Ready              3       Member   b4e9.b0d3.6600     12     B0      Ready3850-Stack#

show version

Switch Ports Model          SW Version    SW Image              Mode ------ ----- -----          ----------    ----------            ---- *    1 56    WS-C3850-48P   03.03.01SE    cat3k_caa-universalk9 INSTALL     2 56    WS-C3850-48P   03.03.01SE    cat3k_caa-universalk9 INSTALL     3 56    WS-C3850-48P   03.03.01SE    cat3k_caa-universalk9 INSTALL

Recover from a 3850 Series Switch Boot Failure

There are multiple reasons for a 3850 Series Switch boot failure, such as a corrupt boot image, a corrupt packages.conf file, or missing files. This section describes possible recovery methods.

Note: Ensure that you have knowledge of the two possible boot modes, INSTALL and BUNDLE, before you continue.

Standard Recovery Methods

This section describes the two standard methods that are used in order to recover from a Catalyst 3850 Series Switch boot failure.

USB Recovery

The 3850 Series Switches have a USB port on the front that is used for console access. This USB port is also used with flash drives for image backup and recovery.

If you become stuck at the switch: prompt with a corrupt image or .conf file, you can easily boot to a file that is stored on the USB drive or copy an image from the USB to internal flash. Complete these steps in order to recover from the boot failure:

1. Verify that the flash drive is recognized and the .bin file exists:

   switch:

   dir usbflash0:

   Directory of usbflash0:/74  -rw-  223734376  cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin

2. Boot to the USB image:

   switch:

   boot usbflash0:cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin

Note: This process boots the switch into BUNDLE mode.

Tip: You can also copy the .bin file from usbflash0: to flash:, and point the boot statement towards internal flash.

Corrupt File Recovery

There are instances when the packages.conf calls files that no longer exist in flash. You can manually boot an image from the switch: prompt file; however, upon reload it calls the packages.conf file again and fails to boot. If this occurs, Cisco recommends that you back up the current packages.conf file and rename it or delete it. This process is mandatory, as the next step fails if a .conf file already exists. Once you extract the .bin file, a new packages.conf file is created. Complete these steps in order to recover from a corrupt packages.conf file:

1. Once booted (in BUNDLE mode), verify the files in the flash:

   Switch#

   dir flash:

   Directory of flash:/15500  -rwx        1243   Aug 1 2013 07:04:02 +00:00  packages.conf

2. Copy or rename the current packages.conf file:

   Switch#

   cp flash:packages.conf flash:packages.conf.badop

   Destination filename [packages.conf.bad]?Copy in progress...C1243 bytes copied in 0.140 secs (8879 bytes/sec)Switch#

   dir flash:

   Directory of flash:/15500  -rwx        1243   Aug 1 2013 07:04:02 +00:00  packages.conf15502  -rw-        1243   Aug 1 2013 11:53:51 +00:00  packages.conf.badSwitch#

   del flash:packages.conf

   Delete filename [packages.conf]?Delete flash:/packages.conf? [confirm]

3. Expand the bundle in order to create a new packages.conf file: 

   Switch#

   software expand running switch 1 to flash:

   Preparing expand operation ...[1]: Expanding the running bundle[1]: Copying package files[1]: Package files copied[1]: Finished expanding the running bundle

4. Verify the boot:

   Switch#

   show boot

   ---------------------------Switch 1---------------------------Current Boot Variables:BOOT variable does not existBoot Variables on next reload:BOOT variable = flash:packages.conf;Manual Boot = noEnable Break = no

5. Reload the switch:

   Switch#

   reload

   Reload command is being issued on Active unit, this will reload the whole stackProceed with reload? [confirm]

Emergency Recovery

If the previous recovery methods fail, the 3850 Series Switches have a trap door method that you can use in order to recover the system. You must have a terminal that is connected to the management port of the switch that runs a TFTP server. Download a valid image file from CCO and store it in the root of the TFTP server.

It is likely that the switch is stuck at the switch: prompt. However, if you are in a boot loop, you can use the Mode button on the front of the switch in order to break the cycle: hold the button for approximately ten seconds, and the switch breaks the cycle and stops at the switch: prompt.

Complete these steps in order to perform an emergency recovery:

1. Set the switch IP address:

   switch:

   set IP_ADDR 192.0.2.123/255.255.255.0

2. Set the default gateway:

   switch:

   set DEFAULT_ROUTER 192.0.2.1

3. Ping the terminal that contains the TFTP server in order to test the connectivity:

   switch:

   ping 192.0.2.1

   ping 192.0.2.1 with 32 bytes of data ...Host 192.0.2.1 is alive.

4. Verify that the emergency files exist in the switch file system:

   switch:

   dir sda9:

   Directory of sda9:/    2  drwx  1024       .    2  drwx  1024       ..   11  -rwx  18958824   cat3k_caa-recovery.bin36903936 bytes available (20866048 bytes used)

5. Run the emergency install feature:

   switch:

   emergency-install tftp://192.0.2.1/cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin

   The bootflash will be erased during install operation, continue (y/n)?YStarting emergency recovery (tftp://192.0.2.1/cat3k_caa-universalk9. SPA.03.02.02.SE.150-1.EX2.bin)...Reading full image into memory......................doneNova Bundle Image--------------------------------------Kernel Address    : 0x6042f5d8Kernel Size       : 0x317ccc/3243212Initramfs Address : 0x607472a4Initramfs Size    : 0xdc6546/14443846Compression Format: .mzipBootable image at @ ram:0x6042f5d8Bootable image segment 0 address range [0x81100000, 0x81b80000] is in range [0x80180000, 0x90000000].@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@File "sda9:cat3k_caa-recovery.bin" uncompressed and installed, entry point: 0x811060f0Loading Linux kernel with entry point 0x811060f0 ...Bootloader: Done loading app on core_mask: 0xf### Launching Linux Kernel (flags = 0x5)Initiating Emergency Installation of bundle tftp://192.0.2.1/ cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.binDownloading bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin...Validating bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin...Installing bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin...Verifying bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin...Package cat3k_caa-base.SPA.03.03.00.SE.pkg is Digitally SignedPackage cat3k_caa-drivers.SPA.03.03.00.SE.pkg is Digitally SignedPackage cat3k_caa-infra.SPA.03.03.00.SE.pkg is Digitally SignedPackage cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg is Digitally SignedPackage cat3k_caa-platform.SPA.03.03.00.SE.pkg is Digitally SignedPackage cat3k_caa-wcm.SPA.10.0.111.0.pkg is Digitally SignedPreparing flash...Syncing device...Emergency Install successful... RebootingRestarting system.