Book excerpt from NX-OS and Cisco Nexus Switching

Excerpt from NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures.

1 2 3 4 5 Page 4
Page 4 of 5
N7010-1# dir bootflash:        311     Jun 20 05:15:05 2009  MDS20090619155920643.lic        309     Jun 20 05:15:56 2009  MDS20090619155929839.lic    2470887     Aug 01 08:13:35 2009  dp42    8533440     Apr 17 23:17:14 2009  lacp_tech_all.log     308249     Aug 01 09:08:39 2009  libcmd.so        134     Jun 19 23:06:53 2009  libglbp.log        175     Jun 20 04:14:22 2009  libotm.log      49152     Jun 19 22:50:53 2009  lost+found/   87081184     Jan 02 06:21:20 2008  congo-s1-dk9.4.0.2.bin   87755113     Dec 11 13:35:25 2008  congo-s1-dk9.4.0.4.bin   92000595     Apr 16 21:55:19 2009  congo-s1-dk9.4.1.4.bin   92645614     Apr 08 06:08:35 2009  congo-s1-dk9.4.1.5.bin   92004757     Jun 02 04:29:19 2009  congo-s1-dk9.4.1.5E2.bin   99851395     Aug 03 05:17:46 2009  congo-s1-dk9.4.2.0.601.bin  100122301     Aug 12 04:42:13 2009  congo-s1-dk9.4.2.1.bin    9905740     Jan 02 06:21:29 2008  congo-s1-epld.4.0.2.img    9730124     Dec 11 13:42:30 2008  congo-s1-epld.4.0.4.img   23584768     Jan 02 06:21:26 2008  congo-s1-kickstart.4.0.2.bin   23785984     Dec 11 13:34:37 2008  congo-s1-kickstart.4.0.4.bin   24718848     Apr 16 21:52:40 2009  congo-s1-kickstart.4.1.4.bin   25173504     Apr 08 06:00:57 2009  congo-s1-kickstart.4.1.5.bin   23936512     Aug 03 05:03:13 2009  congo-s1-kickstart.4.1.5E2.bin   25333248     Aug 03 05:18:37 2009  congo-s1-kickstart.4.2.0.601.bin   25234944     Aug 12 04:40:52 2009  congo-s1-kickstart.4.2.1.bin      12558     Aug 01 08:51:22 2009  shrun     916893     Apr 17 23:23:03 2009  stp_tech.og       4096     Dec 11 14:04:50 2008  vdc_2/       4096     Dec 11 14:04:50 2008  vdc_3/       4096     Dec 11 14:04:50 2008  vdc_4/     592649     Apr 17 23:18:16 2009  vpc_tech.log        942     Jul 10 09:45:27 2009  wiresharkUsage for bootflash://sup-local  982306816 bytes used  827592704 bytes free 1809899520 bytes totalN7010-1# dir bootflash://sup-remote      12349     Dec 05 02:15:33 2008  7k-1-vdc-all.run       4096     Apr 04 06:45:28 2009  eem/      18180     Apr 02 23:47:26 2009  eem_script.cfg   99851395     Aug 03 05:20:20 2009  congo-s1-dk9.4.2.0.601.bin  100122301     Aug 12 04:46:18 2009  congo-s1-dk9.4.2.1.bin      19021     Apr 03 21:04:50 2009  eem_script_counters.cfg      19781     Apr 05 23:30:51 2009  eem_script_iptrack.cfg      29104     Jun 19 22:44:51 2009  ethpm_act_logs.log          0     Jun 19 22:44:51 2009  ethpm_syslogs.log        175     Jun 20 04:14:37 2009  libotm.log      49152     Jun 19 22:38:45 2009  lost+found/   87755113     Apr 07 23:54:07 2009  congo-s1-dk9.4.0.4.bin   92000595     Apr 16 21:55:19 2009  congo-s1-dk9.4.1.4.bin   92645614     Apr 08 06:08:35 2009  congo-s1-dk9.4.1.5.bin   92004757     Jun 02 04:29:19 2009  congo-s1-dk9.4.1.5E2.bin   10993389     Mar 22 04:55:13 2009  congo-s1-epld.4.1.3.33.img   23785984     Apr 07 23:47:43 2009  congo-s1-kickstart.4.0.4.bin   24718848     Apr 16 21:52:40 2009  congo-s1-kickstart.4.1.4.bin   25173504     Apr 08 06:00:57 2009  congo-s1-kickstart.4.1.5.bin   23936512     Jun 02 04:26:35 2009  congo-s1-kickstart.4.1.5E2.bin   25333248     Aug 03 05:19:26 2009  congo-s1-kickstart.4.2.0.601.bin   25234944     Aug 12 04:45:24 2009  congo-s1-kickstart.4.2.1.bin        310     Sep 19 03:58:55 2008  n7k-rhs-1.lic      12699     Jan 23 14:02:52 2009  run_vpc_jan22      11562     Mar 13 07:52:42 2009  startup-robert-cfg      16008     Mar 12 02:02:40 2009  startup-vss-cfg      17315     Mar 19 06:24:32 2009  startup-vss-cfg_roberto_mar18         99     Apr 04 06:51:15 2009  test1       9991     Jun 19 23:12:48 2009  vdc.cfg       4096     Jan 22 13:37:57 2009  vdc_2/       4096     Jan 22 00:40:57 2009  vdc_3/       4096     Sep 11 12:54:10 2008  vdc_4/     111096     Dec 20 04:40:17 2008  vpc.cap          0     Feb 03 08:02:14 2009  vpc_hw_check_disable      18166     Apr 03 03:24:22 2009  vpc_vss_apr02      18223     Apr 02 22:40:57 2009  vss_vpc_apr2Usage for bootflash://sup-remote  863535104 bytes used  946364416 bytes free 1809899520 bytes totalN7010-1# copy bootflash://supbootflash://sup-1/        bootflash://sup-active/   bootflash://sup-remote/bootflash://sup-2/        bootflash://sup-local/    bootflash://sup-standby/N7010-1# copy bootflash://sup-local/congo-s1-epld.4.0.4.img bootflash://sup-remote/congo-s1-epld.4.0.4.imgN7010-1# dir bootflash://sup-remote      12349     Dec 05 02:15:33 2008  7k-1-vdc-all.run       4096     Apr 04 06:45:28 2009  eem/      18180     Apr 02 23:47:26 2009  eem_script.cfg      19021     Apr 03 21:04:50 2009  eem_script_counters.cfg      19781     Apr 05 23:30:51 2009  eem_script_iptrack.cfg      29104     Jun 19 22:44:51 2009  ethpm_act_logs.log          0     Jun 19 22:44:51 2009  ethpm_syslogs.log        175     Jun 20 04:14:37 2009  libotm.log      49152     Jun 19 22:38:45 2009  lost+found/   87755113     Apr 07 23:54:07 2009  congo-s1-dk9.4.0.4.bin   92000595     Apr 16 21:55:19 2009  congo-s1-dk9.4.1.4.bin   92645614     Apr 08 06:08:35 2009  congo-s1-dk9.4.1.5.bin   92004757     Jun 02 04:29:19 2009  congo-s1-dk9.4.1.5E2.bin   99851395     Aug 03 05:20:20 2009  congo-s1-dk9.4.2.0.601.bin  100122301     Aug 12 04:46:18 2009  congo-s1-dk9.4.2.1.bin    9730124     Aug 12 22:02:57 2009  congo-s1-epld.4.0.4.img   10993389     Mar 22 04:55:13 2009  congo-s1-epld.4.1.3.33.img   23785984     Apr 07 23:47:43 2009  congo-s1-kickstart.4.0.4.bin   24718848     Apr 16 21:52:40 2009  congo-s1-kickstart.4.1.4.bin   25173504     Apr 08 06:00:57 2009  congo-s1-kickstart.4.1.5.bin   23936512     Jun 02 04:26:35 2009  congo-s1-kickstart.4.1.5E2.bin   25333248     Aug 03 05:19:26 2009  congo-s1-kickstart.4.2.0.601.bin   25234944     Aug 12 04:45:24 2009  congo-s1-kickstart.4.2.1.bin        310     Sep 19 03:58:55 2008  n7k-rhs-1.lic      12699     Jan 23 14:02:52 2009  run_vpc_jan22      11562     Mar 13 07:52:42 2009  startup-robert-cfg      16008     Mar 12 02:02:40 2009  startup-vss-cfg      17315     Mar 19 06:24:32 2009  startup-vss-cfg_roberto_mar18         99     Apr 04 06:51:15 2009  test1       9991     Jun 19 23:12:48 2009  vdc.cfg       4096     Jan 22 13:37:57 2009  vdc_2/       4096     Jan 22 00:40:57 2009  vdc_3/       4096     Sep 11 12:54:10 2008  vdc_4/     111096     Dec 20 04:40:17 2008  vpc.cap          0     Feb 03 08:02:14 2009  vpc_hw_check_disable      18166     Apr 03 03:24:22 2009  vpc_vss_apr02      18223     Apr 02 22:40:57 2009  vss_vpc_apr2Usage for bootflash://sup-remote  873283584 bytes used  936615936 bytes free 1809899520 bytes totalN7010-1#

Configuration Files: Configuration Rollback

The configuration rollback feature enables you to take a snapshot, or checkpoint, of the Cisco NX-OS configuration and then reapply that configuration to your device at any point without having to reload the device. Rollback allows any authorized administrator to apply this checkpoint configuration without requiring expert knowledge of the features configured in the checkpoint.

You can create a checkpoint copy of the current running configuration at any time. Cisco NX-OS saves this checkpoint as an ASCII file that you can use to roll back the running configuration to the checkpoint configuration at a future time. You can create multiple checkpoints to save different versions of your running configuration.

When you roll back the running configuration, you can trigger the following rollback types:

  • Atomic: Implement the rollback only if no errors occur. This is the default rollback type.

  • Best-effort: Implement a rollback and skip any errors.

  • Stop-at-first-failure: Implement a rollback that stops if an error occurs.

When you are ready to roll back to a checkpoint configuration, you can view the changes that will be applied to your current running configuration before committing to the rollback operation. If an error occurs during the rollback operation, you can choose to cancel the operation or ignore the error and proceed with the rollback. If you cancel the operation, Cisco NX-OS provides a list of changes already applied before the error occurred. You need to clean up these changes manually.

Configuration rollback limitations are as follows:

  • Allowed to create up to ten checkpoint copies per VDC.

  • You are not allowed to apply a checkpoint file of one VDC into another VDC.

  • You are not allowed to apply a checkpoint configuration in a nondefault VDC if there is a change in the global configuration portion of the running configuration compared to the checkpoint configuration.

  • The checkpoint filenames must be 75 characters or less.

  • You are not allowed to start a checkpoint filename with the word auto.

  • You cannot name a checkpoint file with summary or any abbreviation of the word summary.

  • Only one user can perform a checkpoint, rollback, or copy the running configuration to the startup configuration at the same time in a VDC.

  • After execution of write erase and reload commands, checkpoints are deleted. You can use the clear checkpoint database command to clear out all checkpoint files.

  • Rollback fails for NetFlow if during rollback you try to modify a record that is programmed in the hardware.

  • Although rollback is not supported for checkpoints across software versions, users can perform rollback at their own discretion and can use the best-effort mode to recover from errors.

  • When checkpoints are created on bootflash, differences with the running-system configuration cannot be performed before performing the rollback, and the system reports “No Changes.”

Example 1-10 demonstrates how to create a configuration rollback.


Note - You need to make sure you are in the correct VDC. If you need to change VDCs, use the switchto vdc syntax.


Example 1-10  Creating a Configuration Rollback

N7010-1# checkpoint changes...........DoneN7010-1# show diff rollback-patch checkpoint changes running-configCollecting Running-ConfigGenerating Rollback PatchRollback Patch is EmptyN7010-1# conf tEnter configuration commands, one per line. End with CNTL/Z.N7010-1(config)# no snmp-server user nxos-adminN7010-1(config)# exitN7010-1# show diff rollback-patch checkpoint changes running-configCollecting Running-ConfigGenerating Rollback Patch!!no username nxos-admin sshkey ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6+TdX+ABH/mq1gQbfhhsjBmm65ksgfQb3Mb3qbwUbNlcAa6fjJCGdHuf3kJox/hjgPDChJOd-kUXHjESlV59OhZP/NHlBrBq0TGRr+hfdAssD3wG5oPkywgM4+bR/ssCzoj6jVG41tGmfPip4pr3dqsMzR21DXSKK/tdj7bipWKy1wSkYQzZwatIVPIXRqTJY7L9a+JqVIJEA0QlJM1l0wZ5YbxccB2GKNKCM2x2BZl4okVgl80CCJg7vmn+8RqIOQ5jNAPNeb9kFw9nsPj/r5xFC1RcSKeQbdYAjItU6cX1TslRnKjlWewCgIa26dEaGdawMVuftgu0uM97VCOxZPQ==no username nxos-adminN7010-1# rollback running-config checkpoint changesNote: Applying config in parallel may fail Rollback verificationCollecting Running-ConfigGenerating Rollback PatchExecuting Rollback PatchGenerating Running-config for verificationGenerating Patch for verificationN7010-1# show snmp user nxos-admin______________________________________________________________                  SNMP USER______________________________________________________________User                          Auth  Priv(enforce) Groups____                          ____  _____________ ______nxos-admin                    sha   des(no)       network-operatorYou can also enable specific SNMP traps:N7010-1(config)# snmp-server enable traps eigrpN7010-1(config)# snmp-server enable traps callhomeN7010-1(config)# snmp-server enable traps linkN7010-1(config)# exitN7010-1#

Operating System Files

Cisco NX-OS Software consists of three images:

  • The kickstart image, contains the Linux kernel, basic drivers, and initial file system.

  • The system image contains the system software, infrastructure, Layers 4 through 7.

  • The Erasable Programmable Logic Device (EPLD) image: EPLDs are found on the Nexus 7000 currently shipping I/O modules. EPLD images are not released frequently,\; even if an EPLD image is released, the network administrator is not forced to upgrade to the new image. EPLD image upgrades for I/O modules disrupt traffic going through the I/O module. The I/O module powers down briefly during the upgrade. The EPLD image upgrades are performed one module at a time.

On the Nexus 7000 with dual-supervisor modules installed, NX-OS supports in-service software upgrades (ISSU). NX-OS ISSU upgrades are performed without disrupting data traffic. If the upgrade requires EPLD to be installed onto the line cards that causes a disruption of data traffic, the NX-OS software warns you before proceeding so that you can stop the upgrade and reschedule it to a time that minimizes the impact on your network.

NX-OS ISSU updates the following images:

  • Kickstart image

  • System image

  • Supervisor module BIOS

  • Data module image

  • Data module BIOS

  • Connectivity management processor (CMP) image

  • CMP BIOS

The ISSU process performs a certain sequence of events, as outlined here:

Step 1.

Upgrade the BIOS on the active and standby supervisor modules and the line cards (data cards/nonsupervisor modules).

Step 2.

Bring up the standby supervisor module with the new kickstart and system images.

Step 3.

Switch over from the active supervisor module to the upgraded standby supervisor module.

Step 4.

Bring up the old active supervisor module with the new kickstart image and the new system image.

Step 5.

Upgrade the CMP on both supervisor modules.

Step 6.

Perform nondisruptive image upgrade for line card (data cards/nonsupervisor modules), one at a time.

Step 7.

ISSU upgrade is complete.

Virtual Device Contexts (VDCs)

The Nexus 7000 NX-OS software supports Virtual Device Contexts (VDCs), VDC(s) allow the partitioning of a single physical Nexus 7000 device into multiple logical devices. This logical separation provides the following benefits:

  • Administrative and management separation

  • Change and failure domain isolation from other VDCs

  • Address, VLAN, VRF, and vPC isolation

Each VDC appears as a unique device and allows for separate Roles-Based Access Control Management (RBAC) per VDC. This enables VDCs to be administered by different administrators while still maintaining a rich, granular RBAC capability. With this functionalit, each administrator can define virtual routing and forwarding instance (VRF) names and VLAN IDs independent of those used in other VDCs safely with the knowledge that VDCs maintain their own unique software processes, configuration, and data-plane forwarding tables.

Each VDC also maintains an individual high-availability (HA) policy that defines the action that the system will take when a failure occurs within a VDC. Depending on the hardware configuration of the system, there are various actions that can be performed. In a single supervisor system, the VDC can be shut down, restarted, or the supervisor can be reloaded. In a redundant supervisor configuration, the VDC can be shut down, restarted, or a supervisor switchover can be initiated.


Note - Refer to Chapter 6, “High Availability,” for additional details.


There are components that are shared between VDC(s), which include the following:

  • A single instance of the kernel which supports all of the processes and VDCs.

  • Supervisor modules

  • Fabric modules

  • Power supplies

  • Fan trays

  • System fan trays

  • CMP

  • CoPP

  • Hardware SPAN resources

Figure 1-5 shows the logical segmentation with VDCs on the Nexus 7000. A common use case is horizontal consolidation to reduce the quantity of physical switches at the data center aggregation layer. In Figure 1-5, there are two physical Nexus 7000 chassis; the logical VDC layout is also shown.

Figure 1-5

Logical Segmentation with VDCs on the Nexus 7000

VDC Configuration

This section shows the required steps to creating a VDC; once the VDC is created, you will assign resources to the VDC. VDC(s) are always created from the default admin VDC context, VDC context 1.


Note - The maximum number of VDCs that can be configured per Nexus 7000 chassis is four; the default VDC (VDC 1) and three additional VDC(s).


Example 1-11 shows how to configure the VDC core on Egypt.

Example 1-11  Creating VDC “core” on Egypt

egypt(config)# vdc coreNote:  Creating VDC, one moment please ...egypt# show vdcvdc_id  vdc_name                          state               mac------  --------                          -----               ----------1       egypt                             active         00:1b:54:c2:38:c12       core                               active        00:1b:54:c2:38:c2egypt# show vdc core detailvdc id: 2vdc name: corevdc state: activevdc mac address: 00:1b:54:c2:38:c2vdc ha policy: RESTARTvdc dual-sup ha policy: SWITCHOVERvdc boot Order: 2vdc create time: Mon Feb 22 13:11:59 2010vdc reload count: 1vdc restart count: 0egypt#

Once the VDC is created, you now have to assign physical interfaces to the VDC. Depending on the Ethernet modules installed in the switch, interface allocation is supported as follows:

The 32-port 10-Gigabit Ethernet Module (N7K-M132XP-12), interfaces can be allocated on a per port-group basis; there are eight port-groups. For example, port-group 1 are interfaces e1, e3, e5, e7; port-group 2 are interfaces e2, e4, e6, e8.

The 48-port 10/100/1000 I/O Module (N7K-M148GT-11) can be allocated on a per-port basis.

The 48-port 1000BaseX I/O Module (N7K-M148GS-11) can be allocated on a per-port basis.

A future module, N7K-D132XP-15, interfaces will be allocated per 2 ports per VDC.


Note - It is not possible to virtualize a physical interface and associate the resulting logical interfaces to different VDCs. A supported configuration is to virtualize a physical interface and associate the resulting logical interfaces with different VRFs or VLANs. By default, all physical ports belong to the default VDC.


Example 1-12 demonstrates how to allocate interfaces to a VDC.

Example 1-12  Allocating Interfaces to a VDC

egypt(config)# vdc coreeqypt(config-vdc)# allocate interface Ethernet1/17egypt(config-vdc)# allocate interface Ethernet1/18

To verify the interfaces allocation, enter the show vdc membership command as demonstrated in Example 1-13.

Example 1-13  Verifying Interface Allocation to a VDC

egypt(config-vdc)# show vdc membershipvdc_id: 1 vdc_name: egypt interfaces:        Ethernet1/26          Ethernet1/28          Ethernet1/30        Ethernet1/32          Ethernet2/2           Ethernet2/4        Ethernet2/6           Ethernet2/8           Ethernet2/26        Ethernet2/28          Ethernet2/30          Ethernet2/32        Ethernet3/4           Ethernet3/5           Ethernet3/6        Ethernet3/7           Ethernet3/8           Ethernet3/9        Ethernet3/11          Ethernet3/12          Ethernet3/13        Ethernet3/14          Ethernet3/15          Ethernet3/16        Ethernet3/17          Ethernet3/18          Ethernet3/19        Ethernet3/20          Ethernet3/21          Ethernet3/22        Ethernet3/23          Ethernet3/24          Ethernet3/25        Ethernet3/26          Ethernet3/27          Ethernet3/28        Ethernet3/29          Ethernet3/30          Ethernet3/31        Ethernet3/32          Ethernet3/33          Ethernet3/34        Ethernet3/35          Ethernet3/36          Ethernet3/39        Ethernet3/40          Ethernet3/41          Ethernet3/42        Ethernet3/43          Ethernet3/44          Ethernet3/45        Ethernet3/46          Ethernet3/47          Ethernet3/48vdc_id: 2 vdc_name: core interfaces:        Ethernet1/17          Ethernet1/18          Ethernet1/19        Ethernet1/20          Ethernet1/21          Ethernet1/22        Ethernet1/23          Ethernet1/24          Ethernet1/25        Ethernet1/27          Ethernet1/29          Ethernet1/31        Ethernet2/17          Ethernet2/18          Ethernet2/19        Ethernet2/20          Ethernet2/21          Ethernet2/22        Ethernet2/23          Ethernet2/24          Ethernet2/25        Ethernet2/27          Ethernet2/29          Ethernet2/31        Ethernet3/1           Ethernet3/2           Ethernet3/3        Ethernet3/10
1 2 3 4 5 Page 4
Page 4 of 5
The 10 most powerful companies in enterprise networking 2022