Quality of Service
QoS processing is performed on the Catalyst 6500 series platform using the policy feature card (PFC) and the port application specific integrated circuit (ASIC) for layer 2 modules. This blog will focus on the LAN based Ethernet switching modules and not the WAN connectivity provided by port adaptors in the legacy flexwan module or the shared port adaptors (SPA) used in the shared port adaptor interface processors (200/400/600). QoS capabilities of WAN interface processor modules are very similar to QoS support on router platforms. All queuing functions are performed on the ASIC of the line card. The Catalyst 6500 supports a variety of line cards with a variety of different queuing architectures. Queuing on the Catalyst 6500 is performed on line cards in ASIC hardware. Scheduling, trust boundaries, and congestion management functions are also performed in the ASIC of the line card. Catalyst 6500 line cards do not perform queuing , while some have transmit queues only, and others have transmit and receive queues. Transmit queues are normally more of a choke point because one output interface may be aggregating multiple interfaces or running at a slower speed than the input of traffic. Transmit queues are assigned a much larger amount of buffer memory than receive queues on line cards that support both transmit and receive queues. Receive queues are used when the bus arbiter is busy and the Ethernet frame must be delayed before accessing the backplane of the switch. Receive queues are seldom utilized because CEF720 line cards access the switch fabric matrix of the SUP720 module at up to 40Gbps per line card. Receive side queuing is more prevalent on Catalyst 6500 switches with a SUP2 module and no switch fabric matrix (SFM) module because the backplane speed is only 32Gbps which can easily be oversubscribed. The 6748 line card supports the following queuing capabilities: Receive Side Buffer: 166KB Transmit Side Buffer: 1.2MB Transmit Queue Structure (per port): 1P3Q8T Receive Queue Structure (per port): 1Q8T Receive Queue Structure (per port with DFC): 2Q8T The receive side queue architecture changes when a distributed forwarding cards (DFC) daughter card is used on the line card. When DFC cards are on Catalyst 6500 line cards, QoS operations are transparently compiled by the multi layer switch feature card (MSFC) on the supervisor and downloaded to the line card. The DFC card is also responsible for locally routing traffic on the line card without any traffic traversing the backplane. DFC cards greatly increase the scalability of the Catalyst 6500, but can be quite expensive. Policing and classification QoS operations are performed in hardware on the policy feature card on the SUP720, not the line card ASIC. All SUP720 PFC variants support ingress (input) and egress (output) policing, while the SUP2 module’s PFC only supported ingress policing capabilities. Network based application recognition (NBAR) is supported on both the SUP2 and SUP720 modules, but NBAR capabilities are processed in software on the MSFC module. NBAR can be very processor intensive and it is advisable to NOT use NBAR capabilities on the Catalyst 6500. The “protocol” keyword in pmap-class configuration mode will utilize NBAR capabilities. The SUP32-PISA support NBAR processing in hardware, although the throughput support is limited to only 1Gbps. The majority of Catalyst 6500 line cards use the weighted round robin (WRR) congestion management algorithm that has been explained in previous switch QoS blogs. The uplink ports on the SUP32 modules use the shaped round robin (SRR) algorithm that was explained in the 2960/2970/3560/3750 switch QoS blog posts. Newer Catalyst 6500 series line cards use the deficit weighted round robin (DWRR) congestion management algorithm that is very similar to the modified deficit round robin (MDRR) congestion management algorithm originally used on the 12000 series router platforms. DWRR overcomes a scheduling limitation of WRR that allows DWRR to more closely align to the administrator’s configured policy which is explained in the paragraph below. Congestion management algorithms utilize an internal byte count based on the Cisco IOS queue weighting configuration. Imagine that a queue configuration results in a byte count of 1600 bytes and (2) 1500 byte frames are waiting to be processed in the queue. The scheduling algorithm processes the first frame in the queue and updates the byte count parameter resulting in 100 bytes remaining (1600 – 1500). As long as there is one or more byte count credits remaining, the next frame of 1500 bytes must be processed resulting in a queue deficit counter of -1400 (100 – 1500). WRR and SRR do not maintain deficit counters and 1600 bytes of credits will be added to the queue scheduling in the next time interval of scheduling the queue. The lack of a deficit counter in WRR and SRR may result in a queue being serviced more than intended. DWRR fixes this issue by maintaining the deficit counter of -1400 next time the queue is scheduled. DWRR would only allocate a byte count of 200 bytes (-1400 + 1600) the second time the queue is scheduled. DWRR will result in a policy that is very close to that configured by the administrator over time. WRR and SRR may result in a policy that is quite different than that configured. The Catalyst 6500 queuing discussed here is occurring over ethernet based interfaces running at speeds of 10Mbps, 100Mbps, 1Gbps, and 10Gbps. A queue resulting in 35% of gigabit ethernet interface bandwidth instead of 30% will probably not be very detrimental to the design of your network. Some Catalyst 6500 line cards do not have any congestion avoidance algorithm capabilities “0T”, while others support the WRED or WTD algorithms discussed in earlier blogs. The Catalyst 6500 switch configuration should also be tuned to follow best practices for CoS to queue mappings which is highly dependent on the queuing of the individual line cards. The references section of this document includes links to documents at Cisco.com that cover the queuing capabilities of each 6500 line card and the recommended queue mappings. The QoS SRND available at www.cisco.com/go/srnd also has many boiler plate configurations for different Catalyst 6500 line cards. Trust Boundaries (IP Prec, DSCP, CoS) may also be configured on the Catalyst 6500 platform. REFERENCES Understanding Quality of Service on the Catalyst 6500 Switch http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11_538840.html Catalyst 6500/6000 QoS FAQ http://www.cisco.com/en/US/products/hw/switches/ps708/products_qanda_item09186a00804d2e3a.shtml Configuring PFC QoS – Sup32 PISA 12.2 ZY Configuration Guide http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2ZY/configuration/guide/qos.html Native IOS – Configuring PFC QoS – Catalyst 6500 Release 12.2SXH and Later Software Configuration Guide http://www.cisco.com/en/US/partner/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/qos.html CatOS – Configuring QoS – Catalyst 6500 Series Software Configuration Guide, 8.7 http://www.cisco.com/en/US/partner/docs/switches/lan/catalyst6500/catos/8.x/configuration/guide/qos.html Native IOS QoS Command Reference http://www.cisco.com/en/US/partner/docs/ios/qos/command/reference/qos_book.html
