Chapter 5: Firewall Load Balancing

Cisco Press

1 2 3 Page 2
Page 2 of 3

Manageability

Firewall rule and policy changes have always been a difficult task that requires careful planning and change window allocation. This is because a firewall provides security between the public and secure segment, and an improper change can completely block user access or create a bypass of security. The same difficulty also applies to a firewall software upgrade.

FWLB solves this problem and enables easy management of the devices. Since the firewalls are being load balanced in a group, an individual firewall can be taken out of rotation for management purposes. This ensures that the user traffic is not interrupted by firewall changes.

Firewalls can be taken out of rotation by removing them from the firewall farm configured on the CSM or by blocking the probe sent by the CSM. By blocking or dropping the probe packets, the firewall is taken out of rotation dynamically by the CSM.

Types of Firewalls

As we discussed before, firewalls are physical devices that enforce security policies between network segments. There are several types of firewalls, namely:

  • Packet-based firewalls

  • Application-based firewalls

  • Application gateway or proxy firewalls

  • Layer 2 or stealth firewalls

Packet-Based Firewalls

Packet-based firewalls are first-generation devices that enforce policies on per-packet bases. They do not have any concept of session and have no application layer understanding. A simple form of packet-based firewall is an access control list (ACL) on a router. The ACL blocks or permits IP packets based on source or destination IP addresses or both. There are more complex ACLs available now on routers that can provide stateful inspection.

Load balancing of packet-based firewalls is simple because session state is not maintained on a firewall, so potentially the same TCP connection can be load balanced on a per-packet basis across multiple firewalls. In other words, asymmetric flows are allowed in this environment.

Figure 5-1 shows an example of load–balancing packet-based firewalls. Since these firewalls are stateless, a TCP SYN for a particular connection can go through the first firewall while the TCP SYN ACK from the server to the client goes through the the second firewall.

Figure 501

Figure 5-1

Load-Balancing, Packet-Based Firewalls

Application-Based Firewalls

Application-based firewalls that are commonly available now can provide stateful inspection of sessions along with network address translation (NAT) functionality. The PIX and Firewall Services Module (FWSM) on the Catalyst 650000 are application-based firewalls. Similar to hardware-based load balancers, application-based firewalls install permitted session information in hardware memory. These firewalls do not allow any packets through them other than those that belong to the inspected and installed sessions in the firewall.

Load balancing of application-based firewalls is complex, as it requires a particular user session to be sent through the same firewall. In other words, if a client TCP SYN is sent through firewall A to reach the server, the TCP SYN ACK from the server to the client should go through firewall A also. Path symmetry needs to be maintained, and this is typically done by using a source or destination address. A hash of the source and destination address is calculated to maintain the symmetry for the client TCP SYN and server TCP SYN ACK response. We will discuss this further in the case study.

Figure 5-2 shows an example of load balancing application-based stateful firewalls. Notice that a particular TCP connection needs to go through the same firewall in this case.

Figure 5-2

Figure 5-2

Load-Balancing Application-Based Firewalls

Application Gateway or Proxy Firewalls

Application gateway–based firewalls provide stateful inspection of sessions. They are not only session aware, but are also familiar with the application protocols that create sessions; for example, HTTP, SMTP, and FTP. The application-level gateway does more than just filter packets and sessions. It acts as a gateway between the protected network and the Internet. As a gateway, it does not allow any packets to travel directly between the Internet and the protected network.

Application gateway firewalls proxy all user requests and fetch the data by initiating a new connection to the end server. The end server sees the request coming from the firewall's IP address. The firewall splices the client-side and server-side TCP connection such that the client gets the data from the end server without the Internet-based end server having knowledge of the client's IP address. This functionality looks similar to NAT but is really above and beyond because the firewall is not just hiding the internal client's IP address; it is also implementing application-based security policies.

Application gateway firewalls can be implemented transparently, but most are implemented in a proxy fashion. This is why they are also referred to as proxy firewalls.

Load-balancing proxy firewalls require the load balancing of sessions from the internal network to the Internet only. From the Internet to the internal network, packets always make it to the correct firewall, as the destination IP is the firewall's physical IP. Thus, dynamic symmetry of sessions is ensured.

Figure 5-3 shows an example of load-balancing proxy firewalls. Notice, in this case, the firewall proxies the client requests and then sets up a separate connection to the end server. In this case, a client/server session is load balanced through the same firewall. The firewall proxies the client connection and issues a connection to the Internet server. The client sends the initial request to a VIP on the load balancer, and the request is translated to the firewall IP on VLAN 3.

Figure 5-3

Figure 5-3

Load-Balancing Proxy Firewalls

Layer 2 or Stealth Firewalls

Layer 2 or stealth firewalls are also known as "bump on the wire" firewalls. These firewalls do not have any IP addresses, and they simply bridge traffic between two segments or VLANs. Both segments or VLANs share the same IP subnet and broadcast domain. In other words, these are undetectable Layer 2 devices that can perform Layer 2 forwarding together with packet and session inspection.

Load balancing stealth firewalls is fairly complex but is supported by the Cisco CSM module. For load-balancing stealth firewalls, the firewalls are connected to unique VLANs on either side—that is, the public or private side. The public-side VLANs connect to a particular CSM pair, and the private or internal VLANs connect to another CSM pair. The firewall bridges traffic between the private and public VLANs. That means, even though they are different VLANs, they are the same broadcast domains. CSMs are configured with alias IP addresses on all VLANs, and they balance traffic across the firewalls by forwarding packets to the remote CSM's alias IP addresses. In other words, the firewall server farm consists of the alias IP addresses on the remote CSM pairs. A stealth firewall is configured so that all traffic moving in both directions across that VLAN moves through the firewall.

Figure 5-4 shows an example of load balancing Layer 2 or stealth firewalls. Notice each Layer 2 firewall exists on a separate VLAN pair and subnet. A VLAN pair (for example, VLAN 2 and VLAN 12) is used to merge a broadcast domain, servicing subnet 10.1.2.0/24.

Figure 5-4

Figure 5-4

Load-Balancing Stealth Firewalls

Case Study: Firewall Load Balancing

Now that you understand the different types of firewalls and the motivations behind FWLB, you can learn how to deploy a fairly complex FWLB solution over multiple secure segments. When deploying FWLB, it's critical to learn about the applications flows that would be load balanced to the firewall, whether they are outbound to the Internet or inbound to the DMZ. Applications that require multiple connections, also called buddy connections, such as FTP, need to be considered and accounted for in the design. For these applications, you need to ensure that both the connections belonging to the same user session go through the same firewall.

In this case study, we will examine a specific customer deployment where CSM is used to load balance firewalls with three secure segments, namely an INET, a DMZ, and a LAN segment. The idea behind the following case study is to understand not only the concepts but the implementation details. To fully understand the sample deployment, we will look at:

  • Server and application requirements

  • Security requirements

  • Infrastructure requirements

  • Design options

  • Test and verification

Server and Application Requirements

Server and application requirements for this case study are defined as follows:

  • The servers should be directly managed in the DMZ from management stations in the LAN segment.

  • Servers should also be able to initiate sessions for application software patches and driver upgrades.

  • The number of servers in the DMZ is about 300.

  • The primary application that is being load balanced in the DMZ is HTTP and HTTPS based.

  • Persistence is needed for the HTTPS-based application.

  • TCP-based probes are OK.

  • Direct access to the DMZ servers from the LAN is made using long-lived flows a minimum of three hours of idle timeout.

  • FTP is used between LAN and DMZ segment servers.

Security Requirements

The security requirements for this case study are as follows:

  • The main requirement is to load balance flows to the firewalls from all the three segments.

  • It is critical that each network path through the firewall be verified before sending connections over it; that is, a LAN's user traffic should only be sent toward the DMZ over a firewall that has its DMZ segment interface functional.

  • FTP and other applications with multiconnections should be sent across the same firewall.

  • High availability is critical.

  • Firewalls do not share state tables; that is, each firewall is unaware of the session states of the other firewalls.

Infrastructure Requirements

Layer 2 and Layer 3 infrastructure requirements for this case study are as follows:

  • Minimal disruption to the Interior Gateway Protocol (IGP) domain

  • Seamless integration of the CSM in the current Layer 2 and Layer 3 infrastructure

  • Robust failover necessary in case of firewall failure

In the data center design, to meet the previously mentioned requirements, the CSMs are configured in a combination of bridge and router modes. Figure 5-5 shows the physical topology being deployed.

FWLB Design Considerations

Following are some of the key aspects of the design that you need to consider for a successful deployment:

  • Use appropriate ICMP probes to track all links of the firewalls. One can easily drop traffic if all paths are not watched.

  • Configure the port-channel between the Catalysts for the server and client VLANs. This port-channel should exclude the CSM fault-tolerant (FT) VLAN.

  • Configure another port-channel with at least two Fast Ethernet interfaces for the CSM FT VLAN.

  • The Multilayer Switching Feature Card (MSFC) should only have Layer 3 presence on either the client- or server-side VLAN of the CSM. As discussed in Chapter 3, this restriction can be removed by using policy-based routing (PBR) on the MSFC.

Figure 5-5

Figure 5-5

Physical Topology of the Multiple-Segment FWLB Design

  • MSFC should route relevant traffic to the alias IP of the CSMs.

  • CSMs should route relevant traffic to the HSRP group IP of the MSFC.

  • Distribute the firewalls and the servers across both the Catalyst 6500s. Must not have single point of failure. Measures, such as port-channeling, are required to avoid split brains, which occurs when both devices in an active/standby pair become active.

  • In FWLB, virtual IPs (VIPs) within the virtual server configuration act as the routes. Make sure proper subnetting and so on are used. It is very easy to create routing loops.

  • The VLAN X option in the virtual server configuration is highly significant. This is to ensure that connections from appropriate VLANs hit the VIP address. In other words, this provides you with more control in enhancing security and in preventing routing loops.

Figure 5-6 shows the logical topology of the FWLB design in consideration.

Figure 5-6

Figure 5-6

Logical Topology of the Multiple-Segment FWLB Design

FWLB Probes

In order to ensure that the path through a firewall is available, ICMP probes are used. ICMP probes are configured with the address command within the probe to define which address needs to be pinged. The address keyword enables the CSM to send the probe to the defined address and not to the real IP (the firewall's IP). CSM uses the real server's MAC address to send the probe across each firewall. In the following example, the address is the alias IP address of the CSMs in the DMZ segment. The CSM makes sure it sends the ICMP echo across all the links of the firewalls. The receiving CSM sends back the response on the link on which the request was received. This way, all the paths are monitored by the CSM.

Within the probe configurations, interval, failed, and retries values can be adjusted to achieve the desired effect. For example, a probe with an interval value of 5, a retries value of 3, and a failed value of 30 sends a probe every 3 seconds, and a failure of 3 consecutive probes results in the server being marked as failed or down. So it takes 15 seconds to mark a server down. The probes are sent again after a delay of 30 seconds to see if the server is back in service or not.

Cat6500_01(config-module-csm)#probe DMZ icmp 
Cat6500_01(config-slb-probe-icmp)#?
SLB ICMP probe config
 address  probe destination IP address
 default  Set a command to its defaults
 exit   exit probe submode
 failed  time in seconds between probes of failed server
 interval time in seconds between probes
 no    Negate a command or set its defaults
 receive  maximum time in seconds to wait for a reply from real server
 retries  number of consecutive errors before marking real server failure
Cat6500_01(config-slb-probe-icmp)#
!
 probe DMZ icmp
 address 11.8.200.65 
 interval 5 
  
!
serverfarm OUT_DMZ
 no nat server 
 no nat client
 real 11.8.200.43
  inservice
 real 11.8.200.44
  inservice
 real 11.8.200.45
  inservice
 probe DMZ
!

Traffic to the Firewalls

To load balance traffic across the firewalls, create a server farm with the IP addresses of the firewalls. Firewalls must be Layer 2-adjacent to the CSM. MSFC must not have a Layer 3 interface on the firewall VLAN. To route traffic across the firewalls, the user needs to create a virtual server with a VIP that defines the route through the firewall. Traffic destined to the specified VIP would be load balanced across the firewall group. Notice that in the following configuration example, a VLAN tag 10 is configured on the virtual server DMZ. This configuration command ensures that only traffic from VLAN 10 (client VLAN) should be considered by the virtual server. This is used for security purposes and also to prevent routing loops.

!
serverfarm OUT_DMZ
 no nat server 
 no nat client
 real 11.8.200.43
  inservice
 real 11.8.200.44
  inservice
 real 11.8.200.45
  inservice
 probe DMZ
!
 vserver DMZ
 virtual 173.73.248.0 255.255.248.0 any
 vlan 10
 serverfarm OUT_DMZ
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!

The replicate csrp sticky command is not needed in the above configuration, as there is no sticky group configured. This command is used to synchronize sticky data tables between active and standby CSMs.

Traffic from the Firewalls

CSM is not a router. It would not know what to do with any IP packet unless configured using virtual servers or static routes and gateways. We need the following configuration to make the CSM handle the traffic originated from the firewall VLAN (12) appropriately. This is a catch-all virtual server that says the following: predictor-forward any packet that comes in from VLAN 12 for which I do not have a flow. Predictor-forward means route as defined; in this case, it's the default gateway. The any key word in the virtual configuration refers to any IP protocol.

!
 serverfarm OUT_FW_CSM
 no nat server 
 no nat client
 predictor forward
!
vserver OUT_FW_CSM
 virtual 0.0.0.0 0.0.0.0 any
 vlan 12
 serverfarm OUT_FW_CSM
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!

Router or Secure Mode

Router mode (also known as Secure mode) is configured on the CSM by having two different subnets on the client and server VLANs respectively. Direct access to the servers from the client VLAN is not allowed in Router mode. An example can be seen on CSM1 and CSM2. Router mode is used in the Internet segment for security enhancement. Similarly, Bridge mode would have worked as well.

module ContentSwitchingModule 2 
 vlan 10 client
 ip address 11.8.200.226 255.255.255.224
 gateway 11.8.200.230
 alias 11.8.200.225 255.255.255.224
!
 vlan 12 server
 ip address 11.8.200.34 255.255.255.224
 alias 11.8.200.33 255.255.255.224
!

The configuration shows that alias IP 11.8.200.33 is used by the PIXs as their upstream default gateway.

Bridge Mode

Bridge mode is configured on the CSM by having the same subnet and IP address on both the client and server VLANs. Examples can be seen on both the DMZ CSM and the LAN CSM. Bridge mode enables broadcast traffic to seamlessly pass through the CSM and also allows direct access of the servers from the front end.

module ContentSwitchingModule 2 
 vlan 14 client
 ip address 11.8.200.66 255.255.255.224
 alias 11.8.200.65 255.255.255.224
!
 vlan 114 server
 ip address 11.8.200.66 255.255.255.224
 route 173.73.248.0 255.255.248.0 gateway 11.8.200.80
!

FWLB Algorithms

Any load-balancing method available on the CSM can be used for FWLB. Typically, the default balancing method, round robin, is used. Since support of multiconnection protocols, such as FTP, is required in this design, we will be using predictor IP hash in our configuration. For multiconnection protocols where some connections are open by clients and others by servers and you need to make sure all the connections belonging to the same session go through the same firewall, you will have to use source IP hash for incoming connection and destination IP hash for outgoing, or vice versa.

Cat6500_01(config-module-csm)#serverfarm OUT_DMZ
Cat6500_01(config-slb-sfarm)#pre
Cat6500_01(config-slb-sfarm)#predictor ?
 forward   forwarding based on destination lookup
 hash    hashing algorithms
 ip-hash   Source IP address hash algorithm (Please use hash address source)
 leastconns least connections algorithm
 roundrobin roundrobin algorithm (default)
Cat6500_01(config-slb-sfarm)#predictor ha
Cat6500_01(config-slb-sfarm)#predictor hash ?
 address IP source/dest address hash algorithm
 url   URL hash algorithm
Cat6500_01(config-slb-sfarm)#predictor hash add
Cat6500_01(config-slb-sfarm)#predictor hash address ?
 /nn or A.B.C.D IP address hash network mask
 destination   Destination IP address hash algorithm
 source     Source IP address hash algorithm
 <cr>
Cat6500_01(config-slb-sfarm)#predictor hash address

Configuration Details of the INET Segment

Following are detailed CSM and Catalyst 6509 configurations of the INET segment.

CSM Configurations

Following is the complete CSM configuration, which is used in the INET segment. This configuration shows the VLAN configuration, probes, server farms, and virtual servers.

module ContentSwitchingModule 2 
 vlan 10 client
 ip address 11.8.200.226 255.255.255.224
 gateway 11.8.200.230
 alias 11.8.200.225 255.255.255.224
!
 vlan 12 server
 ip address 11.8.200.34 255.255.255.224
 alias 11.8.200.33 255.255.255.224
!
 probe DMZ icmp
 address 11.8.200.65 
 interval 5 
!
 probe LAN icmp
 address 11.8.200.97 
 interval 5 
!
 serverfarm OUT_DMZ
 no nat server 
 no nat client
 predictor hash address source
 real 11.8.200.43
  inservice
 real 11.8.200.44
  inservice
 real 11.8.200.45
  inservice
 probe DMZ
!
serverfarm OUT_LAN
 no nat server 
 no nat client
 predictor hash address source
 real 11.8.200.43
  inservice
 real 11.8.200.44
  inservice
 real 11.8.200.45
  inservice
 probe LAN
!
 serverfarm OUT_FW_CSM
 no nat server 
 no nat client
 predictor forward
!
 vserver DMZ
 virtual 173.73.248.0 255.255.248.0 any
 vlan 10
 serverfarm OUT_DMZ
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!
 vserver DMZ_V2
 virtual 11.8.200.64 255.255.255.224 any
 vlan 10
 serverfarm OUT_DMZ
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!
 vserver LAN
 virtual 11.0.0.0 255.0.0.0 any
 vlan 10
 serverfarm OUT_LAN
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!
 vserver OUT_FW_CSM
 virtual 0.0.0.0 0.0.0.0 any
 vlan 12
 serverfarm OUT_FW_CSM
 replicate csrp sticky
 replicate csrp connection
 persistent rebalance
 inservice
!

Catalyst 6509 Layer 3 Configurations

Following are the Layer 3 configurations on the Catalyst 6509. The interface VLAN 10 is the one that links the CSM with the MSFC. VLAN 11 connects the MSFC with the edge router.

!
interface Vlan10
 ip address 11.8.200.231 255.255.255.224
 no ip redirects
 standby 1 ip 11.8.200.230
 standby 1 priority 105
 standby 1 preempt
 standby 1 track Vlan11
!
interface Vlan11
 ip address 173.73.245.6 255.255.255.0
 no ip redirects
 standby 2 ip 173.73.245.10
 standby 2 priority 105
 standby 2 preempt
 standby 2 track GigabitEthernet1/1
 standby 2 track Vlan10
!
ip default-gateway 173.73.245.1
!
ip classless
ip route 0.0.0.0 0.0.0.0 173.73.245.1
ip route 11.0.0.0 255.0.0.0 11.8.200.225
ip route 173.73.0.0 255.255.0.0 11.8.200.225

Configuration Details of the DMZ Segment

Following are detailed CSM and Catalyst 6500 configurations of the DMZ segment.

CSM Configurations

Following is the CSM configuration used in the DMZ segment. Notice in this segment that the CSM performs dual tasks. It load balances server-originated traffic to the LAN and INET segment across the firewalls and also load balances inbound HTTP and HTTPS requests across the application server farm.

Related:
1 2 3 Page 2
Page 2 of 3
SD-WAN buyers guide: Key questions to ask vendors (and yourself)