Archive

Posts Tagged ‘HP’

Configuring TMG NLB Array with HP Network Configuration Utility on Cisco 4506

June 29, 2012 Leave a comment

Recently we migrated our edge Forefront TMG standard machine to a Forefront TMG Enterprise standalone array to create redundancy for incoming traffic (NLB) and outgoing traffic (ISP-R). We bought 2 HP DL360 G7 servers with 24GB Mem, 4 x 300 SAS disks, 2 x Quad core CPU’s to support 3500 users (in theory).  Little information was to be found on how to configure the NLB configuration if you had 2 cisco 4506 core switches. So I thought I ‘ll write it down. The setup looks like this:

Port-channel 1 is used between the 2 core switches and all vlan are allowed on this trunk. Port-channel 2 is configured between the 4506 and the TMG-FE-x server. VLAN 10 and 11 are used for internal traffic (lets say student and teacher traffic). VLAN 50 is used for the DMZ subnet (webserver running here). VLAN 100 is the internet connection from ISP1 and VLAN 101 is the internet connection from ISP2. We are going to configure NLB for VLAN 10, 11, 50 and 100. We choose not to use it for ISP 2 (VLAN 101). VLAN 99 is used for the Intra-Array adapter between the TMG’s.

Start off by running the HP Network Configuration utility on TMG-FE-1 and TMG-FE-2 and configure it like this:

Define VLAN 10 and 11 on TMG-FE-1 teaming interface (configured on port-channel 2)

Define VLAN 10 and 11 on TMG-FE-2 teaming interface (configured on port-channel 2)

The IP addresses for the TMG-FE-1 are: VLAN 50 = 10.10.50.2, VLAN 100 = 100.100.100.2, VLAN 101 = 101.101.101.2.

The IP addresses for the TMG-FE-2 are: VLAN 50 = 10.10.50.3, VLAN 100 = 100.100.100.3, VLAN 101 = 101.101.101.3.

Now its time to config the first Core switch (left one):

interface Port-channel1
description 20GB connection to other Core
switchport
switchport mode trunk
switchport nonegotiate

interface Port-channel2
description 2GB connection to TMG-FE-1
switchport
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk

interface TenGigabitEthernet1/1
description Uplink to other Core 20Gb Channel
switchport mode trunk
switchport nonegotiate
logging event link-status
udld port aggressive
flowcontrol receive off
channel-group 1 mode on
!
interface TenGigabitEthernet1/2
description Uplink to other Core 20Gb Channel
switchport mode trunk
switchport nonegotiate
logging event link-status
udld port aggressive
flowcontrol receive off
channel-group 1 mode on

interface GigabitEthernet5/1
description Connection to TMG-FE-1
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
logging event link-status
channel-group 2 mode on
spanning-tree portfast trunk

interface GigabitEthernet5/11
description Connection from TMG-FE-1 to ISP2
switchport access vlan 101
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet5/13
description Connection to TMG-FE-1 (Array NIC)
switchport access vlan 99
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet6/1
description Connection to TMG-FE-1
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
logging event link-status
channel-group 2 mode on
spanning-tree portfast trunk

interface GigabitEthernet6/10
description Connection from TMG-FE-1 to ISP1
switchport access vlan 100
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet6/12
description Connection to TMG-FE-1 (DMZ NIC)
switchport access vlan 50
switchport mode access
switchport nonegotiate
spanning-tree portfast

Config for the second core:

interface Port-channel1
description 20GB connection to other Core
switchport
switchport mode trunk
switchport nonegotiate

interface Port-channel2
description 2GB connection to TMG-FE-2
switchport
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk

interface TenGigabitEthernet1/1
description Uplink to other Core 20Gb Channel
switchport mode trunk
switchport nonegotiate
logging event link-status
udld port aggressive
flowcontrol receive off
channel-group 1 mode on
!
interface TenGigabitEthernet1/2
description Uplink to other Core 20Gb Channel
switchport mode trunk
switchport nonegotiate
logging event link-status
udld port aggressive
flowcontrol receive off
channel-group 1 mode on

interface GigabitEthernet5/1
description Connection to TMG-FE-2
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
logging event link-status
channel-group 2 mode on
spanning-tree portfast trunk

interface GigabitEthernet5/11
description Connection from TMG-FE-2 to ISP2
switchport access vlan 101
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet5/13
description Connection to TMG-FE-2 (Array NIC)
switchport access vlan 99
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet6/1
description Connection to TMG-FE-2
switchport trunk allowed vlan 10,11
switchport mode trunk
switchport nonegotiate
logging event link-status
channel-group 2 mode on
spanning-tree portfast trunk

interface GigabitEthernet6/10
description Connection from TMG-FE-2 to ISP1
switchport access vlan 100
switchport mode access
switchport nonegotiate
spanning-tree portfast

interface GigabitEthernet6/12
description Connection to TMG-FE-2 (DMZ NIC)
switchport access vlan 50
switchport mode access
switchport nonegotiate
spanning-tree portfast

If you finished installing & configuring Forefront (not included here! maybe in another post one day) on both machines (make sure you have networks configured for VLAN 10, 11 and 50) then its time to config the NLB’s. Select the VLAN 10 network and choose Configure Load Balanced networks. Select the appropriate (VLAN 10 network) and select configure NLB. Now configure your primary NLB VIP address : 10.10.10.1. Choose MULTICAST. Repeat the same action for VLAN 11 : IP address 10.10.11.1, for VLAN 50 : IP address 10.10.50.1 and for VLAN 100 : IP address 100.100.100.1 (if you need more VIP’s for extra services like OWA, MAIL then enter them here)

Note: We ran into problems when enabling MULTICAST NLB for VLAN 100. It worked when we placed an workstation in VLAN 100 but it did not work from any other external subnet (different ISPs). I think it can be solved by asking your ISP to configure static ARP entries on their router that point to your VLAN 100 NLB VIP’s. We choose to configure VLAN 100 as an UNICAST NLB. That worked allright.

Still we need one thing to do on our Cisco 4506 Core switches : creating static ARP entries to avoid flooding on the Cores. So we need to find out the Multicast MAC Addresses and Unicast MAC Addresses of our VIP’s. Start a Dos Box on the TMG server and run : nlb ip2mac 10.10.10.1, nlb ip2mac 10.10.11.1, nlb ip2mac 10.10.50.1, nlb ip2mac 100.100.100.1. a Multicast MAC Address will start with 03:bf and a Unicast MAC Address will start with 02:bf. If you are good at converting decimal values to hexadecimal values you can convert them yourself fairly easy. Decimal 10 = hexadecimal 0a, decimal 100 = hexadecimal 64. etc etc. Our VIPs will look like:

IP 10.10.10.1 = MAC 03:bf:0a:0a:0a:01
IP 10.10.11.1 = MAC 03:bf:0a:0a:0b:01
IP 10.10.50.1 = MAC 03:bf:0a:0a:32:01
IP 100.100.100.1 = MAC 02:bf:64:64:64:01

Configure the 4506 core switches with static arp entries (only needed for the Multicast VIP’s):

arp 10.10.10.1 03bf.0a0a.0a01 ARPA
arp 10.10.11.1 03bf.0a0a.0b01 ARPA
arp 10.10.50.1 03bf.0a0a.3201 ARPA

To avoid flooding traffic on all ports we must tweak it some more (only on ports that are needed):

mac address-table static 03bf.0a0a.0a01 vlan 10 interface Po1 Po2
mac address-table static 03bf.0a0a.0b01 vlan 11 interface Po1 Po2
mac address-table static 03bf.0a0a.3201 vlan 50 interface Po1 Gi6/12

Now you are done. Hope it was helpfull somehow and feel free to comment.

%d bloggers like this: