knowledge base


CISCO Nexus 5548U Core Switch



sh ip traffic sh port-channel summary sh port-channel database sh port-channel traffic sh lacp interface ethernet 101/1/45 sh lacp counters sh lacp interface sh running-config sh lldp neig
Cisco Server Link Aggregation
conf t int Eth101/1/45-48 description to Oracle DBServer (Bond0) channel-group 11 mode active (LACP with Server) end core-N5K# sh port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 10 Po10(SU) Eth NONE Eth1/23(P) Eth1/24(P) 11 Po11(SU) Eth LACP Eth101/1/45(I) Eth101/1/46(I) Eth101/1/47(I) Eth101/1/48(P) 101 Po101(SU) Eth NONE Eth1/29(P) Eth1/30(P) Eth1/31(P) Eth1/32(P) 102 Po102(SU) Eth NONE Eth1/25(P) Eth1/26(P) Eth1/27(P) Eth1/28(P) Delete Port-Channel Group core-N5K(config-if)# no interface port-channel1 core-N5K(config)# sh port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 10 Po10(SU) Eth NONE Eth1/23(P) Eth1/24(P) 11 Po11(SD) Eth NONE -- 12 Po12(SD) Eth NONE -- 101 Po101(SU) Eth NONE Eth1/29(P) Eth1/30(P) Eth1/31(P) Eth1/32(P) 102 Po102(SU) Eth NONE Eth1/25(P) Eth1/26(P) Eth1/27(P) Eth1/28(P) core-N5K(config)# no interface port-channel11 core-N5K(config)# no interface port-channel12 core-N5K(config)# sh port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 10 Po10(SU) Eth NONE Eth1/23(P) Eth1/24(P) 101 Po101(SU) Eth NONE Eth1/29(P) Eth1/30(P) Eth1/31(P) Eth1/32(P) 102 Po102(SU) Eth NONE Eth1/25(P) Eth1/26(P) Eth1/27(P) Eth1/28(P) ----------------------------------------------------------------------------------- options bonding miimon=100 mode=4 ad_select=bandwidth ? Bear in mind that with 802.3ad all of the links in your bond need to be connected to the same switch, unless the switch supports sharing the 802.3ad information with other switches through a stack/virtual switch type configuration. That said, you could still achieve your configuration with 4 nics and two switches by just making a bond to each switch, for instance: NIC1/port1 -> bond0 -> switch1 NIC1/port2 -> bond1 -> switch2 NIC2/port1 -> bond0 -> switch1 NIC2/port2 -> bond1 -> switch2 That way you get redundancy against a single NIC or switch failure, and increased bandwidth while the switches are working, but you'll have to manage the IP failover yourself. Alternatively you could switch to bonding mode 6, drop the requirement for 802.3ad and run the configuration you suggested in your question. ------------------------------------------------------------------------ [root@oradb1 network-scripts]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 1 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable Active Aggregator Info: Aggregator ID: 3 Number of ports: 1 Actor Key: 17 Partner Key: 1 Partner Mac Address: 00:00:00:00:00:00 ------------------------------------------------------------------------ /etc/sysconfig/network-scripts -> mode=4 (802.3ad) [root@oradb1 network-scripts]# cat ifcfg-Bündel-Verbindung_1 DEVICE=bond0 BONDING_OPTS="miimon=1 updelay=0 downdelay=0 mode=802.3ad" TYPE=Bond BONDING_MASTER=yes BOOTPROTO=none IPADDR=10.9.9.90 PREFIX=21 GATEWAY=10.9.9.7 DNS1=10.9.9.21 DNS2=10.9.9.72 DEFROUTE=yes PEERDNS=yes PEERROUTES=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_FAILURE_FATAL=no IPV6_PRIVACY=no NAME="Bündel-Verbindung 1" UUID=9debfb6c-f4f4-4ba7-8343-52525f3bb789 ONBOOT=yes ------------------------------------------------------------------------ [root@oradb1 ~]# ifconfig bond0 bond0: flags=5187 mtu 1500 inet 10.9.9.90 netmask 255.255.248.0 broadcast 10.9.15.255 inet6 fe80::210:e0ff:fe94:be15 prefixlen 64 scopeid 0x20 ether 00:10:e0:94:be:15 txqueuelen 0 (Ethernet) RX packets 842577 bytes 74518341 (71.0 MiB) RX errors 0 dropped 24327 overruns 0 frame 0 TX packets 25955 bytes 4254788 (4.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ------------------------------------------------------------------------ [root@oradb1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0_Sklave_1 HWADDR=00:10:E0:94:BE:14 TYPE=Ethernet NAME="bond0 Sklave 1" UUID=4b131059-8338-493d-a00e-ae6c96c2a0be DEVICE=eno1 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@oradb1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0_Sklave_2 HWADDR=00:10:E0:94:BE:15 TYPE=Ethernet NAME="bond0 Sklave 2" UUID=b55ea7fb-c29a-4c65-812c-8906e89207b0 DEVICE=eno2 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@oradb1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0_Sklave_3 HWADDR=00:10:E0:94:BE:16 TYPE=Ethernet NAME="bond0 Sklave 3" UUID=f16cc4d2-bc50-4c4c-b4a3-d4cda28ffeba DEVICE=eno3 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@oradb1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0_Sklave_4 HWADDR=00:10:E0:94:BE:17 TYPE=Ethernet NAME="bond0 Sklave 4" UUID=bb2547e2-49c7-42a2-b137-54cbd2a916fb DEVICE=eno4 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@oradb1 ~]# ------------------------------------------------------------------------- [root@oradb1 ~]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 1 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable Active Aggregator Info: Aggregator ID: 3 Number of ports: 1 Actor Key: 17 Partner Key: 1 Partner Mac Address: 00:00:00:00:00:00 Slave Interface: eno2 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: 00:10:e0:94:be:15 Aggregator ID: 1 Slave queue ID: 0 Slave Interface: eno4 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: 00:10:e0:94:be:17 Aggregator ID: 2 Slave queue ID: 0 Slave Interface: eno1 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:10:e0:94:be:14 Aggregator ID: 3 Slave queue ID: 0 Slave Interface: eno3 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: 00:10:e0:94:be:16 Aggregator ID: 4 Slave queue ID: 0 ------------------------------------------------------------------ [root@oradb1 ~]# ifconfig -a bond0: flags=5187 mtu 1500 inet 10.9.9.90 netmask 255.255.248.0 broadcast 10.9.15.255 inet6 fe80::210:e0ff:fe94:be15 prefixlen 64 scopeid 0x20 ether 00:10:e0:94:be:15 txqueuelen 0 (Ethernet) RX packets 851599 bytes 75211754 (71.7 MiB) RX errors 0 dropped 24550 overruns 0 frame 0 TX packets 26210 bytes 4301419 (4.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno1: flags=6211 mtu 1500 ether 00:10:e0:94:be:15 txqueuelen 1000 (Ethernet) RX packets 851600 bytes 75211837 (71.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 26210 bytes 4301419 (4.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno2: flags=6147 mtu 1500 ether 00:10:e0:94:be:15 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno3: flags=6147 mtu 1500 ether 00:10:e0:94:be:15 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eno4: flags=6147 mtu 1500 ether 00:10:e0:94:be:15 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s29u1u8c2: flags=4163 mtu 1500 inet 169.254.182.77 netmask 255.255.255.0 broadcast 169.254.182.255 inet6 fe80::21:28ff:fe57:4717 prefixlen 64 scopeid 0x20 ether 02:21:28:57:47:17 txqueuelen 1000 (Ethernet) RX packets 10 bytes 608 (608.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 150 bytes 31079 (30.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 0 (Lokale Schleife) RX packets 181 bytes 11345 (11.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 181 bytes 11345 (11.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0


Create Port Channel LACP for two Server NICs 10 Gbit/(s)
conf t int port-channel 13 switchport switchport access vlan 1 switchport mode access desc PortChannel-to-SERVER ! int Eth103/1/1-2 switchport access vlan 1 speed 10000 channel-group 13 mode active ! wr


Setup Cisco 2960-X Switch
conf t service password-encryption line con 0 logg sync exit line vty 0 15 logging synchronous exit no ip domain-lookup ntp server exit wr copy run tftp:


Set Switchport 39-48 in access to VLAN60
conf t vlan 60 exit int ra gi1/0/39-48 switchport mode access switchport access vlan 60 spanning-tree portfast int gi1/0/50 switchport mode trunk end sh cdp n sh span vlan 60 conf t int gi1/0/50 description Uplink, Core-N5K, eth1/9 end sh vlan b term le 0 sh run