vSphere4 VLAN configuration on Nexus 5020

Recently I am doing a feature testing for Nexus 1000v which is a virtual switch incorporating into the vCenter in the form of a VM.  One of the prerequisites to setup this 1000v is having several VLANs-One for management, one for packet and another for control. I’ve always been using standard vSwitch without VLAN tagging for previous tests, so I wanted to use two ESX machines test VLAN first and later setup 1000v VM. And of course, I saw some tricky issues.

Talk about my setup first:

• 2 ESX 4 U1 servers

• Each server has one 1G Ethernet card and 1 FCoE CNA card

• The Ethernet card connects to public network and CNA connects to a Nexus 5020 which uses private network

• ESX and VMware are booting from EMC SAN

I created one VM on each ESX under the private network using CNA 10G Ethernet connection. Then I created a VLAN 5 on Nexus 5020. I set both VMs to use VLAN 5, but found the two can’t ping each other. Suddenly I realized trunking needs to be enabled for those ports using VLAN. ESX supports three kinds of VLAN tagging–Virtual Machine Guest Tagging (VGT Mode), External Switch Tagging (EST Mode) and ESX Virtual Switch Tagging (VST Mode). The VST mode is policy-based and easy to configure but still needs port trunking. Seems VMware side only has ESX 3.5 VLAN documents which also mention to enable “spanning-tree portfast” for the ports, so I did it. You also need to specify the allowed VLAN(range) on the port, otherwise traffic will be blocked. After these steps, the VMs could ping each other. I changed the VLAN tag arbitrarily as VLAN 6 which I didn’t create physically on Nexus 5020. VMs couldn’t ping again. I logged in to Nexus switch, created VLAN 6 and allowed the VLAN ID, problem gone. So another tip I got is you still need a physical VLAN on switch first before you use any of the VLAN tagging methods.

As a summary, here are the commands to configure VLAN on Nexus 5020 in my case

NX-5020# config

NX-5020(config)# vlan 5    //create the vlan

NX-5020(config)# show vlan //make sure the vlan is created

NX-5020(config)# show run //check the Ethernet interface settings of your target port

(I recommend to simply clear the settings on the port for easy troubleshooting)

NX-5020(config)# interface Ethernet PortID   //select the port

NX-5020(config-if)# switchport mode trunk  //Enable the trunking mode

NX-5020(config-if)# spanning-tree port type edge trunk  //Enable the port-fast on trunk, you will probably see

//warning message, just ignore

NX-5020(config-if)# switchport trunk allowed vlan ID1,ID2….    //Allow the VLANs, you can also use “all” to

//allow all VLANs
Now I’m configuring my Nexus 1000v now, still figuring out the topology, will update once it’s done. Hope this post can solve a bit confusion during your vSphere4 VLAN configuration on Nexus 5020.