Thursday, January 31, 2013

VCAP5-DCD Study Prep

I've turned major disappointment into productive energy, and I will complete my VCAP5-DCD in two weeks.

There are three primary (I guess they can't all be primary, so we'll call them Tier 1) sources of information. In no particular order:

  1. The student books from the vSphere 5: ICM course. I'm always surprised how much detail can be found in the slide notes from these two books. For example, there's a discussion about ensuring that NFS exports haven't been configured to deny root read / write access, since root owns the NFS vmkernel interface on the ESXi hosts. (Contrast this to Linux / UNIX systems where nfsuser most likely owns the connection). Great material for reviewing the foundation of your VMware knowledge.
  2. The VMware vSphere 5.1 Clustering Deepdive. Who doesn't read this all the time, anyway?
  3. VMware vSphere Design v5 - Great source for looking beyond the technical aspects of a design engagement.
I've read these three several times, but I always refer to them. They're heavily defaced with notes and highlights and post-its. As opposed to my Sendmail bat book, which is nearly pristine after all these years.

This time around, I'm going to read two new books (perhaps I should have done this last time):
  1. Managing and Optimizing VMware vSphere Deployments, and
  2. Mastering VMware vSphere 5.
I suspect that I'll be half-way prepared for my VCAP5-DCA once I'm done with these books. I've been waiting until DCD was finished to really focus on the DCA exam, though I suspect that will be slightly easier. I've read lots of posts from people who sailed through the DCA only to get stymied by the DCD.

Of course, this is all just preparation for the ultimate goal: VCDX. I fear my opportunity to be a VCDX hipster has passed (I was VCDX before VCDX was cool), but I'd still like to at least give it a shot. Curious how the process works.

Anyway, the five books (six if you count the VCP study material as two) above should finally get my mind right for this beast of an exam. If you've found other material useful, let me know and share with the community!

Wednesday, January 30, 2013

Failed: VCAP5-DCD

Unpleasant but true: I failed the VCAP5-DCD exam today. Judging from the score, I missed the passing score by a single question. You can probably sense the frustration from wherever you're reading this.

I had doubled-down in the last week and started re-reading the vSphere 5.1 Clustering Deepdive and the VMware vSphere 5 Design books. I always learn something new when I re-read these books. Because I'm not doing a lot of design work lately, I end up reading a lot but not getting the opportunity to apply that knowledge. I'll whiteboard designs throughout the day just to keep in practice, but that's no substitute for a real design engagement.

The only gripe I'll voice at this time is this: design is an iterative discussion, not a series of questions with answers. The designs I've delivered over the years are good by all measures, and I attribute that quality of work to the many discussions I've had with all parties involved. And if I'm not sure of something, I can pick up the phone and get clarification. Whereas the exam is a one-sided conversation, and clarification is not to be had. Sour grapes, right? Yeah. But I'm a social engineer (well, not a social engineer...), and talking with people is the best part of my job.

So now the 14 day wait begins. My goal is to achieve VCAP5-DCD prior to attending PEX 2013. I'd like to have a VCAP pin to complement my VCP pin.


Friday, January 25, 2013

Port Group Options for vSS - Part 1

One aspect of vSphere networking that I always need to look up is configuring the Port Group options. I'm creating these two posts to once and for all learn these settings and their impacts, and of course to share this information with other VMware users.

To the left, you'll see the properties for a Port Group named "VM Network" on a vSS named vSwitch0. This is the vSS that is created during the initial installation of ESXi 5.1. The values you see for the VM Network Port Group are the defaults. But what does each value mean, and what's the impact of mucking about with changing them?

We'll walk through each tab on the port group's properties page and discuss each one, including a bit about why you'd want to change them. You'll find this information useful if you're preparing for your VCP5-DV exam, or if you're interested in learning about vSphere networking in general.

To clarify, this post is regarding options for a Virtual Machine Port Group. VMKernel Port Groups are slightly different, so I'm focusing on VM Port Groups for now.

The General Tab

Aside from the gratuitous waste of screen real estate, you'll find two options to set on the General Tab for your vSS Port Group: Network Label, and VLAN ID.

The Network Label is extremely important to get right on all of your hosts; in this case, right means consistent. This is required by vMotion, which will validate that a VM's network requirements can be satisfied on the destination host. Differences in Port Group names, even as trivial as capitalization, can cause at best warnings, and at worst, failures with vMotion. No foolish consistency here.

The VLAN ID associates your Port Group with a specific VLAN. In the example, you'll notice that this field is set to None (0). In my test lab, I'm not using VLANs, so there is no need to change the default. However, if you are bringing multiple VLANs into your vSS (say, one for web servers, one for database servers, one for security servers), you must enter the VLAN ID that will provide the correct network access for your interfaces. VM interfaces in this port group will ONLY have access to the VLAN you define here.

The Security Tab

You'll find some more interesting options in the Security Tab: Promiscuous Mode, MAC Address Changes, and Forged Transmits. Note that the options are not checked by default; this tells you that you are currently using the configuration of the vSS itself. If you need to change these settings for a specific port group, check the box and set the appropriate value.

Promiscuous Mode: Enabling this option allows a network interface in this port group to see all frames on the VLAN that's defined for this port group. Why would you want to do this? The most common reason is that you've virtualized a network / packet analyzer (read: wireshark) and you want to view all frames on a VLAN, not just broadcasts and traffic to / from your VM. The default setting here is Reject, which will satisfy most of your VM connectivity requirements.

MAC Address Changes: Each virtual interface on a VM has its own MAC address; this address uniquely identifies your interface on a network, and is the true source and destination of network traffic. A virtual interface's MAC address is contained within the VM's configuration file (stored in its datastore folder as a .vmx file). In the event that the MAC address is changed from the value in the .vmx file, one of two things can happen (depending on your selection for the MAC Address Changes setting): if you've selected Accept, traffic will immediately move to the new MAC address; but if you've selected Reject, traffic to that interface will be dropped. Personally, I find that this setting is less significant in vSphere 5.1 than in previous versions. In 5.1, once a MAC address has been generated for a virtual interface, it does not change unless there is a collision with another virtual interface. Because vSphere 5.1 tracks generated MAC addresses, this condition is very unlikely. Of course, the default behavior handles these changes without interrupting your VM's connectivity.

Forged Transmits: This setting determines how vSphere handles frames that are sent with a source MAC address that is different from the source interface. By default, this type of traffic is passed. Changing this option to Reject would drop all frames with a forged source MAC address. Why would you want to do this? If you have any security concerns regarding a host masquerading as another host on your network, you may want to set this to Reject. In this scenario, ESXi would detect the different in source MAC and the virtual interface's MAC and quietly drop the frame.

Let's stop here for now. The next two tabs, Traffic Shaping and NIC Teaming, can be daunting, so make sure you're comfortable with these two tabs first.

Tuesday, January 22, 2013

Installing VMTools on Fedora 17

Since I've been spending some quality time with a few Fedora17 VMs lately, I thought I'd share the process I've been using to install VM Tools. The process can be frustrating at first, but once you understand what the VM Tools installer is looking for, you can get everything up and running in 10 minutes.

Here's a summary of the steps, if you prefer a litany over the narrative:

1) Run a software update to make sure you've got the latest packages on your VM.
2) In the vSphere client, start the Tool installer.
3) Copy the VMwareTools-<build information>.tar.gz to /usr/tmp.
4) Extract the tools package from the archive.
5) Install the latest kernel-devel package, along with gcc (if you don't have a compiler already).
6) Run the install script.

Now, the narrative. First, let's start with a powered off VM. You'll see in the picture below that Tools are not installed.
VMware Tools: Not running (Not installed)

Power on your VM and log in. (You will need to be root, or to have root privileges, to perform this task). Next you'll want to initiate the Tools install process via the vSphere client (VM > Guest > Install / Upgrade VMware Tools). Click OK on the message that pops up (it's just telling you how much better interactions with you VM will be after Tools is installed). When you start the Tools install, vSphere will mount an ISO to your VM that contains the Tools application. That means on your Fedora VM you'll see a message pop up about how to handle the new media.

Fedora's Prompt for the newly discovered media.
Now we can get to the fun part. Select Open with Files, then Extract the file to a working location, like /usr/tmp. Once this is done, you'll have a new directory named vm-tools-distrib.Within that directory, you'll find This script will walk you through the process of configuring and compiling the VMware Tools.

Of course, if you don't have Perl on your system, this file is not going to do much. Getting Perl is easy: yum -y install perl. And while you're at it, get gcc (yum -y install gcc) and the kernel headers for the running kernel (first, update your system to the latest kernel, then run yum -y install kernel-devel). Otherwise the configure script (which executes after the install script) will fail. Like this:

Once you've got Perl, gcc, and kernel-devel loaded, you can run the script. Unless you have a good reason to, avoid changing the default settings. The script takes a few minutes, and when it's done you'll need to restart your X session to complete the install. Then you'll see this:

Tools running and current (which is based on a Latin word that means running, so...).
Wow. That was a long-winded post for something as simple as a tools install. But if you're not comfortable with Linux command-line administration, you should be able to get Tools up and running without any problems by following the steps above. Let me know if this helps, or if you run into trouble.

Also, if you're looking for help with VMTools on Fedora18, I've got a post about that here.

Sunday, January 20, 2013

UCS Emulator 2.1

While I was writing the second half of the last post on UCS, I had an epiphany: the UCS emulator is made primarily for developers to use in the creation of new APIs and plug-ins for UCS. Although it has been used many times by admins and engineers for demonstration purposes, it is first and foremost a tool for developers.

With regard to the clustering post, the emulator specifically does NOT allow the connect local-mgmt command to be issued. (update: ok, so it DOES let you do this. However, trying to run enable cluster a.b.c.d gives an error no matter if the IP you choose for your VIP is in the same VLAN as the FI's management interface or not). This command connects you to the local management port for certain administrative functions, including cluster configuration. I had planned on hopping on the emulator in the morning and connecting, but here's what you get when you log into the UCSPE via the command line:

Frustrating, yes. But now that I've accepted that the emulator wasn't intended for true testing of configurations changes, I'm not mad.

Of course, this means it'll take some time to grab screenshots of the cluster conversion. But I'll post the steps at least, so you can see what's involved.

Oh yeah: here's a link to the UCS Emulator. You'll need to use your ID to access the download. If you don't have one, sign up for free. No biggie.

Friday, January 18, 2013

Migrating from Stand-alone to a Cluster in UCS

Although Cisco UCS has been around for a while now, it's still a new technology to many engineers and environments. I may be biased a bit here, but UCS does some unique and innovative things that other server vendors don't.

A common scenario I've seen lately is one in which a customer has a single Fabric Interconnect that is a few years old, and they want to improve the availability and performance of their UCS by replacing it with a pair of new Fabric Interconnects. (Let's call these FIs now; I'm tired of tapping out the full name.)

Of course, this is a well travelled path. The short answer is to do the following:

1) create a one-node cluster with your original FI.
2) add the first new FI to your cluster.
3) make the new FI the primary cluster node.
4) remove the original FI from the cluster.
5) finally, add the second new FI to your cluster.

Yes, that's the short answer. Long answer to be posted in the morning.

Thursday, January 17, 2013

Fedora17 on ESXi 5.x

Anyone who has loaded Fedora17 on a new VM in ESXi 5.x has seen this screen during the first boot:

Short version: there's a known bug with the fprintd package that ships with Fedora 17. If the VM you're using for Fedora doesn't have a USB controller added to it, fprintd will crash and halt your system.

You have two options to fix this problem; both are easy to implement. Before proceeding, decide if fprintd is something you can't live without.

If you can live without it, do this:
  1. Restart the VM.
  2. At the boot loader, select Advanced Options for Fedora Linux.
  3. On the next screen, select the option with (recovery mode) at the end.
  4. You'll see a system prompt with root@hostname; this is your recovery console.
  5. Run this command: yum -y remove fprintd
  6. When fprintd is gone, reboot the system.
Your VM will now boot without any complaints.

However, if you need fprintd, there's a fix for that, too. Remember that fprintd wants a USB controller at system startup. So give it one:
  1. Right click your fedora VM and choose Edit Settings...
  2. On the hardware tab, click Add...
  3. Highlight USB controller and click Next.
  4. Select the type of controller you want (if you're not sure, take the default).
  5. Click Next to review your settings, then click Finish.
  6. Click OK to commit the change.
Your VM now has a USB controller. Restart your VM, and fprintd will load properly, which means Fedora will load properly.

VCAP5-DCD Study Material - PSP Options

No design engagement can be complete without consideration given to the customer's change management policies. You need to clearly understand what the customer considers a "change" and what can be considered normal, day-to-day administration and operation. Such understanding becomes significantly more important with VMware design, as many of the automation features in vSphere can fall into the "change" category.

A particular example is the Path Selection Plug-ins (or PSPs) for storage pathing. PSPs are part of the VMware Multipathing Module, which is called the Native Multipathing Module (or NMP). PSPs allow for highly available access from your ESXi host to your storage. The PSP that you select will determine what happens in the event that one of your storage paths has failed, and more importantly, what happens when that failure has been corrected.

Here's a table that summarizes the behavior of each PSP:

PSP SelectionAutomatic Failover?Automatic Failback?
Fixed (VMware)YesYes
MRU (Most Recently Used)YesNo
Round Robin*YesNo

The point to take away from this table is that when using the Fixed PSP, ESXi will automatically revert to the failed path when the failure has been corrected. Relative to the Change Management discussion, this failback may become a change that your client's organization wants to manage, as opposed to letting VMware automatically change paths. Whether this is a well-reasoned requirement is immaterial; a requirement is a requirement.

A Note about Round Robin
Round Robin is a special case, in that it has two modes: Active / Active and Active / Passive. These modes are determined by your storage controller configuration (e.g., if you've got two storage controllers and both are actively handling storage connections, you'll use Active / Active for Round Robin). In the event of a failure, this PSP will select the next storage path to the active controller. It will not fail back to the failed path once the failure has been corrected, though. Path changes only occur in Round Robin when the actively selected path fails.

Lots of other automation within vSphere can be discussed with regard to change management: vMotion via DRS, sDRS, even DPM (which is great, by the way! Expect a blog post on this soon). So plan accordingly and account for change management in your engineering.