Mike's Realm chown -R us ./base


vCenter Orchestrator Import Signed Certificate

So your trying to import a signed certificate that was created with the certificate signing request (CSR) you get an error in vCenter Orchestrator Configuration about the cert not being the correct format. If your using Microsoft Active Directory Certificate Services here are the exact steps:

  1. Export a certificate signing request from the VMware vCenter Orchestrator Configuration: Server Certificate area (goto http://vco-server:8282 then click Server Certificate on the left).  If this option isn't displayed select the option to install a self signed certificate and then you will get the option to export a certificate signing request.
  2. Copy and paste the contents of the CSR file you downloaded from vCO Config area to your Cert Server web interface (http://CERTSERVER/certsrv).
  3. Select Web Server from the drop down and submit.
  4. Now ensure DER encoded is selected and download the certificate chain.
  5. Change the file extension on the file you just downloaded from .p7b to .csr
  6. You should now be able to upload it immediately using the "Import certificate signing request signed by CA" option inside the VMware vCenter Orchestrator Configuration: Server Certificate area (again goto http://vco-server:8282 and select Server Certificate on the left)
  7. You get a green bubble by Server Certificate and everything is happy.

vCenter Server Source of Active Directory User Account Lockouts?

So a user account is getting locked out from your vCenter server?  Check the windows security event logs, they typically clearly point out the culprit.  If they are not much help you can start with the common things that are applicable for any server causing account lockouts:


These can be running under the locked out user account

Persistent Drive Mappings

Using the locked out user account credentials

Disconnected TS/RDS Sessions

A process can be running that is using the locked out user credentials

ODBC Connections

Ensure you did not use the user account that is being locked out for an ODBC connection for the vCenter database

Scheduled Tasks

Scheduled task(s) can be setup to run as the locked out user

Once you've exhausted all of that...

VMware Specific Areas to Check

vSphere Client

Can be running with out of date credentials and caused the lockout, you can use the sessions area in vCenter to check for active sessions

vCenter Plugins

Guided Consolidation

Uses a specified user account to poll servers to see if they are good candidates for virtualization

Update Manager

Has a proxy configuration area you can define a user account to login to the proxy with

VMware Data Recovery

Data Recovery uses stored credentials to connect to vCenter, ensure the specified user isn't the one being locked out

NetApp's Vitual Storage Console

I don't think there is specifically a place you can have cached credentials in here, but I registered this to my vCenter using my account and it ultimately ended up locking me out about a month later when I changed my password due to expiration

Still Stumped?

Look at any Monitoring Tools (especially trials and free utilities you may have forgotten about) you ever installed on the box that might be running with the locked out user account: Veeam tools, vKernel tools, Vizioncore tools, Quest vFoglight Quick View, etc.  You can always disable services and wait to see if the lockouts continue if you really get stumped.


vCenter: Physical or Virtual? High Availability? Clustered?

First please know that vCenter being down does not take your whole vSphere environment down.  It limits you on creating new tasks (like deploying a new VM from template) until vCenter is back up.  When vCenter is down HA/FT continue to function.

Physical or Virtual

vCenter is the heart of VMware's virtualized infrastructure, but many folks are reluctant to virtualize their vCenter.   Running vCenter as a VM is completely supported by VMware.  You get all kinds of benefits from running vCenter as a Virtual Machine:

  • HA will also protect vCenter in the event the host it is running on goes down
  • You can vMotion vCenter from one host to another for maintenance and other things
  • Prior to upgrading vCenter to a newer version you can snapshot to help with rolling back more easily
  • Best of all--You gain the benefit of virtualizing yet another system and move towards virtualizing 100% of your data center.

If you are thinking about Virtualizing vCenter glance over this page out of the VMware Library:
VMware Online Library: Install vCenter Server in a Virtual Machine

High Availability

First the biggest thing is that you should remember  HA/FT will continue to operate without vCenter--all decisions will be made using a snapshot of what the extra resources were in the cluster prior to vCenter going down.

Lets look at the major things vCenter does:

  • VMware Distributed Resource Scheduler (DRS)
  • VMware High Availability (HA)/Fault Tolerance (FT) - Configuration
  • VMware VMotion + Storage VMotion
  • VMware Update Manager (Guest and Host)

After reviewing those items--does anything stick out that makes you think vCenter needs to be up 24/7?  Would HA be sufficient protection so you only have a small amount of downtime in the event the host running your vCenter VM went down?  I think yes.


You can look into vCenter Server Heartbeat, this is licensed as an addon to vCenter.  vCenter Server Heartbeat is basically an Active/Passive cluster for vCenter that can be setup to run locally or across your WAN.  vCenter Server Heartbeat also has the advantage that it can protect more then just the vCenter, it also protects the addons like vCenter Converter and vCenter Update Manager--even Guided Consolidation can be protected.  It is more costly then just running vCenter as a VM and protecting it with HA but the benefit of having an Active/Passive clustered vCenter + addons across the LAN/WAN may be beneficial for your organization.

Microsoft Cluster Services / Veritas Cluster Services

vCenter can be protected via "third party solutions" such as MSCS or VCS and VMware will support you to some degree but they do not certify these configurations.  If you have an issue VMware may determine the cause to be the third party software and not be of much assistance beyond that...  If you are thinking of going this route read over this VMware KB: Supported vCenter Server high availability options


Introducing: VMware vCenter Mobile Access (vCMA) from VMware Labs

VMware vCenter Mobile Access (vCMA) is a cool fling from VMware Labs. It allows mobile access to your vSphere environment via your vCenter(s).  Setting up vCMA takes very little effort as it is packaged as a virtual appliance.  You simply download vCMA as an OVF, deploy the OVF Template, and power on vCMA.  Once powered on, config the network and your ready to go.  Note that vCMA does not use a service account or static connector to vCenter, each user will login to vCenter via vCMA with their own credentials--think of vCMA as a web-based version of the vSphere  Client.


Introducing InventorySnapshot from VMware Labs

Checkout this awesome new Fling from VMware Labs, it's called InventorySnapshot.   Basically what it does is allows you to snapshot your vCenter and reproduce it on another vCenter.  Say you were doing an out of place migration and didn't want to bring your old database along for some reason, or just in your lab trying to replicate your production config.  You don't have to reproduce all the objects though, you can specifically restore just Resource Pool settings, DRS settings, Roles & Permissions, or again the whole damn inventory.

InventorySnapshot supports reproducing the following vCenter objects:

  • Datacenter Folders
  • Datacenters
  • Clusters
  • Resource Pools
  • vApps
  • Hierarchy
  • Roles & Permissions
  • Configuration Settings
  • Custom Fields

As you can see the only major item they are missing is Alarms, which they are working to support. The developers Balaji Parimi and Ravi Soundararajan did an excellent job documenting their Fling with a 17 page doc, they took the time to write a large troubleshooting and layout a few caveats/known bugs.


VMware Capacity Planner: Guided Consolidation

VMware Partners get access to a very nice tool called Capacity Planner--this is what VMware Partners use to come in and do an assessment of your environment and determine what can be virtualized, and how many hosts are required to make it happen.  VMware Partners use the tool free of charge from VMware--the partners and are encouraged to do these assessments at no cost for customers (after all, the results from a capacity planner assessment leads into a vSphere PoC for new customers typically).  However that left customers crying that they should have something they can run themselves--why should they have to bring in a partner to do this?

VMware answered customers by creating  vCenter Guided Consolidation--it doesn't pack anything near the features VMware Capacity Planner does--but it offers the very basic functionality of if a particular server (and associated workload) is a good candidate for virtualization.

Installing Guided Consolidation

So you need vCenter for this if it wasn't apparent.  It's simply a plugin to vCenter--now the install process actually stumped me for a bit, I browsed the vCenter media for the installer and came up empty handed...then a college pointed me to the Autorun menu on the vCenter media and I face palm'd...I never let Autorun actually run--haha.

Poking around wanting to find the installer after seeing it in the Autorun menu I found that damn installer.  It's at ./vpx/VMware-gcs.exe on the media.  GCS-Perhaps Guided Consolidation Server?

Using Guided Consolidation

This is broken up into 3 main steps:

  1. Find - Discover your physical servers
    • Find offeres these ways to discover
      • Manually enter Hostname/IP Addresses
      • Domain Discovery -- via Active Directory Domains
      • Scan an IP Range
      • Suck in a text file containing a list of Hostnames/IP addresses to scan
  2. Analyze - Perform analysis on the servers to determine if they are good candidates for virtualization
    • Once a machine is actually being analyzed this is where Guided Consolidation can use some improvement--there is no log or status as to what is happening--you typically have to wait at least an hour before any status changes appear.
    • Also note that confidence won't change to High until it analysis has been in-progress for at least 12 hours (might be 24 actually) so don't be alarmed
  3. Consolidate - Complete the actual consolidation by P2V
    • Another area that has ALOT of room for improvement in Guided Consolidation
    • After you are happy with the analysis of a server select it then click the Plan Consolidation button.
    • A wizard appears that one would think would offer similar functionallity when doing P2Vs using regular VMware Converter--but no, it's not the same at all.
    • Select your destination(s) vcenter/cluster/hosts
      • Guided Consolidation will analyze the hosts and recommend the best destination for the VM with the Amazon-esc 5-Star rating icons.
    • Review the recommendation--you can change the destination host as well as destination VM name in this window
    • Once you click next, you have one more chance to review....and then a finish button?! what?--I want to change vCPUs, Memory, Target Disk Layout!

And so now you can see the limitations of Guided Consolidation--it's great at the core function of determining if a server is a good canidate to virtualize or not--however the built in P2V process leaves alot to be desired for the seasoned VMware Admin.  I recommend you use the analysis portion and continue to use vCenter Converter to perform the actual P2V so you have more control over the P2V process--changing Target Disk Layouts, vCPU count, Memory amounts, etc.

For more information on VMware vCenter Guided Consolidation check out the vSphere 4 Admin Guide @ VMware.com

Tagged as: , , 2 Comments

Introducing PXE Manager for vCenter

PXE Manager for vCenter enables ESXi host state (firmware) management and provisioning, Specifically, it allows:

  • Automated provisioning of new ESXi hosts stateless and stateful (no ESX)
  • ESXi host state (firmware) backup, restore, and archiving with retention
  • ESXi builds repository management (stateless and statefull)
  • ESXi Patch management
  • Multi vCenter support
  • Multi network support with agents (Linux CentOS virtual appliance will be available later)
  • Wake on Lan
  • Hosts memtest
  • vCenter plugin
  • Deploy directly to VMware Cloud Director
  • Deploy to Cisco UCS blades

What does that mean?  It automates the provisioning of ESXi hosts in either a stateless or stateful mode.  (notice no ESX support here!) via network boot using Pre-boot eXecution Environment (PXE)

How does that work?

Remember ESXi has a very small footprint--it's quite small that PXE booting ESXi is very easy.

Stateless and Stateful?

Stateful means the host keeps the "ESXi state" upon reboot--meaning the same version.  Think back to Microsoft RIS (remote installation services) days for VMware ESXi.

Stateless means the host doesn't keep the ESXi state upon reboot.  Why the heck would you want to do that you might ask?  I say why the heck wouldn't you want to do that?  Patching and upgrades becomes a breeze, throw a host into maintence mode--all the VMs evacuate to other hosts in the cluster then reboot the host.  When it comes up it's running the latest and greatest version of ESXi.  No extra leg work patching the host, it gets it automatically upon boot!  Think of the possibilities with DPM in the mix, a good amount of your environment can be automatically upgraded nightly when hosts get powered back on by DPM.


Provisioning becomes much easier--no need to install ESXi, and along with that the extra hardware required (SD Cards + Reader, Mirrored OS Drives, etc.)  Just rack new hardware and configure the BIOS for PXE boot and go!

Want to learn more?  Max Daneri threw together a great overview powerpoint

Ready to download? Grab it from VMware Labs and while your there check out other cool new things VMware is working on.


VMware: Windows could not parse or process the unattend answer file for pass specialize

When using guest customization specifications in vCenter you may come across the following error when deploying a VM using a specification:

Windows could not parse or process the unattend answer file for pass specialize.  The settings specified in the answer file cannot be applied.  The error was detected while processing settings for component [Microsoft-Windows-Shell-Setup].

The Solution

Edit the specification and double check your product key, an invalid key is a common reason for this error.  In my experience Windows 2008 and Windows 2008 R2 keys cannot be interchanged--yet I see this attempted to be done in guest specifications.