Configuring Nested Hyper-V on ESXi 5.5

In order to allow for installation of a nested Hyper-V on ESXi 5.5, there is a requirement to configure the virtual machine settings once the guest operating system has been deployed.

Firstly, we need to add two items to the existing virtual machine configuration file in a powered off state. Prior to making these changes we will make a backup of the current virtual machine configuration in the event we need to roll back.

cp /vmfs/volumes/<datastore>/<virtual machine>.vmx /vmfs/volumes/<datastore>/<virtual machine>.vmx.backup

In order to  enable nested Virtualization Technology to run 64-bit virtual machines the following is required to added to the configuration file using a text editor.

vhv.enabled = "TRUE"

Now in order to run a hypervisor inside a virtual machine, we will add the following item to override the default setting. ¬†This prevents the error message “Hyper-V cannot be installed: A hypervisor is already running” if you attempt to install the Hyper-V server role in the guest operating system.

hypervisor.cpuid.v0 = "FALSE"

Finally, we need to expose¬†hardware virtualization to the guest operating system so that the processors support the required¬†virtualization capabilities. If you do not expose the hardware virtualization you will receive the error message “The processor does not have the required virtualization capabilities” on installing the Hyper-V server role.

This can be performed using the vSphere Web Client to edit the virtual machine CPU settings as below:

 

VMHyperVCPUSettings

 

Once , the above configuration changes have been applied to the virtual machine you should be able to install the Hyper-V server role.

 

 

Unable to configure Lockdown Mode on an ESXi Host

I was recently configuring Lockdown Mode in my lab environment when I discovered an issue where I could not configure the status on a single ESXi host system where the state was Disabled from the vSphere Web Client and greyed out from the DCUI.

I discovered that the symptom was caused as the permissions for the DCUI user were removed from the ESXi host system. In order to resolve the issue I performed the following on the ESXi host system.

1) Connect to the ESXi host system using the vSphere Client.

2) Select the ‘Permissions’ tab.

3) Right click and select ‘Add Permission’.

4) Select the DCUI user and assign the Administrator role to the user account.

5) Select ‘OK’.

Following the above change I was able to modify the Lockdown Mode from the DCUI and then manage the ESXi host system from the vSphere Web Client.

Enabling logging for Likewise agents in ESXi

In order to join ESXi hosts to the domain Likewise agents are used to join to the Active Directory domain and for user authentication requests. In order to troubleshoot Active Directory integration you will need to enable logging of the agent as by default they do not generate a log file.

This can be performed by enabling logging for the netlogond, lwoid and lsassd daemons.

1) Modify the file ‘/etc/init.d/netlogond’ file to change the line ‘PROG_ARGS=”–start-as-daemon –syslog‘ to one of the below, if you are using a scratch partition use the second option:

PROG_ARGS="--start-as-daemon --logfile /var/log/netlogond.log --loglevel debug" 

PROG_ARGS="--start-as-daemon --logfile /scratch/log/netlogond.log --loglevel debug"

2) Modify the file ‘/etc/init.d/lwoid‘¬†file to change the line ‘PROG_ARGS=”–start-as-daemon –syslog¬†to one of the below, if you are using a scratch partition use the second option:

PROG_ARGS="--start-as-daemon --logfile /var/log/lwiod.log --loglevel trace" 

PROG_ARGS="--start-as-daemon --logfile /scratch/log/lwiod.log --loglevel trace"

3) Modify the file ‘/etc/init.d/lsassd‘ file to change the line ‘PROG_ARGS=”–start-as-daemon –syslog”‘ to one of the below, if you are using a scratch partition use the second option:

PROG_ARGS="--start-as-daemon --logfile /var/log/lsassd.log --loglevel trace"

PROG_ARGS="--start-as-daemon --logfile /scratch/log/lsassd.log --loglevel trace"

4) Restart each of the services to apply the changes:

/etc/init.d/netlogond restart

/etc/init.d/lwiod  restart

/etc/init.d/lsassd restart

Unable to join ESXi host to the domain – Error when handling SMB socket

I was recently joining ESXi (5.5.0, 1892794) hosts to a domain to which the task would ¬†fail with the status ‘Errors in Active Directory operations’. ¬†On further investigation of the Likewise agent log on the impacted ESXi host, the following was being written to the log file:

 

ERROR:[SMBSocketReaderMain() /build/mts/release/bora-1471401/likewise/esxi-esxi/src/linux/lwio/server/rdr/socket.c:660] Error when handling SMB socket

 

This issue is due to¬†¬†the size of the Kerberos Ticket Granting Service (TGS) being¬†very high. From the¬†network capture for SMB errors in the Likewise agent logs where the ‘Security Blob Length’ and¬†‘Byte Count’ values are¬†greater than the ¬†Max Buffer Size on the domain controller to which the ESXi host is setting up a SMB session which by default is 16644 bytes or 4356 bytes if total memory is less than or equal to 512 MB on the host.

Below is an example of the above values in an SMB network capture:


Security Blob Length: 19314
Byte Count (BCC): 19371

 

In order to resolve this issue, I was required to add a DWORD value name ‘SizeReqBuf’ to the registry key ‘HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters’ where the the value data (Decimal)¬†is greater than the values being returned from the network capture and then restart the domain controller(s).

 

 

PowerCLI: Changing ESXi Host root password

In order to change the root password of an ESXi Host using PowerCLI, firstly you will need to connect to the host in question as the root user:

Connect-VIServer esxi1.domain.local-User root-Password assdKIUU1235

Now by invoking the ‘Set-VMHostAccount’ cmdlet, we can change the root password as follows:

Set-VMHostAccount ‚ÄďUserAccount root ‚ÄďPassword 123GHYJ!rys

Retrieving ESXi Host inventory information using PowerCLI

Over time I have produced a number of powershell scripts using the vSphere PowerCLI snap-in to return information from vCenter

Firstly, I looked into returning inventory information for all the ESXi Hosts where the following would be retrieved:

  • Name
  • Hardware Vendor
  • Hardware Model
  • CPU Model
  • Datacenter
  • Cluster
  • Hypervisor
  • Hypervisor Version
  • Clock Speed
  • Memory
  • Hyperthreading Active
  • Number of Cores

In order to invoke the PowerCLI cmdlets ¬†we will need to add the snap-ins to the current powershell session and connect to the vCenter server (in the below example this is named ‘server.domain.local’).

if (-not (Get-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue)) 
   { 
   Add-PSSnapin VMware.VimAutomation.Core > $null
   } 

Connect-VIServer server.domain.local

Now, in order to retrieve information for each ESXi host we will build a collection by invoking the Get-VMHost cmdlet

$VMHosts = Get-VMHost

From the ESXi hosts returned in the collection we will loop through each one and return the required information as above and return this as calculated properties and store in a variable to export later to a CSV file.

$Inventory = ForEach ($VMHost in $VMHosts)
   { 
   "" | Select-Object -Property @{N="Name";E={$VMHost.Name}},
   @{N="Vendor";E={(Get-View -ViewType HostSystem -Filter @{"Name" = $VMHost.Name}).Hardware.Systeminfo.Vendor}},
   @{N="Model";E={(Get-View -ViewType HostSystem -Filter @{"Name" = $VMHost.Name}).Hardware.Systeminfo.Model}},
   @{N="CPU Model";E={$VMHost.ExtensionData.Summary.Hardware.CpuModel}},
   @{N="Datacenter";E={(Get-Datacenter -VMHost $VMHost.Name).Name}},
   @{N="Cluster";E={(Get-Cluster -VMHost $VMHost.Name).Name}},
   @{N="Hypervisor";E={$VMHost.Extensiondata.Config.Product.Name}}, 
   @{N="Hypervisor Version";E={$VMHost.Extensiondata.Config.Product.Version}}, 
   @{N="Clock Speed (Mhz)";E={$VMHost.ExtensionData.Summary.Hardware.CpuMhz}},
   @{N="Memory (MB)";E={$VMHost.MemoryTotalMB}},
   @{N="Hyperthreading Enabled";E={$VMHost.HyperThreadingActive}},
   @{N="Number of Cores";E={$VMHost.ExtensionData.Summary.Hardware.numCpuCores}}
   }

As I want the output to create a unique filename each time the inventory script is invoked to the output folder D:\Output, therefore I will use the Guid.NewGuid method when specifying a filename.

$Inventory | Export-Csv -NoTypeInformation -Path ("D:\Output\" + [guid]::NewGuid() + "-inv_hosts.csv")

Below is a table to include the cmdlets required to extract the inventory information in the requirements.

Name Cmdlet Property
Name Get-VMHost Name
Hardware Vendor Get-View Hardware.Systeminfo.Vendor
Hardware Model Get-View Hardware.Systeminfo.Model
CPU Model Get-VMHost ExtensionData.Summary.Hardware.CpuModel
Datacenter Get-Datacenter Name
Cluster Get-Cluster Name
Hypervisor Get-VMHost ExtensionData.Config.Product.Name
Hypervisor Version Get-VMHost ExtensionData.Config.Product.Version
Clock Speed (Mhz) Get-VMHost ExtensionData.Summary.Hardware.CpuMhz
Memory (MB) Get-VMHost MemoryTotalMB
Hyperthreading Active Get-VMHost HyperThreadingActive
Number of Cores Get-VMHost ExtensionData.Summary.Hardware.numCpuCores

 

Error “CRITICAL_STRUCTURE_CORRUPTION” on Windows Server 2012 R2 using ESXi

I recently experienced BSOD with the error message “CRITICAL_STRUCTURE_CORRUPTION” for a virtual machine running Windows Server 2012 R2 running on a ESXi Host 5.0.0, 623860.

This is a known issue and was resolved in ESXi 5.0 Update 3.

There is also a workaround to create a CPUID mask for the virtual machines with the above symptom.

1) Power down the virtual machine.

2) Edit Settings for the virtual machine.

3) Select the ‘Options’ tab and browse to ‘Advanced>CPUID Mask’ and select ‘Advanced’.

4) Paste the below into the edx register field under Level 80000001.

----:0---:----:----:----:----:----:----

5) Apply the settings and power on the virtual machine