Cannot change the host configuration when force mounting a VMFS volume

Following a recent upgrade to vSphere 5, a single VMFS volume was not mounted on a number of hosts within one of the clusters, after a short period of investigation the LUN was correctly mapped to the hosts and no configuration issues could be found. The VMFS volume was listed when selecting to ‘Add Storage’ from the vSphere Client when managing the host from my vCenter installation, however on adding the storage and selecting ¬†the keep the existing signature, the task would fail with the following error message:

“Cannot change the host configuration”

I then attempted to connect the existing VMFS volume and retain the existing disk signature, by managing the ESXi host using the vSphere client where this operation succeeded. I was also able to add the VMFS volume by managing another ESXi host by performing the following from the below from the command prompt to obtain information regarding the VMFS volume that could not be mounted;

esxcli storage vmfs snapshot list

Make a note of either the Volume Name or the VMFS UUID and ensure the can mount value is set to ‘true’, also the reason for¬†un-mountability¬†and non-resignaturability should be listed.

To mount the LUN so that it is persistent accross reboots run the following command;

esxcli storage vmfs snapshot mount -l <label>|-u <uuid> 

Where¬†label is the ‘Volume Name’ and uuid is the ‘VMFS UUID’ noted in the previous step.

For my specific issue, I did not require to resignature the volume, however the command to perform this action is as follows;

esxcli storage vmfs snapshot resignature -l <label>|-u <uuid>

From browsing the VMware KB, the cause of this issue is;

This issue may occur if:
  • Multiple ESX/ESXi 4.x and 5.0 hosts are managed by the same vCenter Server and these hosts are in the same datacenter.
  • A snapshot LUN containing a VMFS datastore is presented to all these ESX/ESXi hosts.
  • One of these ESX/ESXi hosts has force mounted the VMFS datastore that resides on this snapshot LUN.
  • A second ESX/ESXi host is attempting to do an operation at the same time.
When one ESX/ESXi host force mounts a VMFS datastore residing on a LUN which has been detected as a snapshot, an object is added to the datacenter grouping in the vCenter database to represent that datastore.
When the second ESX/ESXi host attempts to do the same operation on the same VMFS datastore, the operation fails because an object already exists within the same datacenter grouping in the vCenter database.
Since an object already exists, vCenter Server does not allow mounting the datastore on any other ESX host residing in that same datacenter.

In this case the LUN was not a snapshot or replica and further reading ( brought to my attention that VMFS volumes can be incorrectly recognised as snapshots, one of the first items to check was the LUN ID was not uniform accross all hosts, this is not the case in my scenario. However, following the volume being mapped using esxcli and subsequent reboots the volume is now persistent to all hosts in the cluster.

Below are two VMware KB articles I referenced in order to resolve my issue;

A calculation for MBps to IOPS (and vice versa…)

Whilst playing around with Storage I/O control (SIOC) in my vSphere 5 lab environment, I found myself trying to calculate the IOPS to be used when limiting a virtual disk throughput to a certain amount of MBps

So here is the calculation I was using:

IOPS = (MBps Throughput / KB per IO) * 1024 

So using the above, if I wanted to configure an IOPS limit to satisfy a 10 MBps throughput using a 8KB IO request size I would require to set the SIOC IOPS limit to 1280.

IOPS = (10/8)*1024 = 1280

It is possible to use the calculation in reverse to determine the MBps required to achieve the  IOPS claim using the following:

MBps = (IOPS * KB per IO) /1024 

In the above example where the requirement is to achieve 1280 IOPS, we would need a throughput of 10MBps to satisfy that claim.

MBps = (1280 * 8) /1024 = 10