OMV (version 2.x) has an iscsitarget plugin available (openmediavault-iscsitarget), however the web interface only supports BlockIO type of targets and I was looking to implement FileIO.

After installing the plugin, it is possible to manually configure the OMV plugin for a fileio target using the  console by following the steps below. I ran into trouble when trying to access this target with ESXi which I cover in a separate article. If you are trying to access an iSCSI target with Microsoft Windows as an initiator, this method may work for you.

1. Install openmediavault-iscsitarget plugin
2. Enable the plugin in the web interface and confirm that it is running
3. Access the console (locally, via ssh, etc.)
4. Create an image file to hold your data (LUN). For OMV, Shared Folders are located at
   /media/your-volume-uuid/shared-folder-name. For example, to create a 1TB “thin” image using dd and the count parameter:
   dd if=/dev/zero of=/path/to/your/thinvolume.img bs=1M seek=1048576 count=0
   
5. Edit the IET config file using the command:
   nano /etc/iet/ietd.conf
6. In this configuration file you will need to add your target information. Add the following lines to the bottom of this file, modifying it as appropriate for your configuration.
   Target iqn.yyyy-mm.com.example:hostname.Target-1
           IncomingUser username password
           OutgoingUser
           Lun 0 Path=/path/to/your/thinvolume.img,Type=fileio
           Alias LUN1
7. Restart iscsi-target with this console command

   /etc/init.d/iscsitarget restart

8. Then use this command to confirm it is running

   /etc/init.d/iscsitarget status

9. Then try to connect to the ISCSI target running OMV either by IP address or by IQN name.

If you have multiple network cards for OMV (such as using one for a dedicated storage network), you may be looking to force OMV to assign all iSCSI traffic to one network card. A word of caution – if you are using the openmediavault-iscsitarget plugin, then following these steps may break the web interface for this plugin when upgrading or removing the plugin, since OMV will complain about this file having been modified.  It is possible to run iscsitarget without the plugin (which is recommended if you are looking to set up a filetypeio configuration), and I cover how to do that in another post.

To bind all iSCSI traffic (IET) to a single network card:

1. Access the console
2. Edit the configuration file for iscsitarget:

   nano /etc/default/iscsitarget

3. Find the options line:
   ISCSITARGET_OPTIONS=””
   and change it to contain your IP address as follows (NOTE: you must use a double dash):
   ISCSITARGET_OPTIONS=”–address x.x.x.x″
   where x.x.x.x is the IP address of the network card in OMV corresponding to your dedicated storage network.
4. Restart iscsitarget

   /etc/init.d/iscsitarget restart

Ran into a situation where an “x” was not available for removing a USB Controller from a Guest.

To resolve, first shut down the guest (if running) and temporarily Unregister it from the host. Then from the datastore, download the .vmx file associated with the guest. Create a backup of the .vmx file, then edit the original with a text editor such as Notepad. Find and delete all of the lines that begin with “usb.”  For example:

usb.present = “TRUE”
usb.pciSlotNumber = “32”
usb:0.present = “TRUE”
usb:0.deviceType = “hid”
usb:0.port = “0”
usb:0.parent = “-1”
usb:1.speed = “2”
usb:1.present = “TRUE”
usb:1.deviceType = “hub”
usb:1.port = “1”
usb:1.parent = “-1”

Once these lines are removed from the .vmx file, save it and then upload the file back to the datastore, overwriting the original.

The final step is to re-register the guest with the host by pointing to the vmx file we just updated on the datastore. You should then find that the USB Controller is gone from the guest configuration.

If you have a fresh install of a mysql server running on Debian (e.g., after running apt-get install mysql-server) you may find that any mysql client connections from other computers to your Debian server fail (one popular example is Kodi / XBMC, etc.). You may also find that mysql connections made from Debian itself (such as a local WordPress database install) are working fine.

One quick test is to use a mysql client on another linux machine to try to connect to the Debian mysql server. From the console of the other linux machine, use the following command (replace the x.x.x.x with the IP address of the Debian server). In this example we are trying to connect with the root user.

mysql -h x.x.x.x -u root

You may then get something like:

ERROR 2003: Can’t connect to MySQL server on ‘x.x.x.x’ (111 “Connection refused”)

By default, mysql server on Debian will bind itself to the loopback address (127.0.0.1). This setting can cause all remote connections to the server to be blocked with Error 111, even though all local connections will work fine (such as a WordPress install noted earlier). As a comparison, you can test local connections by issuing the above command directly from the Debian console, which should yield an error such as:

ERROR 1045: Access denied for user ‘root’@’localhost’ (using password: NO)

This basically confirms local connections are hitting the Debian mysql server. To enable remote connections for Debian’s mysql server, modify the mysql configuration file using these steps:

sudo nano /etc/mysql/my.cnf

1. Find the line that says
   bind-address     127.0.0.1
2. and comment it out by inserting a hash in front of it
   #bind-address     127.0.0.1
3. Save and close the my.cnf file.
4. Then restart the mysql server:

sudo /etc/init.d/mysql restart

Then try connecting to the Debian mysql server from another computer.

After installing open-vm-tools, ESXi reported that VMware Tools were Installed and running on the Debian 8 guest. However any attempt to gracefully power off the guest within ESXi would fail with these errors:

[msg.tools.haltFailed] The request to Power off this virtual machine failed because the corresponding VMware Tools script did not run successfully.

Also present in the logs was this error:

msg.tools.poweronFailed] The VMware Tools power-on script did not run successfully in this virtual machine. If you have configured a custom power-on script in this virtual machine, make sure that it contains no errors.

For this scenario, the solution was to first uninstall open-vm-tools. For Debian / Ubuntu:

sudo apt-get remove open-vm-tools

then install open-vm-tools-desktop

sudo apt-get update
sudo apt-get install open-vm-tools-desktop

You should then be able to power off and power on the guest within ESXi.

Ran into a second error after swapping out a pass-through device in ESXi. This device was the same make and model as the previous device.

Failed - Module 'DevicePowerOn' power on failed.

* Module 'DevicePowerOn' power on failed.
* Missing pciPassthru0.id entry in the configuration file.
* Failed to start the virtual machine.

To resolve it, try these steps:

1. Completely remove the old and/or new pass-through device from the Guest configuration
2. Save the Guest configuration
3. Confirm you are able to successfully boot the Guest without any pass-through device
4. After the Guest starts, shut it down
5. Add the new pass-through device to the Guest configuration.
6. Save the Guest configuration
7. You should now be able to boot it successfully with the new pass-through device

Failed - Module 'DevicePowerOn' power on failed.

* Module 'DevicePowerOn' power on failed.
* The systemId does not match the current system or the deviceId, and the vendorId does not match the device currently at 1:0.0.
* Failed to start the virtual machine.

To resolve, first shut down the guest (if running without the pass through device) and temporarily Unregister it from the host. Then from the datastore, download the .vmx file associated with the guest. Create a backup of the .vmx file, then edit the original with a text editor such as Notepad. Find and delete the following lines:

Note: the old pass through device ID of 01:00.0" was specific to my configuration as noted in the original error message, so yours may differ. I’ve also cleared out all unique device information below with xxxxx since it will be based on your specific configuration and is not relevant.

pciPassthru0.id = "01:00.0"
pciPassthru0.deviceId = "xxx"
pciPassthru0.vendorId = "xxx"
pciPassthru0.systemId = "xxxxxxxxxxxxx"

Once these lines are removed from the .vmx file, save it and then upload the file back to the datastore, overwriting the original.

The final step is to re-register the guest with the host by pointing to the vmx file we just updated on the datastore. You should then be able to add the new pass through device and power on the guest.

During the install of FreeNAS, the process may hang with the following errors:

UNMAP failed, disabling BIO_DELETE
UNMAP. CDB ...
CAM status: SCSI Status Error
SCSI status: Check Condition
SCSI sense: ILLEGAL REQUEST ... (Invalid field in parameter list)
Command byte 0 is invalid
Error 22, Unretryable error

This was with both FreeNAS-9.10.1-U2 and FreeNAS-9.10.2-U1 (though it is unlikely that this specific incompatibility is isolated to only these two builds). One thing to try is to first shut down the guest and then reconfigure the VM. For the Hard Disk configuration options of the VM, the default setting for Virtual Device Node is SCSI Controller. Changing it to SATA Controller resolved the error and FreeNAS was able to then install successfully.

I ran into a situation where the host had crashed because of a driver problem (vmw_ahci). After changing the driver and rebooting the host, one of the guest VMs then failed to power on with the following errors:

• File system specific implementation of LookupAndOpen[file] failed
• Object type requires hosted I/O
• Cannot open the disk ‘…vmdk’ or one of the snapshot disks it depends on.
• Module ‘Disk’ power on failed.
• Failed to start the virtual machine.

The solution was to repair the disk using vmkfstools. This particular VM was split into two files, vmname.vmdk and vmname-s001.vmdk. You will want to perform the commands on the primary (pointer) vmdk (not vmname-s###.vmdk).
From the ESXi console, issue the following command substituting your actual path:

vmkfstools -x check /path/to/vmname.vmdk

In this case vmkfstools reported that the disk should be repaired. It would be prudent at this point to back up your vmdk files before trying the repair. Then initiate the repair on the primary vmdk with:

vmkfstools -x repair /path/to/vmname.vmdk

Once you get a notification that the repair has completed, try to start your VM again.

Upgrading from ESXi 5.5 to 6.5, performance of the host was relatively poor on a JetWay JNF9G-QM77. Guests were extremely sluggish.

There were also several errors  tracked in the various logs related to the boot drive and the datastore associated with the boot drive.

In the vobd log:

[vmfsCorrelator]: [esx.problem.vmfs.heartbeat.timedout] datastore1
[vmfsCorrelator]: [vob.vmfs.heartbeat.recovered] Reclaimed heartbeat for volume (datastore1): [Timeout] 
[vmfsCorrelator]: [esx.problem.vmfs.heartbeat.recovered] datastore1

In the vmkwarning log:

WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "" state in doubt; requested fast path state update...

In the vmauthd log:

ScsiDeviceIO ... CmdSN 0x4305 from world 69114 to dev "" failed ... Invalid sense data ...

The solution was to disable the new AHCI driver of ESXi 6.5 (vmw_ahci). From the ESXi console issue the following command:

esxcli system module set --enabled=false --module=vmw_ahci

Then after a reboot ESXi should be using the sata-ahci driver. You can confirm this by checking the list of storage adapters for the host, looking for one titled Panther Point AHCI Controller.
After this driver swap, performance of the host should be restored.