Monthly Archives: April 2017

ESXi 6.5 Host Initiator Unable to Connect to OpenMediaVault 2.x Guest iSCSI Target

I ran into a problem when trying to have an OpenMediaVault 2.x Guest machine act as an iSCSI target for an ESXi 6.5 Host iSCSI initiator. The goal was to have OMV share a datastore for the ESXi host using the iSCSI protocol.

To create iSCSI targets on OMV, the easiest way is to install the available plugin (openmediavault-iscsitarget) from the web GUI. However a major limitation with this plugin is that you are not able to create fileio targets using the web GUI. In another post I detail how to get around that limitation but at this point we’ll assume you have successfully created a target (either fileio or blockio) on OMV and are now trying to get an ESXi host to discover and connect to it.

You may find that ESXi successfully discovers the OMV target but then fails to connect to it. I happened to be logged in to the console of the OMV guest at the time and saw several console error messages appear when ESXi was attempting to connect to the OMV target. They were similar to the below. You may need to reboot OMV and then try to rediscover the target in ESXi in order to reproduce them.

To flush the errors out I ended up having to reboot the OMV guest machine. The root cause appears to be a bugged version of iscsitarget that is installed from the OMV repositories when running the Linux 3.2 kernel. The solution is to install a backports kernel for OMV and then install updated iscsitarget packages that the OMV maintainers have made available.

The steps:

  1. Back up / make note of any iSCSI initiator settings in ESXi
  2. Remove the iSCSI configuration from the ESXi host
  3. Reboot OMV
  4. Back up / make note of any iSCSI target configuration settings you have configured for OMV
  5. Uninstall the iSCSI plugin (openmediavault-iscsitarget) from OMV using the web GUI
  6. Reboot OMV
  7. Log in to the OMV console
  8. Run several commands from the OMV console to fully remove and flush out the iSCSI configuration. One or more of these may not be necessary depending on your configuration:
    apt-get purge openmediavault-iscsitarget
    apt-get purge iscsitarget
    apt-get purge iscsitarget-dkms
    apt-get autoremove
    apt-get autoclean
  9. Install the OMV-Extras.org Plugin. This has to be downloaded and installed manually since it is not available by default within the web GUI. The official guide:
    http://forum.openmediavault.org/index.php/Thread/5549-OMV-Extras-org-Plugin/
  10. Navigate to the plugin configuration page in the web GUI and confirm that the OMV-Extras.org main repository is enabled.
  11. Install the 3.16 backports kernel.  At this point I configured it to be the default for OMV. The official guide:
    http://forum.openmediavault.org/index.php/Thread/8584-Install-Backports-Kernel/
  12. Reboot so that you are running 3.16 kernel
  13. Now reinstall the openmediavault-iscsitarget plugin. With the 3.16 kernel running, the plugin will automatically use the newer packages of iscsitarget available from the OMV-Extras.org repository (noted below for reference only).
    http://omv-extras.org/debian/pool/main/i/iscsitarget/
  14. At this point you should then be able to configure the iSCSI target on OMV and have ESXi successfully connect to it.

How to Utilize iSCSI fileio with OpenMediaVault 2.x

OMV (version 2.x) has an iscsitarget plugin available (openmediavault-iscsitarget), however the web interface only supports BlockIO type of targets and I was looking to implement FileIO.

After installing the plugin, it is possible to manually configure the OMV plugin for a fileio target using the  console by following the steps below. I ran into trouble when trying to access this target with ESXi which I cover in a separate article. If you are trying to access an iSCSI target with Microsoft Windows as an initiator, this method may work for you.

  1. Install openmediavault-iscsitarget plugin
  2. Enable the plugin in the web interface and confirm that it is running
  3. Access the console (locally, via ssh, etc.)
  4. Create an image file to hold your data (LUN). For OMV, Shared Folders are located at
    /media/your-volume-uuid/shared-folder-name
    . For example, to create a 1TB “thin” image using dd and the count parameter:
    dd if=/dev/zero of=/path/to/your/thinvolume.img bs=1M seek=1048576 count=0
  5. Edit the IET config file using the command:
    nano /etc/iet/ietd.conf
  6. In this configuration file you will need to add your target information. Add the following lines to the bottom of this file, modifying it as appropriate for your configuration.
    Target iqn.yyyy-mm.com.example:hostname.Target-1
            IncomingUser username password
            OutgoingUser
            Lun 0 Path=/path/to/your/thinvolume.img,Type=fileio
            Alias LUN1
  7. Restart iscsi-target with this console command
  8. Then use this command to confirm it is running
  9. Then try to connect to the ISCSI target running OMV either by IP address or by IQN name.

Bind iSCSI to a Single Network Card on OpenMediaVault

If you have multiple network cards for OMV (such as using one for a dedicated storage network), you may be looking to force OMV to assign all iSCSI traffic to one network card. A word of caution – if you are using the openmediavault-iscsitarget plugin, then following these steps may break the web interface for this plugin when upgrading or removing the plugin, since OMV will complain about this file having been modified.  It is possible to run iscsitarget without the plugin (which is recommended if you are looking to set up a filetypeio configuration), and I cover how to do that in another post.

To bind all iSCSI traffic (IET) to a single network card:

  1. Access the console
  2. Edit the configuration file for iscsitarget:
  3. Find the options line:
    ISCSITARGET_OPTIONS=””
    and change it to contain your IP address as follows (NOTE: you must use a double dash):
    ISCSITARGET_OPTIONS=”–address x.x.x.x″
    where x.x.x.x is the IP address of the network card in OMV corresponding to your dedicated storage network.
  4. Restart iscsitarget

Unable to Remove USB Controller from ESXi 6.5 Virtual Machine

Ran into a situation where an “x” was not available for removing a USB Controller from a Guest.

To resolve, first shut down the guest (if running) and temporarily Unregister it from the host. Then from the datastore, download the .vmx file associated with the guest. Create a backup of the .vmx file, then edit the original with a text editor such as Notepad. Find and delete all of the lines that begin with “usb.”  For example:

usb.present = “TRUE”
usb.pciSlotNumber = “32”
usb:0.present = “TRUE”
usb:0.deviceType = “hid”
usb:0.port = “0”
usb:0.parent = “-1”
usb:1.speed = “2”
usb:1.present = “TRUE”
usb:1.deviceType = “hub”
usb:1.port = “1”
usb:1.parent = “-1”

Once these lines are removed from the .vmx file, save it and then upload the file back to the datastore, overwriting the original.

The final step is to re-register the guest with the host by pointing to the vmx file we just updated on the datastore. You should then find that the USB Controller is gone from the guest configuration.

Unable to Connect to a MySQL Server Running on Debian Linux

If you have a fresh install of a mysql server running on Debian (e.g., after running apt-get install mysql-server) you may find that any mysql client connections from other computers to your Debian server fail (one popular example is Kodi / XBMC, etc.). You may also find that mysql connections made from Debian itself (such as a local WordPress database install) are working fine.

One quick test is to use a mysql client on another linux machine to try to connect to the Debian mysql server. From the console of the other linux machine, use the following command (replace the x.x.x.x with the IP address of the Debian server). In this example we are trying to connect with the root user.

You may then get something like:

ERROR 2003: Can’t connect to MySQL server on ‘x.x.x.x’ (111 “Connection refused”)

By default, mysql server on Debian will bind itself to the loopback address (127.0.0.1). This setting can cause all remote connections to the server to be blocked with Error 111, even though all local connections will work fine (such as a WordPress install noted earlier). As a comparison, you can test local connections by issuing the above command directly from the Debian console, which should yield an error such as:

ERROR 1045: Access denied for user ‘root’@’localhost’ (using password: NO)

This basically confirms local connections are hitting the Debian mysql server. To enable remote connections for Debian’s mysql server, modify the mysql configuration file using these steps:

  1. Find the line that says
    bind-address     127.0.0.1
  2. and comment it out by inserting a hash in front of it
    #bind-address     127.0.0.1
  3. Save and close the my.cnf file.
  4. Then restart the mysql server:

Then try connecting to the Debian mysql server from another computer.