ESXi 6.5 fails to Discover or Connect to a Synology iSCSI Target with CHAP Authentication

If you are having trouble getting ESXi to discover a Synology iSCSI target with CHAP authentication, check your vobd log for errors that may look similar to the below:

[iscsiCorrelator]: [vob.iscsi.discovery.login.error] discovery failure on vmhba64 to x.x.x.x because the target returned a login status of 0201.
[esx.problem.storage.iscsi.discovery.login.error] iSCSI discovery to x.x.x.x on vmhba64 failed. The Discovery target returned a login error of: 0201.

This basically confirms some kind of problem negotiating the CHAP authentication. After double checking that you are using the correct CHAP credentials and you are still not able to get the ESXi host to discover the Synology target, try the steps below:

  1. In the Synology Storage Manager for iSCSI Target, first disable Mutual CHAP (if enabled) and check that CHAP is enabled.
    Then in ESXi Configure iSCSI as follows:
  2. Set only a Dynamic Target using the IP address and port of the Synology target. Remove any Static Target that you may have set.
  3. Temporarily set CHAP Authentication to “Do not use CHAP”
  4. Set Mutual CHAP Authentication to “Do not use CHAP”
  5. Click Save Configuration
  6. Click Refresh on the Devices tab and confirm the Synology is not being discovered
  7. Now go back the iSCSI configuration in ESXi
  8. Set CHAP Authentication back to “Use CHAP”
  9. Click the down arrow and enter your CHAP credentials
  10. Confirm the Static Target is now populated with the Synology target address information. If not, try populating it now.
  11. Click Save Configuration
  12. Click Refresh on the Devices tab

The Synology target should then show up in ESXi.

 

Posted in Uncategorized | Leave a comment

ESXi Upgrade from Version 5.5 to 6.5 Fails

During the upgrade process from 5.5 to 6.5 (using the ISO method), I ran into the following error:

CONFLICTING_VIBS ERROR: Vibs on the host are conflicting with vibs in metadata. 
Remove the conflicting VIBs or use Image Builder to create a custom upgrade ISO 
providing newer versions of the conflicting vibs.

The vib listed as the problem was HUAWEI_bootbank_hio_2.0.0.42-1OEM.550.0.1331820

The next step was to cancel the upgrade process and reboot back into 5.5. Once you are back into ESXi, bring up the console and list all installed vibs by using the following command (pipe in the more command to break the list into pages):

esxcli software vib list | more

In this particular case the problematic vib was called hio

To remove it, initiate the following command:

esxcli software vib remove --vibname hio

Once it completes, reboot for the removal to cleanly take effect. Then reboot again and boot off your ISO installer to restart the ESXi upgrade process.

Posted in Uncategorized | Leave a comment

CentOS 7 Fails to Boot – XFS Corruption – Enters Emergency Mode

In the process of flushing yum and trying to get CentOS to update, the system froze and I had to do a hard reset. Upon rebooting, the OS failed to load with several error messages:

XFS: Internal error XFS_WANT_CORRUPTED_GOTO at line 1757 of file fs/xfs/xfs_alloc.c.
...
Corruption of in-memory data detected. Shutting down filesystem
Please unmount the filesystem and rectify the problem(s)
Failed to recover EFIs
task mount:365 blocked for more than 120 seconds.

Looks like some file system corruption and the last error about the system trying to perform a mount just kept repeating. Running xfs_repair in the console did nothing in this environment.

Next I tried booting from a CentOS install ISO and I chose the option to Rescue an existing installation. Choosing any of the 3 options in rescue mode would just result it the system hanging indefinitely without ever reaching a console. I chose Option 3 to skip to console and alt tabbing revealed:

Running ... mount -t xfs -o defaults,ro /dev/mapper/centos-root /mnt/sysimage

Now it is clear that the system is hanging on trying to mount the root partition which we know has problems. However I could not find a way to enter the console without rescue mode automatically trying to mount the root partition.

Next I tried booting a CentOS live ISO. Turns out it has no xfs tools so that was a dead end.

I decided to try booting a SystemRescueCd which has some xfs tools on it.

I chose the default boot option and reached a console. Ideally one should create a dump of the damaged partition by running xfs_metadump, restore the dump to an image using xfs_mdrestore, and then perform the repair on that image. That way you can evaluate the repair done to the image to see if it will work or irreversibly damage your actual data.

I went ahead and just did the repair on the actual data:

xfs_repair -L /dev/mapper/centos-root

The repair completed and after a reboot CentOS started successfully.

Posted in Uncategorized | 4 Comments

WordPress Gmail SMTP Plugin – Emails Don’t Send and No Error Displayed

When sending a test email, it was hanging on the authentication method:

Auth method requested: XOAUTH2
Auth methods available on the server: 
LOGIN,PLAIN,XOAUTH2,PLAIN-CLIENTTOKEN,OAUTHBEARER,XOAUTH

Installing curl resolved it. For Debian and Ubuntu:

apt-get install curl libcurl3 php5-curl

 

Posted in Uncategorized | Leave a comment