Loss of datastore and unable to passthrough after upgrading to ESXi 7

When an X58 + ICH10 based machine (Dell Precision T3500) was upgraded to ESXi 7, the storage adapter and associated datastore was lost.  The ICH10 in this case was not playing well with the native vmw_ahci driver.  In AHCI mode, the adapter shows up as a 6 port controller in vSphere (Host -> Manage ->  Hardware of PCI devices. However none of the attached drives appear under Storage.  Also attempting to toggle PCI passthrough for the device results in an error:

An error occurred during host configuration.  Operation failed, diagnostics report: GetDeviceID failed.

Refreshing vSphere would then show as the host needing a reboot for it to take effect. However upon rebooting passthrough is still  disabled / not active.

To resolve, the SATA adapter was switched from AHCI to IDE / ATA mode in the BIOS.  This allowed the controller and all attached drives to appear in vSphere by using the native vmkata driver.  This change will also passthrough to be enabled, but it requires a reboot.

However another issue was that a VMFS datastore on the drive attached to this controller was no longer accessible.  To resolve, the datastore had to be force mounted. These steps were used to manually mount the datastore:

  1. In the console / shell:
    esxcli storage vmfs snapshot list
  2. Check fore an output similar to this:Volume Name: datastore1
    VMFS UUID: <snip>
    Can mount: true
    Reason for un-mountability:
    Can resignature: true
    Reason for non-resignaturability:
    Unresolved Extent Count: 1
  3. Determine the Volume Name, e.g., (datastore1), this acts as the label
  4. Mount it using the label. In this example, datastore1:
    esxcli storage vmfs snapshot mount -l datastore1
  5. Refresh the Storage section of vSphere

Note: performance in ATA / IDE mode may be reduced compared to AHCI

 

Posted in Uncategorized | Leave a comment

Force Upgrade of VMware ESXi with Unsupported Processor

When trying to upgrade an ESXi host using the console, I received the error below. In this case I was attempting to upgrade to version 7 Update 3k.

[HardwareError]
Hardware precheck of profile ESXi-7.0U3k-21313628-standard failed with warnings: <CPU_SUPPORT WARNING: The CPU in this host is not supported by ESXi 7.0.3. Please refer to the VMware Compatibility Guide (VCG) for the list of supported CPUs.>

The error will prevent the upgrade from continuing. However, you can force the upgrade of a system with a legacy processor using these steps:

  1. In the root file system of ESXi, edit /bootbank/boot.cfg using vi
  2. On the line that starts with kernelopt=, add the following to the end: allowLegacyCPU=true
  3. Save the file and reboot the host
  4. Then attempt the upgrade again (e.g.,  using esxcli software profile update)
Posted in Uncategorized | Leave a comment

RMCARD Firmware Upgrade

These steps can be used to upgrade the firmware and data for an RMCARD205 using Windows 10.  They may also work with an RMCARD305.

There are two files used to upgrade the device:

  1. cpsrm2scfw_XXX.bin
  2. cpsrm2scdata_XXX.bin

First make sure FTP is enabled on the RMCARD. Log in to the web interface and check under System ->Network Service -> FTP Service.

The steps to upgrade the device using the FTP Service:

  1. Download the latest firmware
  2. Extract the downloaded files to “C:\”
  3. Open a command prompt window
  4. Type ftp
  5. Type open
  6. Type [current IP address of the RMCARD, e.g., 192.168.1.120]
  7. Provide the administrator username and password used for the web interface
  8. Type bin
  9. Type put C:\cpsrm2scfw_XXX.bin (replace XXX filename with what was extracted in step 2).
  10. Wait until upload is complete and it gives you a summary of number of bytes sent and the speed of the transfer.
  11. Type quit
  12. If you receive the message 500 Update Failed: Upload incorrect file or timeout, then repeat steps 4 through 10. Perform step 11 as soon as the upload is complete.
  13. Once successful, reload the web interface. You should see a message displayed similar to:  Data version (vX.X.X) and firmware version(vX.X.X) do not match, please update again.
  14. Repeat steps 4 through 8
  15. Type put C:\cpsrm2scdata_XXX.bin
  16. Type quit
  17. Reload the web interface. You can log in and check Firmware Version and Firmware Update Date under System -> About
Posted in Uncategorized | Leave a comment

IKEv2 VPN Suddenly Stops Working with Authentication Error

Updated to be compatible with the newer pfSense release (2.7)

With a previously working IKEv2 configuration on pfSense, you may suddenly start receiving these messages:

iOS: User Authentication Failed
Windows 10: IKE authentication credentials are unacceptable

The reason in this case was related to the certificate.  The first step is to log in to pFsense webConfigurator, then verify the certificate is still valid using the Certificate Manager and the Valid Until date.

If the certificate is valid and recently renewed, it could be that the IPsec service still has the previous certificate in memory. This can be verified by running a shell command. In the webConfigurator, choose Diagnostics then Command Prompt. Execute the following command and look for the validity dates of your certificate:

ipsec listcerts

If it has the expired certificate, a quick fix is simply stopping and starting the IPsec service. Go to VPN->IPsec->Disable then Enable the tunnel, applying changes each time.  You can also simply reboot the firewall.

If you want to automate the restart of IPsec you can use cron:

In PfSense Package Manager, install cron.  Then go to Services -> cron -> and Add a new job. For example, a job that restarts the service on the first day of every month:

0 0 1 * * root /usr/local/sbin/strongswanrc restart | logger 2>&1

Notes:

Attempting to use this command (/usr/local…logger 2>&1) in a console session will result in one of the errors below depending on your pfSense version,  and the certificate will not get updated. However, it will work as intended if you configure it as a cron job per the above.
charon is already running (/var/run/charon.pid exists) — skipping daemon start
strongswan already running?

If you have Service Watchdog enabled and watching the ipsec service, you’ll see the following 2 lines in the System Log shortly after the service is stopped.
servicewatchdog_cron.php: Service Watchdog detected service ipsec stopped. Restarting ipsec (IPsec VPN)
servicewatchdog_cron.php: Forcefully reloading IPsec

In pfSense versions 2.5 and earlier, this cron job used to work, but in version 2.7 and higher it now returns the error: Ambiguous output redirect.

0 0 1 * * root /usr/local/sbin/strongswanrc restart 2>&1 | logger &

 

 

Posted in Uncategorized | Leave a comment

Remote Desktop Connection Fails for Local Account on Domain Computer

I tried saving an RDP file with a prepopulated username to connect to a remote server using a local account on that server.  The local machine was an Azure AD joined to the same domain as the remote server.

Typically the username would be .\localuser or servername\localuser however in both cases the prefix is stripped out and the username shows up as simply localuser.  If you supply the correct password you get the error: The logon attempt failed

If you click the More choices button, you discover the username is now appended @domain.com.  If you choose the option for Use a different account, then add back the prefix from above the connection will be successful.  However I wanted the RDP file to do this step automatically.

The solution is to modify the RDP file using a text editor.

Find and modify the line username:s:value to be similar to the following:

username:s:servername\.\localuser

Then save and try the RDP file again. The username should correctly prepopulated with the prefix as .\localuser

 

 

Posted in Uncategorized | Leave a comment

Rename Windows 10 User Account Username and Profile Folder

This is a quick guide for changing both the login username and profile folder for a user account in Windows 10.

  1. Log in with an account that has administrator rights and is not the one you want to modify
  2. Computer Management -> Local Users and Groups ->Users
  3. Right click and rename the desired user
  4. Navigate to C:\Users
  5. Right click and rename the folder of the user account
  6. Open Registry Editor and locate the following:
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList
  7. Under ProfileList there are several keys corresponding to SIDs of various user accounts. Inspect each one, looking at the value of ProfileImagePath.  This stores the location of the profile folder, and one will have a value that matches what was renamed in step 4.
  8. Once located, right click on ProfileImagePath and choose Modify.
  9. Provide the new folder path and click OK
  10. Reboot to flush out any remaining references
Posted in Uncategorized | Leave a comment

ERROR 1064 (42000) when installing phpMyAdmin

Attempting to install phpMyAdmin 5.1.1 on Ubuntu 18.04 (Bionic Beaver) from the PPA linked below, I received error 1064.

https://launchpad.net/~phpmyadmin/+archive/ubuntu/ppa

An error occurred while installing the database: 
mysql said: ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'IDENTIFIED BY 'PASSWORD'' at line 1 . Your options are:

These are the steps I used to complete the install.

  1. When the error is reached, copy the password shown. Then abort the install.
  2. At the console, access mysql:
    sudo mysql -u root -p
  3. Delete the user the installer tried to create
    DROP USER ‘phpmyadmin’@’localhost’;
  4. Create the user (using the same password as the installer), the database and set the permissions
    CREATE USER ‘phpmyadmin’@’localhost’ IDENTIFIED BY ‘PASSWORD’;
    CREATE DATABASE phpmyadmin;
    GRANT ALL PRIVILEGES ON *.* TO ‘phpmyadmin’@’localhost’ WITH GRANT OPTION;
    FLUSH PRIVILEGES;
    exit
  5. Go back and try to reinstall. When the error is reached, choose ignore. This should complete the install.
  6. Access phpMyAdmin web interface and login with username phpmyadmin and the password from step 4.
  7. You will see the error
    The phpMyAdmin configuration storage is not completely configured, some extended features have been deactivated. Find out why.
    
    Or alternately go to 'Operations' tab of any database to set it up there.
  8. Click on Find out why
  9. At the top, choose the option for: Create missing phpMyAdmin configuration storage tables. 
  10. Once complete, phpMyAdmin should now be working and without any errors.

Note: If you later change the password for the phpmyadmin account, you will start getting the following error message:

mysqli::real_connect(): (HY000/1045): Access denied for user 'phpmyadmin'@'localhost' (using password: YES)
Connection for controluser as defined in your configuration failed.

To clear the error, you will need to update the file below with the same password:
/etc/phpmyadmin/config-db.php

Posted in Uncategorized | Leave a comment

Veeam 11 Error When Editing Backup Job After Upgrade to ESXi Host

After upgrading the ESXi 7.0 host (Update 3), I was receiving the following error in Veeam Backup and Replication 11  (11.0.1.1261) when trying to edit an existing Backup Job:

Host with id ‘a-b-c-d-e’ was not found

The solution was to modify the Veeam database for a guest VM to point to the correct host ID.

  1. Using Microsoft SQL Server Management Studio, navigate to the dbo.BObjects table
    Databases -> VeeamBackup -> Tables -> dbo.BObjects
  2. Right click and Edit Top 200 Rows
  3. Locate the id from the error message in the host_id column
  4. Replace it with the correct host_id (with dashes).  Note – you can get a list of all host IDs by using PowerShell in a Veeam B&R console session with the command Get-VBRServer.  If needed you can also crosscheck the guest VM by matching the object_id column in the database with the Vmid of the VM on the host.
Posted in Uncategorized | Leave a comment

Unable to delete Time Machine Shared Folder in OMV 5

In OpenMediaVault the button may be greyed out if the Shared Folder is still Referenced. References can include file shares but also plugins.  In this case there was a Shared Folder that had been used for Time Machine and after removing the SMB share and verifying plugins, the Delete button was still greyed out.

First you can try to verify what is referencing the shared folder in this OMV file.
/etc/openmediavault/config.xml

Find the shared folder in question and note the UUID. Then search for that UUID in the same file.

In this case I had an afp section with data between the tags <afp> and </afp>. It had the the UUID stored as <sharedfolderref>UUID</sharedfolderref>.

The afp section was used by the the old netatalk plugin that was in OMV 4 but is no longer part of OMV 5 (the system had been upgraded).  However it was not possible to uninstall the plugin since it didn’t appear in the list of available packages.

Alternatively the plugin can be uninstalled via command line. Close the config.xml file then run the console command:

sudo dpkg -P openmediavault-netatalk

Once uninstalled, you can go back to the config.xml file and verify that the afp section was removed automatically.

Refresh the OMV control panel -> Access Rights Management -> Shared Folders. The folder in question should no longer be Referenced and the delete button no longer grayed out.

Posted in Uncategorized | Leave a comment

Set a Quota for Time Machine with SMB on OMV 5.6

This details an OpenMediaVault configuration where an SMB share was created and Time Machine support was enabled for it.  There was a need to limit the size of the Time Machine backups so that it does not consume all available space, and to do that you use the filesystem quotas:

  1. In OpenMediaVault navigate to Storage->File Systems
  2. Select the device that has the Time Machine shared folder
  3. Click the Quota button at the top. 
  4. Note – without any quotas set, Used Capacity for users will show zero even if data is already there.
  5. Set a quota value for the user associated with Time Machine.
  6. Click the grey save button
  7. Click the blue save button
  8. Apply changes
  9. If you click the Quota button again, you can verify that Used Capacity is populating for that user (if data is already in the shared folder).
Posted in Uncategorized | Leave a comment