We have recently encountered issues with Veeam replication jobs where they would fail with the following errors:
Error: Failed to open VDDK disk [[DATASTORENAME_0] PATH_0] ( is read-only mode – [true] ) –tr:{ –tr:VDDK error: 1 (Unknown error). Value: 0x0000000000000001 –tr:Failed to open virtual disk [DATASTORENAME_0] PATH_0 (flags: 4) –tr:Failed to execute VDDK command ‘Open’
In this scenario, the replications were using Linux Veeam virtual hot-add appliance backup proxies. When we checked the Veeam backup proxies there were hot-add snapshot disks still attached from the replicas after the jobs had completed. This was locking those disks as read-only and preventing Veeam from being able to take additional snapshots and perform the replications.
Here are two possible solutions:
SSHD.conf file configurations
These issues can be caused by timeouts and disconnects that can be resolved by tuning the SSH configuration on the proxy. To make these adjustments modify the /etc/ssh/sshd_config file and add the following lines:
ClientAliveInterval 300
TCPKeepAlive yes
ClientAliveCountMax 99999
MaxSessions 200
MaxStartups 1000:30:2000
You may need to restart the sshd service or restart the machine for these changes to take effect.
Disabling multipathing
In the Session.log log file there may be errors similar to the following:
Error Failed to get guest OS path for newly attached disk [Datastore] VM_replicaNew/VM_1-000003.vmdk: there are 2 new disks with uuid 00000000-0000-0000-0000-000000000000 (System.Exception)
These point to issues that can happen on linux systems where multipath is enabled and hot-add is used.
These commands will disable multipathing, they will also reboot the system so do not attempt this while jobs are running:
sudo systemctl disable multipathd
sudo systemctl disable multipath-tools
sudo reboot
Veeam has more information on this issue here: KB4460: Failed to get guest OS path for newly attached disk there are 2 new disks with uuid (veeam.com)
In our case we made both changes and our issues so far have not returned.