TOPIC:

We've started to migrate our virtual machines from a standalone server to HAL enabled pool. HA is enabled, iscsi-cfg indicates connection is made between both master/slave and that all is up to date.

One of the virtual machines transferred over has a filesystem corruption. It's a centos 7 based virtual machine. When we switch the VM on we are immediately given a screen (after choosing the OS):

Metadata corruption detected at xfs_inode_buf_verify+.......
Corruption of in-memory data detected. Shutting down filesystem
Please umount the filesystem and rectify the problem(s)

The VM only allows for emergency mode.

Any advice? We would not want other VM's to experience the same so hoping this can be fixed?

A 2nd VM has also failed booting. If we migrate it to the master server we're immediately prompted "hard disk failure" and no other boot options. If we shutdown the server (and it restarts on slave) it appears to boot fine, however, no network to the VM (which is unrelated)

If I check iscsi-cfg status there is no connection issue and both drbd devices are up to date. I also have 3 other VM's running on master at the moment which appears unaffected.

I'm going to postpone on moving VM's to the server until I have better clarity as to what may cause this and how to fix it afterwards

3rd VM in some corrupted form - What is strange is I am sure some of these VM's did work after the migration to the HAL pool.

I've attempted to move the VM's back to the origin standalone server. The filesystems are still broken.

I really hope to get feedback on this as all of our communication servers are currently down.

The VM's also does not get removed from the HAL pool. When I move it to the standalone server, the VM appears to run on both, however, the HAL pool VM is stuck in the pool.

The transferred VM is also irresponsive, and when I attempted to reboot the transferred VM I get a Internal error: xenopsd internal error: Storage_interface.Internal_error("Storage_access.No_VDI") - When I go to the VM storage there is none.

This is horrible.

I've now tried to move a working VM from the HAL pool back to the standalone. In this particular case the VM transfer was complete, came up at the standalone server and almost immediately DISSAPEARED. On the HAL pool the VM is still there but the storage is missing.

We have about 9 other VM's still on the HAL storage which I now also cannot move away as it appears to delete the storage. HAL is running, iscsi is running. I am also not able to start the VM on the other pool member.

At this stage I want to part ways with HAL but cannot as our data is now stuck in pool that appears to corrupt the individual VM's in it. Please please help.