[root@xs2 ~]# cat out
Sep 10 11:25:03 xs2 xapi: [ info|xs2|502 UNIX /var/xapi/xapi|session.login_with_password D:406e19999915|xapi] Session.create trackid=0e5f63e092e60d01cc417709b5c6930f pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:03 xs2 xapi: [ info|xs2|503 UNIX /var/xapi/xapi|session.login_with_password D:b0ec0beaf4e8|xapi] Session.create trackid=1a1b208b46516168879421ffd6a79d22 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:03 xs2 ha-lizard: 14401 get_pool_host_list: enabled flag set - returning only hosts with enabled=true
Sep 10 11:25:03 xs2 xapi: [ info|xs2|504 UNIX /var/xapi/xapi|session.login_with_password D:53257bb9a31c|xapi] Session.create trackid=a4329fee4cc8516d0d958603bc826e6f pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:03 xs2 ha-lizard: 14401 get_pool_host_list: returned 26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f 79769bcb-ad49-4ce1-8b27-51234c5b418f
Sep 10 11:25:03 xs2 xapi: [ info|xs2|505 UNIX /var/xapi/xapi|session.login_with_password D:1df40e0511ae|xapi] Session.create trackid=5067300e9962044dd6e24611870d6aca pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:03 xs2 ha-lizard: 14401 get_pool_ip_list: returned 192.168.199.200
Sep 10 11:25:03 xs2 xapi: [ info|xs2|506 UNIX /var/xapi/xapi|session.login_with_password D:a32f95c588e9|xapi] Session.create trackid=f3433db184c6516e5fe576beb1dc643f pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:03 xs2 ha-lizard: 14401 get_pool_ip_list: returned 192.168.199.200 192.168.199.159
Sep 10 11:25:05 xs2 ha-lizard: 3926 ha-lizard Watchdog: ha-lizard running - OK 
Sep 10 11:25:07 xs2 iscsi-ha: 4418 iscsi-ha Watchdog: iscsi-ha running - OK 
Sep 10 11:25:08 xs2 iscsi-ha: 14270 Spawning new instance of iscsi-ha
Sep 10 11:25:08 xs2 iscsi-ha: 14270 check_logger_processes Checking logger processes
Sep 10 11:25:08 xs2 iscsi-ha: 14270 check_logger_processes No processes to clear
Sep 10 11:25:08 xs2 iscsi-ha: 14737 Mail Spool Directory Found /dev/shm/iscsi-ha-mail
Sep 10 11:25:08 xs2 iscsi-ha: 14737 This iteration is count 21
Sep 10 11:25:08 xs2 iscsi-ha: 14737 Checking if this host is a Pool Master or Slave
Sep 10 11:25:08 xs2 iscsi-ha: 14737 This host's pool status = slave:192.168.199.200
Sep 10 11:25:09 xs2 iscsi-ha: 14771 local_ip_list: Local IP list returned 127.0.0.1  192.168.199.159  10.10.10.2
Sep 10 11:25:09 xs2 xapi: [ info|xs2|508 UNIX /var/xapi/xapi|session.login_with_password D:fb3ad578e591|xapi] Session.create trackid=76182a1031caca36f1c0a36aebae8127 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:09 xs2 xapi: [ info|xs2|509 UNIX /var/xapi/xapi|session.login_with_password D:b7e0a32a3064|xapi] Session.create trackid=8e1c227d745853b9b7abcd102d46d211 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:09 xs2 iscsi-ha: 14771 iSCSI target: /etc/init.d/tgtd status stopped. Expected Stopped . [tgtd is stopped]
Sep 10 11:25:09 xs2 iscsi-ha: 14771 DRBD Running on this host: version: 8.4.3 (api:1/proto:86-101) srcversion: 19422058F8A2D4AC0C8EF09    1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----     ns:0 nr:12292 dw:12292 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Sep 10 11:25:09 xs2 iscsi-ha: 14771 check_drbd_resource_state: DRBD Resource: iscsi1 in Secondary mode
Sep 10 11:25:09 xs2 iscsi-ha: 14771 DRBD Resource: iscsi1 in Connected state
Sep 10 11:25:09 xs2 iscsi-ha-14732-INFO-/etc/iscsi-ha/iscsi-ha.sh: Scanning for Volume Group -> iscsi-sr: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:09 xs2 iscsi-ha-14732-INFO-/etc/iscsi-ha/iscsi-ha.sh: Volume Group for iSCSI-SR found OK: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:15 xs2 ha-lizard: 3926 ha-lizard Watchdog: ha-lizard running - OK 
Sep 10 11:25:16 xs2 ha-lizard: 14393 Spawning new instance of ha-lizard
Sep 10 11:25:16 xs2 ha-lizard: 14870 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:16 xs2 ha-lizard: 14870 This iteration is count 15
Sep 10 11:25:16 xs2 ha-lizard: 14870 Checking if this host is a Pool Master or Slave
Sep 10 11:25:16 xs2 ha-lizard: 14870 This host's pool status = slave:192.168.199.200
Sep 10 11:25:16 xs2 xapi: [ info|xs2|510 UNIX /var/xapi/xapi|session.login_with_password D:0aba1777e989|xapi] Session.create trackid=582b27129377d230eaea482564636410 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:16 xs2 xapi: [ info|xs2|511 UNIX /var/xapi/xapi|session.login_with_password D:83e306758d08|xapi] Session.create trackid=7d152e15f6d356139307c7156f75775c pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:16 xs2 ha-lizard: 14870 Successfully updated global pool configuration settings in /etc/ha-lizard/ha-lizard.pool.conf. Settings take effect on subsequent run
Sep 10 11:25:16 xs2 ha-lizard: 14870 DISABLED_VAPPS=() ENABLE_LOGGING=1 FENCE_ACTION=stop FENCE_ENABLED=1 FENCE_FILE_LOC=/etc/ha-lizard/fence FENCE_HA_ONFAIL=0 FENCE_HEURISTICS_IPS=192.168.199.1 FENCE_HOST_FORGET=0 FENCE_IPADDRESS= FENCE_METHOD=POOL FENCE_MIN_HOSTS=2 FENCE_PASSWD= FENCE_QUORUM_REQUIRED=1 FENCE_REBOOT_LONE_HOST=0 FENCE_USE_IP_HEURISTICS=1 GLOBAL_VM_HA=1 MAIL_FROM="root@localhost" MAIL_ON=1 MAIL_SUBJECT="SYSTEM_ALERT-FROM_HOST:$HOSTNAME" MAIL_TO="root@localhost" MONITOR_DELAY=15 MONITOR_KILLALL=1 MONITOR_MAX_STARTS=20 MONITOR_SCANRATE=10 OP_MODE=2 PROMOTE_SLAVE=1 SLAVE_HA=1 SLAVE_VM_STAT=0 SMTP_PASS="" SMTP_PORT="25" SMTP_SERVER="127.0.0.1" SMTP_USER="" XAPI_COUNT=2 XAPI_DELAY=10 XC_FIELD_NAME='ha-lizard-enabled' XE_TIMEOUT=10
Sep 10 11:25:16 xs2 ha-lizard: 14870 master_ip: Pool Master IP Address =  192.168.199.200
Sep 10 11:25:16 xs2 ha-lizard: 14870 This slave - xs2: 79769bcb-ad49-4ce1-8b27-51234c5b418f selected as allowed to become master: setting ALLOW_PROMOTE_MASTER=1
Sep 10 11:25:16 xs2 ha-lizard: 14870 check_xapi: Pool Host 192.168.199.200 xapi status = 1
Sep 10 11:25:16 xs2 ha-lizard: 14870 Pool Master is OK - calling function check_ha_enabled - updating local status
Sep 10 11:25:16 xs2 xapi: [ info|xs2|512 UNIX /var/xapi/xapi|session.login_with_password D:6c2c94052d3a|xapi] Session.create trackid=14dc0f32346cbaf39964804f6fd05a61 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 xapi: [ info|xs2|513 UNIX /var/xapi/xapi|session.login_with_password D:3c918151df6b|xapi] Session.create trackid=fa0f4effbb410d6ca9d06538952d8f73 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 ha-lizard: 14870 check_ha_enabled: Checking if ha-lizard is enabled for pool: 3243bef3-b686-d02f-0d53-9e9529ee9639
Sep 10 11:25:17 xs2 ha-lizard: 14870 check_ha_enabled: ha-lizard is enabled
Sep 10 11:25:17 xs2 ha-lizard: 14870 Checking state file for status if ha-lizard is enabled
Sep 10 11:25:17 xs2 ha-lizard: 14870 Statefile /etc/ha-lizard/state/ha_lizard_enabled found: checking if ha-lizard is enabled
Sep 10 11:25:17 xs2 ha-lizard: 14870 ha-lizard is enabled - continuing
Sep 10 11:25:17 xs2 ha-lizard: 14870 check_xs_ha: Checking XenServer HA status
Sep 10 11:25:17 xs2 xapi: [ info|xs2|514 UNIX /var/xapi/xapi|session.login_with_password D:51c874261f0a|xapi] Session.create trackid=f050290425c0c27649a7a529f64aa150 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 xapi: [ info|xs2|515 UNIX /var/xapi/xapi|session.login_with_password D:a8d51b9136df|xapi] Session.create trackid=79d2786f76986c7ad6bb0bfcc5b80c44 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 ha-lizard: 14870 Calling Function write_pool_state - updating local state files
Sep 10 11:25:17 xs2 iscsi-ha: 4418 iscsi-ha Watchdog: iscsi-ha running - OK 
Sep 10 11:25:17 xs2 xapi: [ info|xs2|516 UNIX /var/xapi/xapi|session.login_with_password D:7fa5eef2836a|xapi] Session.create trackid=c351749ad240ca3e9736833b9a5451b7 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 xapi: [ info|xs2|517 UNIX /var/xapi/xapi|session.login_with_password D:05eb4f2dce26|xapi] Session.create trackid=8922941b2cc4499844fc12a4cc4c34a9 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 xapi: [ info|xs2|518 UNIX /var/xapi/xapi|session.login_with_password D:0a02e72fee84|xapi] Session.create trackid=8f6bb937eae16031a96ae756df1ec4e2 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:17 xs2 ha-lizard: 14870 get_vms_on_host: No VMs found on host: 26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f
Sep 10 11:25:17 xs2 xapi: [ info|xs2|519 UNIX /var/xapi/xapi|session.login_with_password D:35f6b75ecc61|xapi] Session.create trackid=646dafab0a57a93dac96071d248797b0 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 ha-lizard: 14870 get_vms_on_host: No VMs found on host: 79769bcb-ad49-4ce1-8b27-51234c5b418f
Sep 10 11:25:18 xs2 xapi: [ info|xs2|520 UNIX /var/xapi/xapi|session.login_with_password D:75724e5b117f|xapi] Session.create trackid=4da36e2c45a3513c962d6cc1d69589db pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 xapi: [ info|xs2|521 UNIX /var/xapi/xapi|session.login_with_password D:6bee2df0477f|xapi] Session.create trackid=b76f55d8846f36a4966b3f8955bf03b1 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 xapi: [ info|xs2|522 UNIX /var/xapi/xapi|session.login_with_password D:1eb941fc4022|xapi] Session.create trackid=8926c7c950244cd4429a86ca309744be pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 xapi: [ info|xs2|523 UNIX /var/xapi/xapi|session.login_with_password D:0da1a9537ac5|xapi] Session.create trackid=1953f10d909cb8c16c79056eb75a6a2e pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 ha-lizard: 14870 check_ha_enabled: Checking if ha-lizard is enabled for pool: 3243bef3-b686-d02f-0d53-9e9529ee9639
Sep 10 11:25:18 xs2 ha-lizard: 14870 check_ha_enabled: ha-lizard is enabled
Sep 10 11:25:18 xs2 xapi: [ info|xs2|524 UNIX /var/xapi/xapi|session.login_with_password D:d152507d79af|xapi] Session.create trackid=265cd8efa186cee84fc7d0741557d5f5 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 xapi: [ info|xs2|525 UNIX /var/xapi/xapi|session.login_with_password D:bcfeb2b56dbf|xapi] Session.create trackid=61f8f7e2f9aaf83cc87ac44828005f9f pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 ha-lizard: 14870 get_pool_host_list: enabled flag set - returning only hosts with enabled=true
Sep 10 11:25:18 xs2 xapi: [ info|xs2|526 UNIX /var/xapi/xapi|session.login_with_password D:83941827ef80|xapi] Session.create trackid=aa2207ee7793402cc32fa3d24b7b1647 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 ha-lizard: 14870 get_pool_host_list: returned 79769bcb-ad49-4ce1-8b27-51234c5b418f
Sep 10 11:25:18 xs2 xapi: [ info|xs2|527 UNIX /var/xapi/xapi|session.login_with_password D:7a069e6e6dab|xapi] Session.create trackid=2d1b522d70186ed77289c9e5920466fa pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:18 xs2 ha-lizard: 14870 get_pool_ip_list: returned 192.168.199.159
Sep 10 11:25:18 xs2 iscsi-ha: 14729 Spawning new instance of iscsi-ha
Sep 10 11:25:18 xs2 iscsi-ha: 14729 check_logger_processes Checking logger processes
Sep 10 11:25:19 xs2 iscsi-ha: 14729 check_logger_processes No processes to clear
Sep 10 11:25:19 xs2 iscsi-ha: 15162 Mail Spool Directory Found /dev/shm/iscsi-ha-mail
Sep 10 11:25:19 xs2 iscsi-ha: 15162 This iteration is count 22
Sep 10 11:25:19 xs2 iscsi-ha: 15162 Checking if this host is a Pool Master or Slave
Sep 10 11:25:19 xs2 iscsi-ha: 15162 This host's pool status = slave:192.168.199.200
Sep 10 11:25:19 xs2 iscsi-ha: 15200 local_ip_list: Local IP list returned 127.0.0.1  192.168.199.159  10.10.10.2
Sep 10 11:25:19 xs2 xapi: [ info|xs2|528 UNIX /var/xapi/xapi|session.login_with_password D:4732c163f41d|xapi] Session.create trackid=3b69252d41802212ad1bd6282c7a32e5 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:19 xs2 xapi: [ info|xs2|529 UNIX /var/xapi/xapi|session.login_with_password D:ad48a388bc7a|xapi] Session.create trackid=0bad42adadc5168bb7329a38f0d54fbf pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:19 xs2 iscsi-ha: 15200 iSCSI target: /etc/init.d/tgtd status stopped. Expected Stopped . [tgtd is stopped]
Sep 10 11:25:19 xs2 iscsi-ha: 15200 DRBD Running on this host: version: 8.4.3 (api:1/proto:86-101) srcversion: 19422058F8A2D4AC0C8EF09    1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----     ns:0 nr:12292 dw:12292 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Sep 10 11:25:19 xs2 iscsi-ha: 15200 check_drbd_resource_state: DRBD Resource: iscsi1 in Secondary mode
Sep 10 11:25:19 xs2 iscsi-ha: 15200 DRBD Resource: iscsi1 in Connected state
Sep 10 11:25:19 xs2 iscsi-ha-15157-INFO-/etc/iscsi-ha/iscsi-ha.sh: Scanning for Volume Group -> iscsi-sr: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:19 xs2 iscsi-ha-15157-INFO-/etc/iscsi-ha/iscsi-ha.sh: Volume Group for iSCSI-SR found OK: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:25 xs2 ha-lizard: 3926 ha-lizard Watchdog: ha-lizard running - OK 
Sep 10 11:25:27 xs2 kernel: [  258.226775]  connection1:0: detected conn error (1020)
Sep 10 11:25:27 xs2 iscsi-ha: 4418 iscsi-ha Watchdog: iscsi-ha running - OK 
Sep 10 11:25:28 xs2 iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Sep 10 11:25:29 xs2 iscsi-ha: 15154 Spawning new instance of iscsi-ha
Sep 10 11:25:29 xs2 iscsi-ha: 15154 check_logger_processes Checking logger processes
Sep 10 11:25:29 xs2 iscsi-ha: 15154 check_logger_processes No processes to clear
Sep 10 11:25:29 xs2 iscsi-ha: 15325 Mail Spool Directory Found /dev/shm/iscsi-ha-mail
Sep 10 11:25:29 xs2 iscsi-ha: 15325 This iteration is count 23
Sep 10 11:25:29 xs2 iscsi-ha: 15325 Checking if this host is a Pool Master or Slave
Sep 10 11:25:29 xs2 iscsi-ha: 15325 This host's pool status = slave:192.168.199.200
Sep 10 11:25:29 xs2 iscsi-ha: 15359 local_ip_list: Local IP list returned 127.0.0.1  192.168.199.159  10.10.10.2
Sep 10 11:25:29 xs2 xapi: [ info|xs2|531 UNIX /var/xapi/xapi|session.login_with_password D:75712f87dc66|xapi] Session.create trackid=a4771a50b20f7e58b52a31b5af93b516 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:29 xs2 xapi: [ info|xs2|532 UNIX /var/xapi/xapi|session.login_with_password D:d3b728a53451|xapi] Session.create trackid=bd5962f4c89fd7f3145c251dbc825b26 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep 10 11:25:29 xs2 iscsi-ha: 15359 iSCSI target: /etc/init.d/tgtd status stopped. Expected Stopped . [tgtd is stopped]
Sep 10 11:25:29 xs2 iscsi-ha: 15359 DRBD Running on this host: version: 8.4.3 (api:1/proto:86-101) srcversion: 19422058F8A2D4AC0C8EF09    1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----     ns:0 nr:12292 dw:12292 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Sep 10 11:25:29 xs2 iscsi-ha: 15359 check_drbd_resource_state: DRBD Resource: iscsi1 in Secondary mode
Sep 10 11:25:29 xs2 iscsi-ha: 15359 DRBD Resource: iscsi1 in Connected state
Sep 10 11:25:30 xs2 iscsid: connect to 10.10.10.3:3260 failed (Connection refused) 
Sep 10 11:25:30 xs2 xapi: [ warn|xs2|192|xapi events D:03bab00dde34|xmlrpc_client] stunnel pid: 4566 caught Http_client.Http_request_rejected("{ frame = false; method = POST; uri = /; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.1; cookie = [  ]; task = ; subtask_of = DummyRef:|03bab00d-de34-b955-8a0f-144bf8e0b799|xapi events; content-type = ; host = ; user_agent = xen-api-libs/1.0 }")
Sep 10 11:25:30 xs2 xapi: [error|xs2|192|xapi events D:03bab00dde34|stunnel] get_reusable_stunnel: fresh stunnel failed reusable check; delaying 10.00 seconds before reconnecting to 192.168.199.200:443 (attempt 1 / 10)
Sep 10 11:25:31 xs2 ha-lizard: 14862 Spawning new instance of ha-lizard
Sep 10 11:25:31 xs2 ha-lizard: 15438 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:31 xs2 ha-lizard: 15438 This iteration is count 16
Sep 10 11:25:31 xs2 ha-lizard: 15438 Checking if this host is a Pool Master or Slave
Sep 10 11:25:31 xs2 ha-lizard: 15438 This host's pool status = slave:192.168.199.200
Sep 10 11:25:31 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:31 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:33 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15477 fd=20
Sep 10 11:25:33 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15481 caught Http_client.Http_request_rejected("{ frame = false; method = POST; uri = /; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [  ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = xen-api-libs/1.0 }")
Sep 10 11:25:33 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:33 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:34 xs2 iscsi-ha-15320-INFO-/etc/iscsi-ha/iscsi-ha.sh: Scanning for Volume Group -> iscsi-sr: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:34 xs2 iscsi-ha-15320-INFO-/etc/iscsi-ha/iscsi-ha.sh: Volume Group for iSCSI-SR found OK: 4a13ab89-f5b5-e2e4-978b-73127607eae4
Sep 10 11:25:34 xs2 iscsid: connection1:0 is operational after recovery (2 attempts)
Sep 10 11:25:34 xs2 ha-lizard: 15438 Failed to update global pool configuration settings in /etc/ha-lizard/ha-lizard.pool.conf - Check Configuration!
Sep 10 11:25:34 xs2 ha-lizard: 15438 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:34 xs2 ha-lizard: 15438 email: Sending ALERT email to root@localhost: Failed to update global pool configuration settings in /etc/ha-lizard/ha-lizard.pool.conf - Check Configuration!
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 11:25:34 EDT 2015
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: Failed to update global pool configuration settings in /etc/ha-lizard/ha-lizard.pool.conf - Check Configuration!
Sep 10 11:25:34 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 11:25:34 xs2 ha-lizard: 15438 email: Message copied to /dev/shm/ha-lizard-mail/1441898734.msg - Suppressing duplicates for 60 Minutes
Sep 10 11:25:34 xs2 ha-lizard: 15438 master_ip: Pool Master IP Address =  192.168.199.200
Sep 10 11:25:34 xs2 ha-lizard: 15438 This slave - xs2: 79769bcb-ad49-4ce1-8b27-51234c5b418f selected as allowed to become master: setting ALLOW_PROMOTE_MASTER=1
Sep 10 11:25:34 xs2 ha-lizard: 15438 check_xapi: Pool Host 192.168.199.200 xapi status = 0
Sep 10 11:25:34 xs2 ha-lizard: 15438 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:35 xs2 ha-lizard: 15438 email: Sending ALERT email to root@localhost: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to HTTP - manual intervention may be required
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 11:25:35 EDT 2015
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to HTTP - manual intervention may be required
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 11:25:35 xs2 ha-lizard: 15438 email: Message copied to /dev/shm/ha-lizard-mail/1441898735.msg - Suppressing duplicates for 60 Minutes
Sep 10 11:25:35 xs2 ha-lizard: 15438 Pool Master NOT OK - Checking if ha-lizard is enabled in latest state file
Sep 10 11:25:35 xs2 ha-lizard: 15438 Checking if ha-lizard is enabled
Sep 10 11:25:35 xs2 ha-lizard: 15438 Statefile /etc/ha-lizard/state/ha_lizard_enabled found: checking if ha-lizard is enabled
Sep 10 11:25:35 xs2 ha-lizard: 15438 ha-lizard is enabled - continuing
Sep 10 11:25:35 xs2 ha-lizard: 15438 Pool Master Monitor = Failed
Sep 10 11:25:35 xs2 ha-lizard: 15438 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:35 xs2 ha-lizard: 15438 email: Sending ALERT email to root@localhost: Server xs2: Failed to contact pool master - manual intervention may be required
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 11:25:35 EDT 2015
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: Server xs2: Failed to contact pool master - manual intervention may be required
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 11:25:35 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 11:25:35 xs2 ha-lizard: 15438 email: Message copied to /dev/shm/ha-lizard-mail/1441898735.msg - Suppressing duplicates for 60 Minutes
Sep 10 11:25:35 xs2 ha-lizard: 15438 Retry Count set to 2. Retrying 2 times in 10 second intervals..
Sep 10 11:25:35 xs2 ha-lizard: 15438 Attempt 0: Checking Pool Master Status
Sep 10 11:25:35 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15563 fd=20
Sep 10 11:25:35 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15566 caught Http_client.Http_request_rejected("{ frame = false; method = POST; uri = /; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [  ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = xen-api-libs/1.0 }")
Sep 10 11:25:35 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:35 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:35 xs2 ha-lizard: 3926 ha-lizard Watchdog: ha-lizard running - OK 
Sep 10 11:25:37 xs2 iscsi-ha: 4418 iscsi-ha Watchdog: iscsi-ha running - OK 
Sep 10 11:25:37 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15581 fd=20
Sep 10 11:25:39 xs2 iscsi-ha: 15317 Spawning new instance of iscsi-ha
Sep 10 11:25:39 xs2 iscsi-ha: 15317 check_logger_processes Checking logger processes
Sep 10 11:25:39 xs2 iscsi-ha: 15317 check_logger_processes No processes to clear
Sep 10 11:25:39 xs2 iscsi-ha: 15627 Mail Spool Directory Found /dev/shm/iscsi-ha-mail
Sep 10 11:25:39 xs2 iscsi-ha: 15627 This iteration is count 24
Sep 10 11:25:39 xs2 iscsi-ha: 15627 Checking if this host is a Pool Master or Slave
Sep 10 11:25:39 xs2 iscsi-ha: 15627 This host's pool status = slave:192.168.199.200
Sep 10 11:25:39 xs2 iscsi-ha: 15661 local_ip_list: Local IP list returned 127.0.0.1  192.168.199.159  10.10.10.2
Sep 10 11:25:39 xs2 iscsi-ha: 15661 iSCSI target: /etc/init.d/tgtd status stopped. Expected Stopped . [tgtd is stopped]
Sep 10 11:25:39 xs2 iscsi-ha: 15661 DRBD Running on this host: version: 8.4.3 (api:1/proto:86-101) srcversion: 19422058F8A2D4AC0C8EF09    1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----     ns:0 nr:12292 dw:12292 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Sep 10 11:25:39 xs2 iscsi-ha: 15661 check_drbd_resource_state: DRBD Resource: iscsi1 in Secondary mode
Sep 10 11:25:39 xs2 iscsi-ha: 15661 DRBD Resource: iscsi1 in Connected state
Sep 10 11:25:45 xs2 ha-lizard: 15438 check_xapi: Pool Host 192.168.199.200 xapi status = 0
Sep 10 11:25:45 xs2 ha-lizard: 15438 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 11:25:45 xs2 ha-lizard: 15438 email: Sending ALERT email to root@localhost: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to HTTP - manual intervention may be required
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 11:25:45 EDT 2015
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to HTTP - manual intervention may be required
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 11:25:45 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 11:25:45 xs2 ha-lizard: 15438 email: Message copied to /dev/shm/ha-lizard-mail/1441898745.msg - Suppressing duplicates for 60 Minutes
Sep 10 11:25:45 xs2 ha-lizard: 15438 Attempt 1: Checking Pool Master Status
Sep 10 11:25:45 xs2 kernel: [  276.854310]  connection1:0: detected conn error (1020)
Sep 10 11:25:45 xs2 ha-lizard: 3926 ha-lizard Watchdog: ha-lizard running - OK 
Sep 10 11:25:46 xs2 iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Sep 10 11:25:46 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15584 caught Http_client.Http_request_rejected("{ frame = false; method = POST; uri = /; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [  ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = xen-api-libs/1.0 }")
Sep 10 11:25:46 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Http_client.Http_request_rejected("{ frame = true; method = POST; uri = /remote_db_access; query = [  ]; content_length = [ 193 ]; transfer encoding = ; version = 1.1; cookie = [ pool_secret=dcaa39b3-f754-0b65-554e-2fef74d6fed3/319120ee-f602-fd6e-2fa3-b6d58dafd7b4/025572fc-ed80-4bf7-9211-e482ea22f24f ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = xen-api-libs/1.0 }")
Sep 10 11:25:46 xs2 xapi: [error|xs2|192|xapi events D:03bab00dde34|stunnel] get_reusable_stunnel: fresh stunnel failed reusable check; delaying 10.00 seconds before reconnecting to 192.168.199.200:443 (attempt 2 / 10)
Sep 10 11:25:46 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:46 xs2 ha-lizard: 15430 ha-lizard already running: Attempt 1 on PIDS: 15438 15433 15432 15430
Sep 10 11:25:47 xs2 iscsi-ha: 4418 iscsi-ha Watchdog: iscsi-ha running - OK 
Sep 10 11:25:48 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15781 fd=20
Sep 10 11:25:48 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15785 caught Xmlrpc_client.Connection_reset
Sep 10 11:25:48 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:48 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:49 xs2 iscsid: connect to 10.10.10.3:3260 failed (Connection refused) 
Sep 10 11:25:49 xs2 iscsi-ha: 15619 iscsi-ha already running: Attempt 1 on PIDS: 15627 15622 15621
Sep 10 11:25:50 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15807 fd=23
Sep 10 11:25:50 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15810 caught Xmlrpc_client.Connection_reset
Sep 10 11:25:50 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:50 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:52 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15819 fd=23
Sep 10 11:25:52 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15823 caught Xmlrpc_client.Connection_reset
Sep 10 11:25:52 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:52 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:53 xs2 iscsid: connect to 10.10.10.3:3260 failed (Connection refused) 
Sep 10 11:25:54 xs2 iscsi-ha: 15619 iscsi-ha already running: Attempt 2 on PIDS: 15627 15622 15621
Sep 10 11:25:54 xs2 xapi: [ info|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] stunnel connected pid=15851 fd=23
Sep 10 11:25:54 xs2 xapi: [ warn|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|xmlrpc_client] stunnel pid: 15855 caught Xmlrpc_client.Connection_reset
Sep 10 11:25:54 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Caught Unix.Unix_error(31, "write", "")
Sep 10 11:25:54 xs2 xapi: [error|xs2|533 UNIX /var/xapi/xapi|session.login_with_password D:d0e8f295f1a0|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message).
Sep 10 11:25:55 xs2 ha-lizard: 15438 check_xapi: Pool Host 192.168.199.200 xapi status = 0