ep 10 16:57:37 xs2 ha-lizard: 16103 check_xapi: Pool Host 192.168.199.200 failed to respond to ICMP ping
Sep 10 16:57:37 xs2 ha-lizard: 16103 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 16:57:37 xs2 ha-lizard: 16103 email: Sending ALERT email to root@localhost: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to ICMP ping - manual intervention may be required
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 16:57:37 EDT 2015
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: check_xapi:  Pool Host on Server: 192.168.199.200 not responding to ICMP ping - manual intervention may be required
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 16:57:37 xs2 ha-lizard: 16103 email: Message copied to /dev/shm/ha-lizard-mail/1441918657.msg - Suppressing duplicates for 60 Minutes
Sep 10 16:57:37 xs2 ha-lizard: 16103 Retrieving number of hosts in pool. Setting NUM_HOSTS = 1
Sep 10 16:57:37 xs2 ha-lizard: 16103 Failed to reach Pool Master - Checking if this host promotes to Master..
Sep 10 16:57:37 xs2 ha-lizard: 16103 State file MASTER UUID = 26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f
Sep 10 16:57:37 xs2 ha-lizard: 16656 fence_host: fence_host called to fencehost:  26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f with action: stop and method: POOL. Check Pool!
Sep 10 16:57:37 xs2 ha-lizard: 16656 Mail Spool Directory Found /dev/shm/ha-lizard-mail
Sep 10 16:57:37 xs2 ha-lizard: 16656 email: Sending ALERT email to root@localhost: fence_host:  fence_host called to fencehost:  26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f with action: stop and method: POOL. Check Pool!
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email from: root@localhost
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh: Traceback (most recent call last):
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:   File "/etc/ha-lizard/scripts/email_alert.py", line 104, in ?
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Sending email to: root@localhost
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:     message = smtplib.SMTP(smtp_server, smtp_port, hostname)
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Subject: SYSTEM_ALERT-FROM_HOST:xs2
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 244, in __init__
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Timestamp: Thu Sep 10 16:57:37 EDT 2015
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:     (code, msg) = self.connect(host, port)
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Process: ha-lizard Version: 1.8.7
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:   File "/usr/lib64/python2.4/smtplib.py", line 306, in connect
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Content: fence_host:  fence_host called to fencehost:  26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f with action: stop and method: POOL. Check Pool!
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh:     raise socket.error, msg
Sep 10 16:57:37 xs2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: Email Alert Message Hostname: xs2
Sep 10 16:57:37 xs2 ha-lizard-ERROR-/etc/ha-lizard/ha-lizard.sh: socket.error: (111, 'Connection refused')
Sep 10 16:57:37 xs2 ha-lizard: 16656 email: Message copied to /dev/shm/ha-lizard-mail/1441918657.msg - Suppressing duplicates for 60 Minutes
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Checking host IPs: 192.168.199.159
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Host IP: 192.168.199.159 Response = OK
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: LIVE HOSTs = 1
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Using network points: 192.168.199.1 as possible additional vote
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Heuristic IP: 192.168.199.1 Response = OK
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Successful Replies = 1
Sep 10 16:57:37 xs2 ha-lizard: 16656 Total enpoints checked = 1 with total successful replies = 1
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Additional heuristic vote success. Incremeting vote by 1
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: Minimum number of hosts needed to allow fencing = 0 + 1
Sep 10 16:57:37 xs2 ha-lizard: 16656 check_quorum: 2 Hosts found. Minimum needed = 0 + 1. Fencing allowed
Sep 10 16:57:37 xs2 ha-lizard: 16656 fence_host: Fencing is disabled or blocked. Check configuration or pool state. FENCE_ENABLED = 1, FENCE_BLOCK = , NUM_HOSTS = 1, FENCE_MIN_HOSTS = 2 - cannot fence host 26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f
Sep 10 16:57:37 xs2 ha-lizard: 16103 Function fence_host returned status 1
Sep 10 16:57:37 xs2 ha-lizard: 16103 Failed to fence Master: 26c15cda-7a26-41e8-a2f7-cdd7a9d5df8f. Checking whether FENCE_HA_ONFAIL is enabled
Sep 10 16:57:37 xs2 ha-lizard: 16103 Marking this host as fenced, Rebooting this host now!
Sep 10 16:57:37 xs2 ha-lizard: 16103 !!!!!!!!!!!!!!!!!!!!!!!!!!!!! SELF FENCING - REBOOT HERE !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!