Forum
Welcome, Guest
Username: Password: Remember me

TOPIC:

ha-cfg commands not working after slave fenced 7 years 2 months ago #1133

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 722
Can you replace /etc/init.d/iscsi-ha and /etc/init.d/iscsi-ha-watchdog with these:

halizard.org/release/misc/init_test/iscsi-ha
halizard.org/release/misc/init_test/iscsi-ha-watchdog

Steps in case you need them:

cd /etc/init.d
wget halizard.org/release/misc/init_test/iscsi-ha
wget halizard.org/release/misc/init_test/iscsi-ha-watchdog
chmod +x iscsi-ha*

Rerun your test and let me know if this solves the issue with DRBD starting too early.

Please Log in or Create an account to join the conversation.

ha-cfg commands not working after slave fenced 7 years 2 months ago #1134

  • Andrew Foster
  • Andrew Foster's Avatar Topic Author
  • Offline
  • Posts: 15
I'm afraid the changes didn't make any difference. :(

Please Log in or Create an account to join the conversation.

ha-cfg commands not working after slave fenced 7 years 2 months ago #1137

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 722
Can you search the log file around the time of the reboot for this line "DRBD not running - attempted start failed".

If xensever 7, log is /var/log/user.log
if xenserver 6, log is /var/log/messages

Let me know if you see this. thanks

Please Log in or Create an account to join the conversation.

ha-cfg commands not working after slave fenced 7 years 2 months ago #1139

  • Andrew Foster
  • Andrew Foster's Avatar Topic Author
  • Offline
  • Posts: 15
Found the error in the logs:
[root@xen-02 log]# grep -C 10 "DRBD not running - attempted start failed" user.log*
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 local_ip_list: Local IP list returned 127.0.0.1
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 service_execute: Execute [ status ] on [ tgtd ]
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 service_execute: systemctl mode being used
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 service_execute: ● tgtd.service - tgtd iSCSI target daemon#012   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled; vendor preset: disabled)#012  Drop-In: /etc/systemd/system/tgtd.service.d#012           └─local.conf#012   Active: inactive (dead)
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 service_execute: Returning exit status [ 3 ]
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 iSCSI target: tgtd status stopped. Expected Stopped . [inactive (dead)]
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 DRBD not running - attempting start
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: /etc/drbd.conf:19: in resource iscsi1, on xen-02:
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: #011IP 10.10.10.2 not found on this host.
user.log.3:Jan  6 09:27:29 xen-02 iscsi-ha: 3820 DRBD not running - attempted start failed
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: Command 'drbdsetup-84 role 1' terminated with exit code 10
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 check_drbd_resource_state: Error retrieving DRBD state for resouce: iscsi1 - check configuration
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha: 3820 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.3-Jan  6 09:27:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor
--
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 local_ip_list: Local IP list returned 127.0.0.1
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 service_execute: Execute [ status ] on [ tgtd ]
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 service_execute: systemctl mode being used
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 service_execute: ● tgtd.service - tgtd iSCSI target daemon#012   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled; vendor preset: disabled)#012  Drop-In: /etc/systemd/system/tgtd.service.d#012           └─local.conf#012   Active: inactive (dead)
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 service_execute: Returning exit status [ 3 ]
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 iSCSI target: tgtd status stopped. Expected Stopped . [inactive (dead)]
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 DRBD not running - attempting start
user.log.3-Jan  6 09:46:28 xen-02 iscsi-ha: 4363 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: /etc/drbd.conf:19: in resource iscsi1, on xen-02:
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: #011IP 10.10.10.2 not found on this host.
user.log.3:Jan  6 09:46:29 xen-02 iscsi-ha: 4363 DRBD not running - attempted start failed
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha: 4363 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: Command 'drbdsetup-84 role 1' terminated with exit code 10
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha: 4363 check_drbd_resource_state: Error retrieving DRBD state for resouce: iscsi1 - check configuration
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha: 4363 email: iteration: 1 is less than threshold - email suppressed
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.3-Jan  6 09:46:29 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor
--
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 local_ip_list: Local IP list returned 127.0.0.1
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 service_execute: Execute [ status ] on [ tgtd ]
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 service_execute: systemctl mode being used
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 service_execute: ● tgtd.service - tgtd iSCSI target daemon#012   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled; vendor preset: disabled)#012  Drop-In: /etc/systemd/system/tgtd.service.d#012           └─local.conf#012   Active: inactive (dead)
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 service_execute: Returning exit status [ 3 ]
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 iSCSI target: tgtd status stopped. Expected Stopped . [inactive (dead)]
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 DRBD not running - attempting start
user.log.5-Jan  4 09:30:20 xen-02 iscsi-ha: 4372 email: iteration: 1 is less than threshold - email suppressed
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: /etc/drbd.conf:19: in resource iscsi1, on xen-02:
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: #011IP 10.10.10.2 not found on this host.
user.log.5:Jan  4 09:30:21 xen-02 iscsi-ha: 4372 DRBD not running - attempted start failed
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha: 4372 email: iteration: 1 is less than threshold - email suppressed
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: Command 'drbdsetup-84 role 1' terminated with exit code 10
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha: 4372 check_drbd_resource_state: Error retrieving DRBD state for resouce: iscsi1 - check configuration
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha: 4372 email: iteration: 1 is less than threshold - email suppressed
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: 1: Failure: (127) Device minor not allocated
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: additional info from kernel:
user.log.5-Jan  4 09:30:21 xen-02 iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: unknown minor

Please Log in or Create an account to join the conversation.

ha-cfg commands not working after slave fenced 7 years 2 months ago #1146

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 722
thanks. If you can wait a few days, I can have something for you to test that would better handle a failed start of DRBD.

Please Log in or Create an account to join the conversation.

ha-cfg commands not working after slave fenced 7 years 1 month ago #1182

  • Andrew Foster
  • Andrew Foster's Avatar Topic Author
  • Offline
  • Posts: 15
Any progress with this?

Please Log in or Create an account to join the conversation.