Forum
Welcome, Guest
Username: Password: Remember me
This is the optional category header for the Suggestion Box.
  • Page:
  • 1

TOPIC:

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2856

  • Christoph Christ
  • Christoph Christ's Avatar Topic Author
  • Offline
  • Posts: 9
Hi guys

I would like to know. how I can fix a probably broken ha-lizard installation. (all defaults). When will be the next version released? My version is 2.2.3-1.

I had installed ha-lizard 2,5 years ago, and everything was working nicely. But I have encounterd several really big problems with my cluster in the last 5 weeks. The slave is sending queries with wrong password to the master - see the backtrace in the log below. Does anybody now how to debug that. so that I can fix it? If the xapi is queried to fast. it starts to create big timeouts. which renders the cluster unusable, imho a very big problem.

# tail -f /var/log/xensource.log
Jan 10 13:04:23 ahbxen1 xapi: [debug||963 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops] Processing event: ["Vm","5d51c38b-5260-2903-ae21-4bbe607fb99c"]
Jan 10 13:04:23 ahbxen1 xapi: [debug||963 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops] xenops event on VM 5d51c38b-5260-2903-ae21-4bbe607fb99c
Jan 10 13:04:23 ahbxen1 xenopsd-xc: [debug||72617 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops_server] VM.stat 5d51c38b-5260-2903-ae21-4bbe607fb99c
Jan 10 13:04:23 ahbxen1 xapi: [debug||963 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops] xenopsd event: processing event for VM 5d51c38b-5260-2903-ae21-4bbe607fb99c
Jan 10 13:04:23 ahbxen1 xapi: [debug||963 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops] Supressing VM.allowed_operations update because guest_agent data is largely the same
Jan 10 13:04:23 ahbxen1 xapi: [debug||963 |org.xen.xapi.xenops.classic events D:ae7d342ca10e|xenops] xenopsd event: Updating VM 5d51c38b-5260-2903-ae21-4bbe607fb99c domid 14 guest_agent
Jan 10 13:04:27 ahbxen1 xapi: [debug||5755278 INET :::80||dummytaskhelper] task dispatch:session.logout D:cfcb0866d8ec created by task D:b4fe67a1bb65
Jan 10 13:04:30 ahbxen1 xapi: [debug||5755279 UNIX /var/lib/xcp/xapi||cli] xe pool-param-get uuid=Stopping ha-lizard= (via= systemctl):= [= OK= ]= param-name=other-config param-key=XenCenter.CustomFields.ha-lizard-enabled username=root password=(omitted)
Jan 10 13:04:30 ahbxen1 xapi: [ info||5755279 UNIX /var/lib/xcp/xapi|session.login_with_password D:655f9bce2d1c|xapi] Session.create trackid=24d93f0af567eed5f81fb70f2557487e pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Jan 10 13:04:32 ahbxen1 xcp-rrdd: [ info||7 ||rrdd_main] memfree has changed to 5363956 in domain 6
Jan 10 13:04:35 ahbxen1 xapi: [ info||5755280 INET :::80|session.login_with_password D:2d12df15927e|xapi] Failed to locally authenticate user root from HTTP request from Internet with User-Agent: xmlrpclib.py/1.0.1 (by www.pythonware.com): Authentication failure
Jan 10 13:04:35 ahbxen1 xapi: [debug||5755282 UNIX /var/lib/xcp/xapi||cli] xe host-list name-label=ahbxen1 minimal=true username=root password=(omitted)
Jan 10 13:04:35 ahbxen1 xapi: [ info||5755282 UNIX /var/lib/xcp/xapi|session.login_with_password D:87466e0161dd|xapi] Session.create trackid=249a86916b75d708a2c52adb1f011eed pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Jan 10 13:04:40 ahbxen1 xapi: [debug||1038 scanning_thread|SR scanner D:f2340ef7fc82|xapi_sr] Automatically scanning SRs = [ OpaqueRef:cf346a4f-4981-412d-a057-9b386d8bd2d6 ]
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] session.login_with_password D:2d12df15927e failed with exception Server_error(SESSION_AUTHENTICATION_FAILED, [ root; Authentication failure ])
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] Raised Server_error(SESSION_AUTHENTICATION_FAILED, [ root; Authentication failure ])
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 1/8 xapi Raised at file ocaml/xapi/xapi_session.ml, line 405
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 2/8 xapi Called from file ocaml/xapi/xapi_session.ml, line 40
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 3/8 xapi Called from file ocaml/xapi/xapi_session.ml, line 40
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 4/8 xapi Called from file ocaml/xapi/server_helpers.ml, line 83
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 5/8 xapi Called from file ocaml/xapi/server_helpers.ml, line 99
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 6/8 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 7/8 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace] 8/8 xapi Called from file lib/backtrace.ml, line 177
Jan 10 13:04:40 ahbxen1 xapi: [error||5755280 INET :::80||backtrace]
Jan 10 13:04:42 ahbxen1 xapi: [ info||5755283 UNIX /var/lib/xcp/xapi||cli] xe message-create name=HA-Lizard - xe_wrapper priority=1 body=xe_wrapper: COMMAND: xe pool-param-get pool-uuid=c1dbc848-aa29-1603-2af7-078466842ac2 username=root password=(omitted)

Before the timeouts ended. I had fixed a bug in /etc/ha-lizard/ha-lizard.func in line 645, where the were reading a pool-param-get what did never work. and obviously this was running much too fast and spawning so many processes, that the rest-api was dead.

(commented out line 645 and the subsequent lines + replaced the if condition to query the pool_uuid

This fixed a million error-notifications with in the error log. After restarting ha-lizard (service ha-lizard restart), everything returned back to normal.

Here is the thread on xcp-ng forum, where I described the problem
xcp-ng.org/forum/topic/6792/desperately-...-and-xcp-ng-crashes/

Kind regards
Christoph

Please Log in or Create an account to join the conversation.

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2857

Dear Christoph,

I am using HA-Lizard version 2.3.1 and I do not see any errors in the logs related to your problem. Maybe you should upgrade to the most recent version of HA-Lizard. Also in /etc/ha-lizard/ha-lizard.func the line number does not match so I assume the recent version have been improved compared to your version.

Upgrading HA-Lizard is easy, you will find matching advice here in the forum.
BR Andreas

Please Log in or Create an account to join the conversation.

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2858

  • Christoph Christ
  • Christoph Christ's Avatar Topic Author
  • Offline
  • Posts: 9
I'd love to - but where can I download the latest version?

kr
Christoph

Please Log in or Create an account to join the conversation.

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2859

Easy if you use the search term "update"

halizard.org/release/

Please Log in or Create an account to join the conversation.

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2861

  • Christoph Christ
  • Christoph Christ's Avatar Topic Author
  • Offline
  • Posts: 9
Thank you!! As I do not know your server, I would recommend. update also www.halizard.com/ha-lizard-software/download-logged-in, there are old versions.

As I did not find any hint on this searching the website of halizard:
Two years ago I did the install with the tar package and install script. Is there a safe way. how to upgrade to the RPM Version? There are a lot dependencies, which have been installed with the installer script

Kind regards
Christoph

Kind regards
Christoph

Please Log in or Create an account to join the conversation.

Last edit: by Christoph Christ.

ha-lizard error messages in xcp-ng xensource.log + dos on the rest-api 1 year 2 months ago #2862

I am a user of HA-Lizard and have no access to the website links.
There is no need to use the rpms, you could still use the tarballs.
If you read also the release notes you will find that your problem with the http access has been addressed with version 2.3.1.

If you search this forum with the term "upgrade" you will find many results explainging in detail how to proceed.
It is a small open source project, so you should help yourself first by reading the topics here, or to ask for a support contract or paid help from Salvatore.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1