New Install WebUI failed (first time) + suggestion for the remove FixLet

aixtools · June 2, 2017, 8:42am

I am test-driving a “playbook” for new installs of BigFix. Everything was going fine until it came time to run the fixlet to install the WebUI interface.

In my case, this is for version 9.5.5 - FYI.

First time around, the fixlet said - correctly, 100% complete - as I could connect to the WebUI as https://FQDN, but I could not login. Neither could I add WebUI Apps to roles I was creating - and even IEMAdmin could not login.

I waited 24 hours - and the problem was still there.

a) the suggestion for the “Remove WebUI” fixlet - have an option to save the log files (and maybe a new fixlet to clean them up later) - because I was sad to see all the logs were gone after I had uninstalled WebUI. Working as designed - but the design surprised me! (Lucky for me I had already cut/pasted some of the lines to a text file elsewhere - so I can share some of that here)

b) not really a suggestion - more a question - for after you have seen the snippets - could the relevance of the “install WebUI” be adjusted so that this would not happen again in the future. I’ll come back to this thought later.

So, before the unstall - after just under 24 hours I had three server-app.log files (.log, .log.0 and .log.1 versions - as an idea of the verbosity of the messages about “not happy”).

The start was (what I now know to be) normal - but a few key extra messages!

The common - expected stuff

USED Default directories and with FQDN as hostname (real short name changed to HOSTNAME-S)

Wed, 31 May 2017 13:10:57 GMT bf:appmonitor:debug Requesting APP_KEY and APP_CERT from the platform.
Wed, 31 May 2017 13:10:57 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug Requesting WEB_KEY and WEB_CERT from the platform.
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-web-credentials
Wed, 31 May 2017 13:10:58 GMT bf:bfquery:debug GET https://HOSTNAME-S:52315/api/serverinfo
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug dbtype set to:  SQLITE
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug Starting server.js process
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug Initializing sites
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug Initializing pending sites
Wed, 31 May 2017 13:10:58 GMT bf:bfquery:debug POST https://HOSTNAME-S:52315/api/dashboardvariables/WebUIAppAdmin
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug sync pending sites
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12049
Wed, 31 May 2017 13:10:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/etl-app-linux.zip
Wed, 31 May 2017 13:11:01 GMT bf:appmonitor:debug Initializing with SAML configuration: false
Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.
Wed, 31 May 2017 13:11:01 GMT bf:appmonitor:debug App server listening on port 5000
Wed, 31 May 2017 13:11:01 GMT bf:appmonitor:debug Redirect server listening on port 80
Wed, 31 May 2017 13:11:01 GMT bf:appmonitor:debug Web server listening on port 443
Wed, 31 May 2017 13:11:11 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/login-app-linux.zip
Wed, 31 May 2017 13:11:28 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/common-app-linux.zip
Wed, 31 May 2017 13:12:08 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Wed, 31 May 2017 13:12:08 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Wed, 31 May 2017 13:12:08 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Wed, 31 May 2017 13:12:08 GMT bf:appmonitor:debug Requesting credentials for etl 1.1.2
Wed, 31 May 2017 13:12:08 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Wed, 31 May 2017 13:12:09 GMT bf:appmonitor:debug Starting etl 1.1.2 on port 5001
Wed, 31 May 2017 13:12:09 GMT bf:appmonitor:debug Requesting credentials for login 1.0.0
Wed, 31 May 2017 13:12:09 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Wed, 31 May 2017 13:12:10 GMT bf:appmonitor:debug Starting login 1.0.0 on port 5002
Wed, 31 May 2017 13:12:10 GMT bf:appmonitor:debug Requesting credentials for common 0.3.0
Wed, 31 May 2017 13:12:10 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Wed, 31 May 2017 13:12:10 GMT bf:appmonitor:debug Starting common 0.3.0 on port 5003
Wed, 31 May 2017 13:12:10 GMT bf:bfquery:debug POST https://HOSTNAME-S:52315/api/dashboardvariables/WebUIAppAdmin
Wed, 31 May 2017 13:13:22 GMT bf:appmonitor:debug sync pending sites
Wed, 31 May 2017 13:13:22 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Wed, 31 May 2017 13:13:30 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12402

After this, it starts looping - notice the timestamps

Wed, 31 May 2017 13:13:30 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12402 failed: Error: bad status [404] The site has not been gathered yet.
Wed, 31 May 2017 13:13:30 GMT bf:appmonitor:error Failed to install site WebUI App Admin version 0: Error: bad status [404] The site has not been gathered yet.
Wed, 31 May 2017 13:13:30 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12124
Wed, 31 May 2017 13:13:30 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12124 failed: Error: bad status [404] The site has not been gathered yet.


Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12402
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12402 failed: Error: bad status [404] The site has not been gathered yet.
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:error Failed to install site WebUI App Admin version 0: Error: bad status [404] The site has not been gathered yet.
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12124
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12124 failed: Error: bad status [404] The site has not been gathered yet.
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:error Failed to install site WebUI Custom version 0: Error: bad status [404] The site has not been gathered yet.

and the next change - message 404 gets replaced by message 401

Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12345
Wed, 31 May 2017 13:15:52 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_16_9/query-app-linux.zip
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12050
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12050 failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error Failed to install site WebUI Patch version 0: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12090
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12090 failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error Failed to install site WebUI Software Distribution version 0: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12392
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites/12392 failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:error Failed to install site Profile Management version 2: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:debug Requesting credentials for query 0.1.0
Wed, 31 May 2017 13:16:53 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Wed, 31 May 2017 13:16:54 GMT bf:appmonitor:error POST https://HOSTNAME-S:52315/api/webui-app-credentials failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:54 GMT bf:appmonitor:error Failed to start query 0.1.0: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:16:54 GMT bf:bfquery:debug POST https://HOSTNAME-S:52315/api/dashboardvariables/WebUIAppAdmin
Wed, 31 May 2017 13:18:04 GMT bf:appmonitor:debug sync pending sites
Wed, 31 May 2017 13:18:04 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Wed, 31 May 2017 13:18:04 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:18:04 GMT bf:appmonitor:error Failed to refresh sites: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:18:04 GMT bf:appmonitor:error Update loop error: TypeError: Cannot read property 'newSites' of undefined
Wed, 31 May 2017 13:19:04 GMT bf:appmonitor:debug sync pending sites
Wed, 31 May 2017 13:19:04 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Wed, 31 May 2017 13:19:04 GMT bf:appmonitor:error GET https://HOSTNAME-S:52315/api/webui-sites failed: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:19:04 GMT bf:appmonitor:error Failed to refresh sites: Error: bad status [401] HTTP 401: Unauthorized
Wed, 31 May 2017 13:19:04 GMT bf:appmonitor:error Update loop error: TypeError: Cannot read property 'newSites' of undefined

This goes on, basically, forever - as the next day - this is how it reports on itself - notice - this is now a 5 second loop!

[root@HOSTNAME-S opt]# tail -50 ./BESWebUI/WebUI/logs/service-app.log
Thu, 01 Jun 2017 10:17:04 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 10:17:04 GMT bf:appmonitor:error POST https://HOSTNAME-S:52315/api/webui-app-credentials failed: Error: bad status [401] HTTP 401: Unauthorized
/var/opt/BESWebUI/WebUI/service-app/node_modules/q/q.js:155
                throw e;
                ^

Error: bad status [401] HTTP 401: Unauthorized
    at /var/opt/BESWebUI/WebUI/service-app/node_modules/bfappmonitor/lib/platform.js:41:24
    at invokeCallback (/var/opt/BESWebUI/WebUI/service-app/node_modules/raw-body/index.js:262:16)
    at done (/var/opt/BESWebUI/WebUI/service-app/node_modules/raw-body/index.js:251:7)
    at IncomingMessage.onEnd (/var/opt/BESWebUI/WebUI/service-app/node_modules/raw-body/index.js:307:7)
    at emitNone (events.js:72:20)
    at IncomingMessage.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:923:12)
    at nextTickCallbackWith2Args (node.js:458:9)
    at process._tickCallback (node.js:372:17)
Thu, 01 Jun 2017 10:17:09 GMT bf:appmonitor:debug Requesting APP_KEY and APP_CERT from the platform.
Thu, 01 Jun 2017 10:17:09 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 10:17:09 GMT bf:appmonitor:error POST https://HOSTNAME-S:52315/api/webui-app-credentials failed: Error: bad status [401] HTTP 401: Unauthorized
/var/opt/BESWebUI/WebUI/service-app/node_modules/q/q.js:155
                throw e;
                ^

So, back to my thoughts

Are the GET https://HOSTNAME-S:52315/api/webui-sites/XXXXX FAILED messages something that can be done as a separate task, creating relevance for a next step once complete. I ask, because after the “Remove WebUI” fixlet, 5 minute wait, and install again (with IP address rather than FQDN) - there was no “404” failures. Everything was already (down)loaded - or so it seems now. (In an attempt to recreate my initial issue I removed again, and installed again using FQDN.)
In any case, the subsequent installs of the WebUI interface log files look normal see:

Remove and Install again, using FQDN again:

[root@HOSTNAME-S logs]# touch service-app.log
[root@HOSTNAME-S logs]# tail -f service-app.log
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug Requesting APP_KEY and APP_CERT from the platform.
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug Requesting WEB_KEY and WEB_CERT from the platform.
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-web-credentials
Thu, 01 Jun 2017 11:16:58 GMT bf:bfquery:debug GET https://HOSTNAME-S:52315/api/serverinfo
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug dbtype set to:  SQLITE
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug Starting server.js process
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug Initializing sites
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug Initializing pending sites
Thu, 01 Jun 2017 11:16:58 GMT bf:bfquery:debug POST https://HOSTNAME-S:52315/api/dashboardvariables/WebUIAppAdmin
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12049
Thu, 01 Jun 2017 11:16:58 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/etl-app-linux.zip
Thu, 01 Jun 2017 11:17:01 GMT bf:appmonitor:debug Initializing with SAML configuration: false
Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.
Thu, 01 Jun 2017 11:17:01 GMT bf:appmonitor:debug App server listening on port 5000
Thu, 01 Jun 2017 11:17:01 GMT bf:appmonitor:debug Redirect server listening on port 80
Thu, 01 Jun 2017 11:17:01 GMT bf:appmonitor:debug Web server listening on port 443
Thu, 01 Jun 2017 11:17:09 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/login-app-linux.zip
Thu, 01 Jun 2017 11:17:23 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_1_38/common-app-linux.zip
Thu, 01 Jun 2017 11:17:54 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Thu, 01 Jun 2017 11:17:54 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Thu, 01 Jun 2017 11:17:54 GMT bf:appmonitor:debug release.json is not found in WebUI Common version 38.
Thu, 01 Jun 2017 11:17:54 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12402
Thu, 01 Jun 2017 11:17:56 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_14_6/appadmin-app-linux.zip
Thu, 01 Jun 2017 11:18:22 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12124
Thu, 01 Jun 2017 11:18:23 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_15_15/custom-app-linux.zip
Thu, 01 Jun 2017 11:19:17 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12345
Thu, 01 Jun 2017 11:19:18 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_16_9/query-app-linux.zip
Thu, 01 Jun 2017 11:19:53 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12050
Thu, 01 Jun 2017 11:19:54 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_17_17/patch-app-linux.zip
Thu, 01 Jun 2017 11:20:37 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12090
Thu, 01 Jun 2017 11:20:38 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_18_14/swd-app-linux.zip
Thu, 01 Jun 2017 11:21:07 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites/12392
Thu, 01 Jun 2017 11:22:45 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/bfmirror/bfsites/enterprisemirror_19_2/prfmgr-app-linux.zip
Thu, 01 Jun 2017 11:23:10 GMT bf:appmonitor:debug Requesting credentials for etl 1.1.2
Thu, 01 Jun 2017 11:23:10 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:10 GMT bf:appmonitor:debug Starting etl 1.1.2 on port 5001
Thu, 01 Jun 2017 11:23:10 GMT bf:appmonitor:debug Requesting credentials for login 1.0.0
Thu, 01 Jun 2017 11:23:10 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:11 GMT bf:appmonitor:debug Starting login 1.0.0 on port 5002
Thu, 01 Jun 2017 11:23:11 GMT bf:appmonitor:debug Requesting credentials for common 0.3.0
Thu, 01 Jun 2017 11:23:11 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:12 GMT bf:appmonitor:debug Starting common 0.3.0 on port 5003
Thu, 01 Jun 2017 11:23:12 GMT bf:appmonitor:debug Requesting credentials for patch 0.3.0
Thu, 01 Jun 2017 11:23:12 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:13 GMT bf:appmonitor:debug Starting patch 0.3.0 on port 5004
Thu, 01 Jun 2017 11:23:13 GMT bf:appmonitor:debug PUT https://HOSTNAME-S:52315/api/webui-app/patch
Thu, 01 Jun 2017 11:23:13 GMT bf:appmonitor:debug Requesting credentials for custom 0.3.0
Thu, 01 Jun 2017 11:23:13 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:14 GMT bf:appmonitor:debug Starting custom 0.3.0 on port 5005
Thu, 01 Jun 2017 11:23:14 GMT bf:appmonitor:debug PUT https://HOSTNAME-S:52315/api/webui-app/custom
Thu, 01 Jun 2017 11:23:14 GMT bf:appmonitor:debug Requesting credentials for swd 0.2.1
Thu, 01 Jun 2017 11:23:14 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:16 GMT bf:appmonitor:debug Starting swd 0.2.1 on port 5006
Thu, 01 Jun 2017 11:23:16 GMT bf:appmonitor:debug PUT https://HOSTNAME-S:52315/api/webui-app/swd
Thu, 01 Jun 2017 11:23:16 GMT bf:appmonitor:debug Requesting credentials for appadmin 0.2.0
Thu, 01 Jun 2017 11:23:16 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:16 GMT bf:appmonitor:debug Starting appadmin 0.2.0 on port 5007
Thu, 01 Jun 2017 11:23:17 GMT bf:appmonitor:debug Requesting credentials for prfmgr 0.1.4
Thu, 01 Jun 2017 11:23:17 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:17 GMT bf:appmonitor:debug Starting prfmgr 0.1.4 on port 5008
Thu, 01 Jun 2017 11:23:18 GMT bf:appmonitor:debug PUT https://HOSTNAME-S:52315/api/webui-app/prfmgr
Thu, 01 Jun 2017 11:23:18 GMT bf:appmonitor:debug Requesting credentials for query 0.1.0
Thu, 01 Jun 2017 11:23:18 GMT bf:appmonitor:debug POST https://HOSTNAME-S:52315/api/webui-app-credentials
Thu, 01 Jun 2017 11:23:19 GMT bf:appmonitor:debug Starting query 0.1.0 on port 5009
Thu, 01 Jun 2017 11:23:19 GMT bf:appmonitor:debug PUT https://HOSTNAME-S:52315/api/webui-app/query
Thu, 01 Jun 2017 11:23:20 GMT bf:bfquery:debug POST https://HOSTNAME-S:52315/api/dashboardvariables/WebUIAppAdmin
Thu, 01 Jun 2017 11:25:13 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:25:13 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Thu, 01 Jun 2017 11:26:23 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:26:23 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Thu, 01 Jun 2017 11:27:33 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:27:33 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Thu, 01 Jun 2017 11:28:43 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:28:43 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites
Thu, 01 Jun 2017 11:29:53 GMT bf:appmonitor:debug sync pending sites
Thu, 01 Jun 2017 11:29:53 GMT bf:appmonitor:debug GET https://HOSTNAME-S:52315/api/webui-sites

Hope this helps!
Michael

joseg · June 2, 2017, 4:08pm

Thanks for posting all the logs and details!

I think the 404’s would be expected, WebUI is the component that requests these sites for gathering and the list of sites might be dynamic and change over time, and they are not gathered until WebUI is installed and requests them, with the exception to that being WebUI Common. So I don’t think you could have the install fixlets depend on having all the site data in place, since it’s not known in advance.

Back to the errors, though, I’m not sure about why there are 401’s, since it seems that WebUI was able to talk to the root server without any problems earlier, so that for me would be the unexpected part. Maybe there are additional messages in BESRelay.log that could give more hints in case there were errors in the root server’s side?

jgstew · June 2, 2017, 5:19pm

These are some good suggestions and things to look deeper into, but in the short term you may need to file a PMR with IBM to get the WebUI working: How to ask for IBM product help: PMRs, RFEs, and more

I have definitely thought of having a fixlet that would examine various logs to see if there are any issues and where as well as an analysis to report on the status of various of these parts which would make troubleshooting much easier.

One issue I run into is that relevance isn’t very good at examining log files efficiently. I generally want to know if there is an error message in a log file more recently than the last success message, or if there is a success message in the log more recently than the last of a particular error message.

CC: @dexdexdex

aixtools · June 7, 2017, 6:20am

Was a few days ago when I posted that - so I think I did not write clearly enough.
I got WebUI working by removing it, and reinstalling it.

Whatever it needed to download the first time, it did not need the second time (no 404 messages) and the WebUI interface came up immediately.

The second time I entered the IP address, rather than the FQDN. So, to see if I could get the 401 message back again I uninstalled again, and reinstalled using FQDN - and it worked fine.

I’ll take a look for BESRelay logs (or whatever is being asked later).

Thanks for the quick replies - I apologize for the slow response.

aixtools · June 15, 2017, 12:12pm

Not sure what is happening now? I have logged on before (> a week ago), but now get a message saying:

From another thread - there is the request for log information.

lots of log data, for something running only only default settings (i.e., not done anything with it yet).
I hope useful log information:

du -hs *

331M BESClient
0 BESClientDeployTool
65M BESInstallers
14G BESServer
624M BESWebReportsServer
2.0G BESWebUI

du -sh *

12K cert
421M Mirror
4.0K service-wrapper.log
1.6G WebUI
16K WebUICommonState.db

[root@hostname WebUI]# du -sh *
237M ETL
30M logs
24M node
22M service-app
1.3G sites

[root@hostname logs]# tail etl.log
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_FIELDS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_PROPERTIES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_RELEVANCE: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_ACTION_SETTINGS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_ACTION_SETTINGS_USER_GROUPS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENTS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_ACTIONS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_ACTION_SUCCESS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_GROUPS: unable to open database fi[root@hostname logs]# grep -c Failed etl*
etl.log:22
etl.log.0:0
etl.log.1:0
etl.log.2:0
etl.log.3:0
etl.log.4:0

Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTIONS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTION_FIELDS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTION_PARAMETERS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTION_TARGET_NAMES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTION_TARGET_STATIC: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ACTION_USER_GROUPS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update ANALYSIS_ACTIVATIONS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update COMPUTER_ACTIONS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update COMPUTER_ANALYSES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update COMPUTER_GROUPS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update COMPUTER_PROPERTY_INFO: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update COMPUTER_PROPERTY_TEXT: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_FIELDS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_PROPERTIES: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_ANALYSIS_RELEVANCE: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_ACTION_SETTINGS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_ACTION_SETTINGS_USER_GROUPS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENTS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_ACTIONS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_ACTION_SUCCESS: unable to open database file
Wed, 14 Jun 2017 18:45:40 GMT bf:bfetl:debug Failed to update CUSTOM_BASELINE_COMPONENT_GROUPS: unable to open database fi[root@hostname logs]#
[root@hostname logs]# date
Thu Jun 15 08:05:41 EDT 2017

[root@hostname logs]# tail service-app.log
Thu, 15 Jun 2017 12:05:01 GMT bf:appmonitor:debug sync pending sites
Thu, 15 Jun 2017 12:05:01 GMT bf:appmonitor:debug GET https://hostname:52315/api/webui-sites
Thu, 15 Jun 2017 12:06:11 GMT bf:appmonitor:debug sync pending sites
Thu, 15 Jun 2017 12:06:11 GMT bf:appmonitor:debug GET https://hostname:52315/api/webui-sites
Thu, 15 Jun 2017 12:07:21 GMT bf:appmonitor:debug sync pending sites
Thu, 15 Jun 2017 12:07:21 GMT bf:appmonitor:debug GET https://hostname:52315/api/webui-sites
Thu, 15 Jun 2017 12:08:32 GMT bf:appmonitor:debug sync pending sites
Thu, 15 Jun 2017 12:08:32 GMT bf:appmonitor:debug GET https://hostname:52315/api/webui-sites
Thu, 15 Jun 2017 12:09:42 GMT bf:appmonitor:debug sync pending sites
Thu, 15 Jun 2017 12:09:42 GMT bf:appmonitor:debug GET https://hostname:52315/api/webui-sites

So, since it seems to be a database error, AND, I ran out of space yesterday (which may have adversely affected the database) - going to uninstall and reinstall. Will keep you posted

aixtools · June 15, 2017, 12:54pm

No idea (unless the database is in a directory named cache) - how this went so bad, but remove and (re)-install fixed it.

FYI