VM manager Connection failed in ILMT

Hi All,

I am newly building the BigFix(10.0.4) and ILMT(9.2.25).

After build the ILMT when I try to connect the Vcenter it’s getting failed,

when I check with VMMAN logs it’s showing this error please suggest to me how to resolve this.

Thanks, in advance.

I am sharing the logs for reference.

2022-02-07 03:04:43 : (75) com.ibm.license.mgmt.datacollector.util.CLILogger::processTask()::[ vmmconf_75.properties ] An error occurred while attempting to connect to the VM manager.
2022-02-07 03:04:43 : (75) com.ibm.license.mgmt.datacollector.util.CLILogger::processTask()::[ vmmconf_75.properties ] An error occurred while attempting to connect to the VM manager.
2022-02-07 03:04:43 : (75) com.ibm.license.mgmt.datacollector.status.StatusUtils::convertErrorCodeToStatus::Resolved status: Connection failed
2022-02-07 03:04:43 : (75) com.ibm.license.mgmt.datacollector.tasks.VMCollectorTask::saveResultsIntoFile()::There is no changes in the current scan results in comparison to the previous scan - scan results not transfered to the agent
2022-02-07 03:34:22 : (75) com.ibm.license.mgmt.vmmanager.ConnectorFactory::createConnector::Connector VMwareConnector created for requested type VMWARE_V_SPHERE
2022-02-07 03:34:45 : (75) com.ibm.license.mgmt.vmmanager.vmware.VMMServiceConnection::connect::Some error occured during connecting to VM Manager: https://#############/sdk
2022-02-07 03:34:45 : (75) com.ibm.license.mgmt.vmmanager.vmware.VMwareConnector::connect:com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: java.net.ConnectException: Connection timed out: connect
at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:129)
at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:220)
at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:142)
at com.sun.xml.internal.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:136)
at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fiber.java:1133)
at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fiber.java:1047)
at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber.java:1016)
at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fiber.java:874)
at com.sun.xml.internal.ws.client.Stub.process(Stub.java:460)
at com.sun.xml.internal.ws.client.sei.SEIStub.doProcess(SEIStub.java:190)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:105)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:159)
at com.sun.proxy.$Proxy53.retrieveServiceContent(Unknown Source)
at com.ibm.license.mgmt.vmmanager.vmware.VMMServiceConnection$ConnectThread.run(VMMServiceConnection.java:127)
at java.lang.Thread.run(Thread.java:822)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:96)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:380)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:236)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:218)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:183)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.net.Socket.connect(Socket.java:682)
at com.ibm.jsse2.bi.connect(bi.java:114)
at sun.net.NetworkClient.doConnect(NetworkClient.java:187)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:494)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:589)
at com.ibm.net.ssl.www2.protocol.https.c.(c.java:216)
at com.ibm.net.ssl.www2.protocol.https.c.a(c.java:164)
at com.ibm.net.ssl.www2.protocol.https.d.getNewHttpClient(d.java:17)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1217)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1068)
at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:6)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1352)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1327)
at com.ibm.net.ssl.www2.protocol.https.b.getOutputStream(b.java:91)
at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:116)
… 15 more

There’s a connection timeout in there - usually means the server running the plugin (usually the BigFix root server) is blocked from reaching the VCenter server by a firewall.

What is your VMWare Hypervisor version? If 5.5 and above, https://{manager-host}/sdk/vimService.wsdl should be used.

You can also test this URL from Browser (preferably from the root server) and validate

Thanks @JasonWalker

Could you elaborate more please, means which firewall is blocking.

means while connect to BigFix root server to VCenter, it’s blocking server level or Vcenter level?

if that is the case may I know which port need to be open?

Thanks @ssakunala,

I am not sure about the VMware version, I will check and update you.

I’ve troubleshot a TON of those, so here are 4 things that you need to cover:

  1. Connectivity - as @JasonWalker mentioned port. The VMware tool uses standard https port (443) to make the connection, so check if you can connect. Launch a browser from the machine where VM Manager tool is installed and try to connect to https://{manager-host} - if you can connect, you should be good (it covers both port and hostname resolution). If either of those don’t work, you need to get them sorted… You can always do the ping & telnet tests as well, whatever you feel more comfortable with!
  2. Credentials - again, if you open it through a browser, it should give you the management interface and you can test your credentials
  3. vSphere Web Services SDK - make sure it is enabled & configured, otherwise, it will not work (full documentation)
  4. One last item worth noting (it took me a lot of hassle to figure out by test and trial) - in some occassions, some versions of vSphere/vCenter do not always propagate permissions correctly for users even though they are granted on the global level; I had to set the same exact permission on each hypervisor within a cluster to get it work. It is a VMware issue but nonetheless if you are banging your head against the wall, it doesn’t help you…
1 Like

Hi Team,

Thanks for the information, now I am facing another issue.

VM Manger status – PENDING

I am successfully connected to the Vcenter and the test connection has also been made successfully, but Data is not retrieved from the Vcenter.

  1. User has full access in Vcenter level,
  2. I check the URL no blocking between ILMT and Vcneter,
  3. Current ILMT version is 9.2.17, I run the Schedule VM Manager Tool Scan Results Upload fixlet(9.2.27) and after that, I run Force VM Manager Tool Scan Results Upload fixlets(9.2.27) still VM manager status shows pending only.

I am providing VMAN logs for your reference, please help on this…

22-05-13 09:16:57 : (main) com.ibm.license.mgmt.datacollector.configuration.DataCollectorProperties::getMainProperties()::Sucessfully read main configuration.
2022-05-13 09:16:57 : (main) com.ibm.license.mgmt.datacollector.configuration.DataCollectorProperties::getLMTProperties()::Sucessfully read lmt configuration.
2022-05-13 09:16:57 : (main) com.ibm.license.mgmt.datacollector.Runner::readAndSaveVersionToLog::Tool version: 9.2.17.0, build number: 20191010-1920
2022-05-13 09:16:57 : (main) com.ibm.license.mgmt.datacollector.Runner::runCollector(Configuration)::Starting ‘testAndPrintRunningStatus’ command.
2022-05-13 09:16:58 : (main) com.ibm.license.mgmt.datacollector.Runner::runCollector(Configuration)::Finished ‘testAndPrintRunningStatus’ command.
2022-05-13 09:16:58 : (main) com.ibm.license.mgmt.datacollector.Runner::initialiseLogger::Logger initialised successfully.
2022-05-13 09:16:58 : (main) com.ibm.license.mgmt.datacollector.security.impl.DataCollectorSecurityServiceImpl::DataCollectorSecurityServiceImpl::Initializing the security service using the keystore: /var/opt/BESClient/LMT/VMMAN/keydb/keys.jceks
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.configuration.DataCollectorProperties::getMainProperties()::Sucessfully read main configuration.
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.configuration.DataCollectorProperties::getVMProperties(File,String)::Sucessfully read VM configurations.
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.configuration.DataCollectorProperties::getLMTProperties()::Sucessfully read lmt configuration.
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.Runner::readAndSaveVersionToLog::Tool version: 9.2.17.0, build number: 20191010-1920
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.Runner::runCollector(Configuration)::Starting ‘test connection by id’ command.
2022-05-13 09:16:59 : (main) com.ibm.license.mgmt.datacollector.Runner::testConnectionCommand(Configuration)::1 VM configurations found. Trying to start tasks.
2022-05-13 09:16:59 : (pool-1-thread-1) com.ibm.license.mgmt.vmmanager.ConnectorFactory::createConnector::Connector VMwareConnector created for requested type VMWARE_V_SPHERE
2022-05-13 09:17:06 : (pool-1-thread-1) com.ibm.license.mgmt.datacollector.util.CLILogger::processTask::[ vmmconf_10.properties ] Test connection operation has been successfully completed.
2022-05-13 09:17:06 : (main) com.ibm.license.mgmt.datacollector.Runner::runCollector(Configuration)::Finished ‘test connection by id’ command.

Thans… In Advance…

This log is from few days ago. What does the latest log show? Did the VM Manager tool generate the data and if so has it been uploaded to the root server side and imported?

https://help.hcltechsw.com/bigfix/9.5/inventory/Inventory/admin/c_troubleshooting_no_vm_manager_data.html

Trace the VM Manager Data end to end and see where it’s missing

Hi @ssakunala,

that day I updated the Vcenter URL in ILMT, from onwards there no new logs are generated in VMMAN logs.

ILMT connection is successful, but the operation status is not changing.

image

The link above has various trouble shooting steps to go through.
The VM Manager tool task runs as a service and should be running, Check if it’s running
Check if the VM Manager tool has generated any configuration file under LMT\VMMAN\config folder under the BigFix client on the root server.
Restart the VM Manager Service to see if any new information is written to the log file
If the connection test is successful, check if any data file is created under LMT\VMMAN\output folder
Check if the “Schedule VM Manager Data Upload” task is running in the BigFix Console
If this task is running and the tool has collected data, then it should be available on the root server, under UploadManagerData folder

@ssakunala

Thanks for the information, when I check the VM manager status by using this command BESClient/LMT/VMMAN/vmman.sh -status I am getting the error, hence I try to reinitiate the Install VM manager fixlet but it’s getting failed.

image

Please help on this, Thanks in Advance…

You will need to stop the service manually or use task manager to kill it. Then try installing it again.

can I stop VM Manager services by using this command:BESClient/LMT/VMMAN/vmman.sh -stop. and need to run the Initiate VM manager fixlet, am I right.

Yes - that is the command to stop it on Unix. Then you can run the install Fixlet.

@itsmpro92

When I try to stop the VM Manager I am getting this error,

[root@M1 VMMAN]# ./vmman.sh run
./java/jre/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory
[root@M
1 VMMAN]# ./vmman.sh status
./java/jre/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory
[root@M*******1 VMMAN]# ./vmman.sh stop
./java/jre/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory

when I check the Java Directory everything is zipped under the java directory, so I am planning to reinstall the VM Manager tool,

can help with this…

It looks like something happened to the installation at some point. I recommend you open a case with IBM to get support for cleaning things up and reinstalling VM Manager.

Hi,

It’s happening after Log4j 2.17.2 Update onwards,

Schedule VM Manager Tool Scan Results Upload(9.2.16) also not completed, it’s showing the waiting status only, might be that is the reason Vmamnager results are not updating to ILMT.

if we run the update VM manager tool(9.2.27) and Schedule VM Manager Tool Scan Results Upload(9.2.27) this is will be fix?

I doubt it - the error you encountered suggests that the VM Manager installation is broken. Opening a Case with IBM is still your best bet.

Сonsidering this information, first of all you need to reinstall VM Manager and then it’ll be clear.

All the best, Jane, manager, Work Time

Hi,

They restored the server, Now VM manager is working fine, and the schedule VM manager fixlet was also completed, and in ILMT VM manager connection also showing ok status.

But the problem is hardware data is not updating in ILMT, when I check the trace logs there is no error it’s showing data is retrieved successfully from the VM manager.

sample trace logs:

2022-06-07 09:22:52 : (pool-1-thread-2) com.ibm.license.mgmt.vmmanager.ConnectorFactory::createConnector::Connector VMwareConnector created for requested type VMWARE_V_SPHERE
2022-06-07 09:22:55 : (pool-1-thread-2) com.ibm.license.mgmt.datacollector.tasks.VMCollectorTask::processTask()::(vmmconf_10.properties): Virtualization hierarchy successfully retrieved from VM Manager at https://..com/sdk (VMWARE_V_SPHERE)
2022-06-07 09:22:55 : (pool-1-thread-2) com.ibm.license.mgmt.datacollector.tasks.VMCollectorTask::processTask()::(vmmconf_10.properties): VM Manager info: class com.ibm.license.mgmt.vmmanager.vmware.model.VMwareInfo[VirtualCenter,6.7.0,VMware vCenter Server 6.7.0 build-19299595]
2022-06-07 09:22:55 : (pool-1-thread-2) com.ibm.license.mgmt.datacollector.util.CLILogger::processTask::[ vmmconf_10.properties ] Data was successfully retrieved from the VM manager at https://
..com/sdk.**

But Hardware data is not updating ILMT

please help on this how to resolve this issue,

Thanks in Advance…