ILMT - BigFix installation fails - Error to download content from IBM - HTTP Error 56

Usage and Config BFI
1

I have installed ILMT and BigFix several times at different customers but never encountered this problem.

In the BESRelay.log the following is logged: “GetURL failure on http://sync.bigfix.com/bfsites/bessupport_1289/BES-90Patch2Issue.BESDomain: HTTP Error 56: Failure when receiving data from the peer: Recv failure: Connection was reset”
In the GatherDB.log: “Unexpected exception during gather of site BES Support: Unexpected HTTP response: 404”

I have installed SQL 2014 Express on a Win Server 2012 R2 server. All the IBM recommended HW/SW prereqs. are in place, .NET Framework 3.5 and Desktop Experience enabled. IE Enhanced security configuration disabled, Windows Firewall disabled, UAC disabled.

I installed BigFix server via BigFix license authorization file. During the request file to IBM for license verification I first had to configure a proxy, then the installation worked without problems

I installed the latest version 9.5.4.38 of BigFix- Server, Console and Client on the same Win Server 2012 R2. The BigFix console is not populated with content after the installation. Only “All Content view” is shown, not BigFix Management and System Lifecycle. Only All Content is shown when I check the twisty/arrow to add or remove buttons. Sometimes this can take a while but at this customer BigFix content will not download from IBM.

I searched the Internet and found that Proxy could cause the error messages shown in the BESRelay.log and the customer is using Proxy to communicate externally

I checked the BigFix diagnostic Tool and all test are successful (Green) on all tabs

I had the customer management to approve a change so the BigFix server could bypass the proxy and communicate directly via the FW to IBM. I checked with telnet on port 80/443 for the following 5 url’s and it works ok, esync.bigfix.com, gatherer.bigfix.com, software.bigfix.com, support.bigfix.com, sync.bigfix.com
I also ran tracert to the 5 IBM url’s and no proxy shown in between, just L3 interface in the customers datacentre routers.

I checked the BigFix diagnostic Tool again and all test are successful (Green) on all tabs, but the BigFix console is not populated with content from IBM and the BESRelay.log shows the same error message.

I reverted back to a snapshot and installed a previous version of BigFix, registered a new authorization file and installed, same problem.

I googled some more and tried again with proxy settings. Only IE Proxy settings, only Proxy setting in BigFix (BESAdmin tool) and both proxy settings, IE and BESAdmin. Several restarts in between and verified Proxy settings in BigFix Console (Computer client settings) and Windows registry, but the same problem.

I tried to open the failing url http://sync.bigfix.com/bfsites/bessupport_1289/BES-90Patch2Issue.BESDomain in the IE browser with no proxy settings, directly via the FW. The result via https is a blank page, via http “this page can’t be displayed. Other files under http://sync.bigfix.com/bfsites/bessupport_1289/ works ok and some don’t. There are many files so just tried a few. We tried the same url at the customers desktop computers via proxy, same result.

I tried the failing url back at my office. Internet Explorer 11 works, Firefox does not work, “XML parsing problems: undefined entity”. Chrome “This page contains the following errors: error on line 10 at column 25: PEReference: %GeneralContentInclude; not found”

After searching the Internet for similar problems, I also tried the following settings, edited the computer client settings from the BigFix console and restarted WIN 2012 R2 server
_BESGather_Comm_UseUrlMoniker=0
Same problem
_HTTPRequestSender_Connect_TimeoutSecond=30
Same problem

I am sorry for a long post, I would very much appreciate help regarding this
Thanks
Mikael

Server Diagnostics
(Version: 9.5.4.38)

  • Relay Control Settings
    •_BESClient_Relay_NameOverride:
    •_BESRelay_Register_Affiliation_AdvertisementList:
    •_BESRelay_Comm_Authenticating: 0
    •_BESRelay_Selection_AutoSelectableRelay: 1
    •_BESRelay_Selection_RelayPriority: 0
    •_BESRelay_Selection_RelayWeight: 100

  • HTTPServer Settings
    •_BESRelay_HTTPServer_DynamicThrottleEnabled: 0
    •_BESRelay_HTTPServer_DynamicThrottleMaxKBPS: 0
    •_BESRelay_HTTPServer_DynamicThrottleMinKBPS: 0
    •_BESRelay_HTTPServer_DynamicThrottlePercentage: 0.0
    •_BESRelay_HTTPServer_HostName:
    •_BESRelay_HTTPServer_HttpLogExpirationDays: 0
    •_BESRelay_HTTPServer_LogFileSizeLimit: 52428800
    •_BESRelay_HTTPServer_MaxConnections: 2048
    •_BESRelay_HTTPServer_MaxRequestLineLength: 32768
    •_BESRelay_HTTPServer_PortNumber: 52311
    •_BESRelay_HTTPServer_ReadTimeOut: 60
    •_BESRelay_HTTPServer_ThrottleKBPS: 0
    •_BESRelay_HTTPServer_UseKeepAlive: 1
    •_BESRelay_HTTPServer_UseSSLFlag: 0
    •_BESRelay_HTTPServer_RequireTLS12: 0
    •_BESRelay_HTTPServer_WriteTimeOut: 300
    •_BESRelay_HTTPServer_WriteMiniDumps: 0

  • Gather Settings
    Enterprise Server\GatherService Settings:•ConnectivityTestResult: 24
    •DownloadMirror URL: http://customer-hostname:52311/bfmirror/downloads/
    BES Client Settings:•_BESGather_ConnectivityTest_SuccessInterval: 10
    •_BESGather_ConnectivityTest_FailureInterval: 1
    •_BESGather_Comm_ParentRelayURL:
    •_BESGather_Comm_UseUrlMoniker: 1
    •_BESGather_Comm_UseDownloadService: 0
    •_BESGather_Download_CacheLimitMB: 1024
    •_BESGather_Download_CheckParentFlag: 0
    •_BESGather_Download_CheckInternetFlag: 1
    •_BESGather_Download_DiffCacheLimitMB: 4
    •_BESGather_Download_DynamicThrottleEnabled: 0
    •_BESGather_Download_DynamicThrottleMaxBytesPerSecond: 4294967295
    •_BESGather_Download_DynamicThrottleMinBytesPerSecond: 0
    •_BESGather_Download_DynamicThrottlePercentage: 0
    •_BESGather_Download_HasHashIndex: 0
    •_BESGather_Download_HasUrlIndex: 0
    •_BESGather_Download_InactivityTimeout: 300
    •_BESGather_Download_LimitBytesPerSecond: 0
    •_BESGather_Download_MaxDownloadJobs: 4294967295
    •_BESGather_Download_NumVersionsToServe: 7
    •_BESGather_Download_RetryLimit: 5
    •_BESGather_Download_RetryMinutes: 10
    •_BESGather_Download_ThrottleGroup: ipaddress
    •_BESGather_Indexer_WaitTime: 32
    •_BESGather_Mirror_ActiveLoopDelayMilliseconds: 1000
    •_BESGather_Mirror_InactiveLoopDelayMilliseconds: 3000
    •_BESGather_Mirror_PreGatherThresholdMinutes: 1440
    •_BESGather_Mirror_SiteVersionPollingPeriod: 30
    •_BESGather_Log_GatherFailedAsError: 1
    •_BESRelay_GatherMirror_ResolveVersionConflicts: 1
    •_BESRelay_GatherMirror_UpstreamCheckPeriodMinutes: 360

  • Post Results Settings
    Enterprise Server\PostResults Settings:•BufferDirectoryMaxCount: 10000
    •BufferDirectoryMaxSize: 3145728
    •FastBufferDirectoryMaxCount: 10000
    •FastBufferDirectoryMaxSize: 3145728
    •TimeoutSeconds: 300
    BES Client Settings:•_BESRelay_PostResults_ResultTimeLimit: 3
    •_BESRelay_PostResults_ResultSizeLimit: 1048576
    •_BESRelay_PostResults_QueryResultSizeLimit: 1048576
    •_BESRelay_PostResults_PostTimeoutSeconds: 600
    •_BESRelay_PostResults_ParentRelayURL:

  • Client Register Settings
    •_Enterprise Server_ClientRegister_BatchCount: 100
    •_Enterprise Server_ClientRegister_BatchDelay: 1000
    •_Enterprise Server_ClientRegister_ClientRegistrationExpirationPeriod: 1440
    •_Enterprise Server_ClientRegister_DisableChildUDPMessages: 0
    •_Enterprise Server_ClientRegister_MaxChildCount: 0
    •_Enterprise Server_ClientRegister_MaxChildRelayCount: 0
    •_Enterprise Server_ClientRegister_MinHopEstimateLimit: 0
    •_Enterprise Server_ClientRegister_MaxHopEstimateLimit: 0
    •_Enterprise Server_ClientRegister_MaxRelayPathLength: 32
    •_Enterprise Server_ClientRegister_ParentRelayURL: http://127.0.0.1:52311/cgi-bin/bfenterprise/clientregister.exe
    •_Enterprise Server_ClientRegister_PingRepetitionCount: 1
    •_Enterprise Server_ClientRegister_PingRequestPort: 52311
    •_Enterprise Server_ClientRegister_UDPMessagePort: 52311

  • Upload Manager Settings
    •_BESRelay_PostFile_ChunkSize: 0
    •_BESRelay_PostFile_ComputerFolderCount: 100
    •_BESRelay_PostFile_ThrottleKBPS: 0
    •_BESRelay_PostFile_TimeoutSeconds: 300
    •_BESRelay_UploadManager_BufferDirectoryMaxCount: 10000
    •_BESRelay_UploadManager_BufferDirectoryMaxSize: 1073741824
    •_BESRelay_UploadManager_CompressedFileMaxSize: 20971520
    •_BESRelay_UploadManager_ChunkSize: not applicable on root server
    •_BESRelay_UploadManager_ThrottleKBPS: not applicable on root server

  • Query Settings
    •_BESRelay_Query_MinTime: 60
    •_BESRelay_Query_MemoryLimit: 100
    •_BESRelay_Query_RemovalTask: 10

2

This looks like something I have seen before. Could you describe your host environment. Since you say you reverted to a snapshot, is this on some flavor of VMWare. If so, what is the host OS and what flavor of VMWare?

3

Ok :slight_smile: The host OS that BigFix is running on is Win Server 2012 R2 server. I will check with the customer what version of VMware they are using for hosting this Win server.

4

Got feedback from the customer. BigFix is installed on a Win Server 2012 R2 server hosted in Vmware Vsphere 5.5u2

Is this setup something you have seen before together with the problem described earlier in this post?

Thanks
Mikael

5

I have seen similiar results on a test server I built. It is very puzzling.

My “server” is a Thinkpad W540 with 32GB RAM and SS drive. The host OS is Windows Server 2012 and running VMWare Workstation 12. The guest OS for the BigFix Server is Windows Server 2012. The OS was installed, .Net 3.5 installed, and all updates applied. Bigfix code was downloaded. BigFix was installed, but as you described, “All Content” is the only thing shown in the console, nothing else gets populated. This was also installed via a license authorization file. DBGather log file also showed the 404 not found error. But I could load the DBGather URL in the Firefox browser.

So the error seems to indicate an internet access error. But the OS could get updates and Firefox had no problem.

In my case, I did get BigFix to populate by changing the host nic being used. I had been using the wifi adapter. When I switched to the host’s ethernet, BigFix was able to populate. Note that other guests did not have any access errors. On the same host, I ran BigFix clients on OSX and on Windows 7.

Since you have different “server” hardware, this may not be applicable to you.

So I did find a workaround, but not a cause and effect. I would have thought that the host nics were seen as the identical in the virtual environments, but that doesn’t seem to be the case.

Can you try a difference host nic ?

6

Thank you very much for your feedback. Next step would be to monitor the network traffic via WireShark

7

After a deep dive investigating this problem it turned out that the customer had an outdated IPS system that prevented traffic between the BigFix server and IBM. It is now resolved and BigFix works as expected.

1 Like