How do I get my relays to store and provide packages/tasks?

Usage and Config
1

Here’s what happens in the logs of a client which is set to use our India relay as its primary and our datacenter relay as its backup. I asked the master IEM server to deploy a package to it.

Current Date: October 9, 2015
   Client version 9.2.4.2 built for OSX 10.60 i386
   Current Balance Settings: Use CPU: True Entitlement: 0 WorkIdle: 10 SleepIdle: 480
   ICU data directory: '/Library/Application Support/BigFix/BES Agent'
   ICU init status: SUCCESS
   ICU report character set: windows-1252
   ICU fxf character set: windows-1252
   ICU local character set: UTF-8
   ICU transcoding between fxf and local character sets: ENABLED
   ICU transcoding between report and local character sets: ENABLED
At 00:01:29 +0530 - 
   Successful Synchronization with site 'mailboxsite' (version 33) - 'http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131'
At 00:03:13 +0530 - 
   Report posted successfully
At 00:07:52 +0530 - 
   Successful Synchronization with site 'actionsite' (version 2662) - 'http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite'
At 00:09:42 +0530 - 
   Report posted successfully
At 00:13:55 +0530 - 
   Report posted successfully
At 00:15:05 +0530 - 
   GatherHashMV command received.
At 00:15:08 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
   Downloaded 'http://IN19INIEM001.company.tld:52311/mailbox/files/44/30/4430d8d5fa34b5442e0526e0dba5c04c66e5432d' as 'Action 1138.fxf'
   Gather::SyncSiteByFile adding files - count: 1
At 00:15:09 +0530 - 
   Successful Synchronization with site 'mailboxsite' (version 34) - 'http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131'
   Processing action site.
At 00:15:12 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
   Relevant - Deploy X-Lite (fixlet:1138)
At 00:15:13 +0530 - 
   ActionLogMessage: (action:1138) Action signature verified for Downloads
   DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0'
At 00:15:14 +0530 - 
   DownloadsAvailable: false (action id 1138)
At 00:15:21 +0530 - 
   Report posted successfully
At 00:15:22 +0530 - 
   Retry warning, attempt 2 succeeded for ForceNonexistence (/Library/Application Support/BigFix/BES Agent/__BESData/__report)
At 00:17:35 +0530 - 
   DownloadPing command received (ID=1138)
   DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0'
At 00:17:36 +0530 - 
   DownloadsAvailable: true (action id 1138)
   DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0'
   DownloadsAvailable: true (action id 1138)
   ActionLogMessage: (action:1138) Non-Distributed - DownloadsAvailable
   ActionLogMessage: (action:1138) Submitting download request
   ActionLogMessage: (action:1138) Download url: 'http://DC01HQIEM001.company.tld:52311/Uploads/5a488acc79a738a407353a1922868969530663df/X-Litepkg.tmp'
At 00:19:03 +0530 - 
   ActionLogMessage: (action:1138) Non-Distributed - DownloadsAvailable
   ActionLogMessage: (action:1138) Action signature verified for Execution
   ActionLogMessage: (action:1138) starting action
At 00:19:03 +0530 - actionsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite)
   Command succeeded (Prefetch download manager collected file) prefetch X-Litepkg.tmp sha1:5a488acc79a738a407353a1922868969530663df size:43395863 http://DC01HQIEM001.company.tld:52311/Uploads/5a488acc79a738a407353a1922868969530663df/X-Litepkg.tmp sha256:02d2421d092509b09de001bfb45c02fcdcc9e47411d0c36152c130872b3d77c9 (action:1138)
At 00:19:04 +0530 - actionsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite)
   Command succeeded extract X-Litepkg.tmp (action:1138)
   Command started - wait installer -pkg "__Download/X-Lite.pkg" -target / (action:1138)
At 00:19:08 +0530 - 
   Report posted successfully
At 00:19:18 +0530 - actionsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite)
   Command succeeded (Exit Code=0) wait installer -pkg "__Download/X-Lite.pkg" -target / (action:1138)
At 00:19:23 +0530 - actionsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite)
   Fixed - Deploy X-Lite (fixlet:586)
At 00:19:23 +0530 - 
   ActionLogMessage: (action:1138) ending action
At 00:19:23 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
   Not Relevant - Deploy X-Lite (fixlet:1138)
At 00:19:24 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
   Not Relevant - Deploy X-Lite (fixlet:587)
At 00:20:34 +0530 - 
   Report posted successfully
At 00:22:52 +0530 - 
   Successful Synchronization with site 'actionsite' (version 2662) - 'http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite'

So you can see it gets the job from the master (DC01HQ) server, asks the IN15IN server for the package which tells it it doesn’t have it, and then it gets it from the master server.

I did not set up this installation of IEM and am unclear about the configuration details. Looking at the way these relays are set up makes me suspect they might not have been properly configured to cache packages; they only have 40GB system HDDs, whereas the master server has its 40GB system drive and a 500GB package storage drive attached from our SAN.

Is there a way I can confirm these machines are actually replicating packages from the master server, and if they’re not, can I alter their configuration now to do so?

Thanks.

2

If the client cannot get the package from its relay it does not try anything else – it’ll just sit in a pending downloads state.

Just because you see:

prefetch X-Litepkg.tmp sha1:5a488acc79a738a407353a1922868969530663df size:43395863 http://DC01HQIEM001.company.tld:52311/Uploads/5a488acc79a738a407353a1922868969530663df/X-Litepkg.tmp sha256:02d2421d092509b09de001bfb45c02fcdcc9e47411d0c36152c130872b3d77c9

The client is not using the URL to download – it’s just taking the hash and asking the relay for the download.

Your clients appear to be working just fine.

3

How do I confirm they’re getting the package from the relay rather than the master server?

4

Just to add: I don’t intend to be obtuse. But I don’t know where the relay is storing packages IF it’s storing packages, and I don’t understand why it is using that URL for the master server when I can’t see why it’d need it.

5

You can check for cached files on the relay here:
C:\Program Files\BigFix Enterprise\BES Relay\wwwroot\bfmirror\downloads\sha1

The confusion is that you’re misunderstanding what is being logged and the purpose of that URL in the log.

Let’s walk through the log (Log messages listed first then summary):

At 00:15:08 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
Downloaded ‘http://IN19INIEM001.company.tld:52311/mailbox/files/44/30/4430d8d5fa34b5442e0526e0dba5c04c66e5432d’ as 'Action 1138.fxf’
Gather::SyncSiteByFile adding files - count: 1
The client has noticed a change in its mailbox site and has downloaded the action

At 00:15:09 +0530 -
Successful Synchronization with site ‘mailboxsite’ (version 34) - 'http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131’
Processing action site.
Processing a new action

At 00:15:12 +0530 - mailboxsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/mailboxsite2970131)
Relevant - Deploy X-Lite (fixlet:1138)
The client knows it now needs to deploy X-Lite (From the action #1138)

At 00:15:13 +0530 -
ActionLogMessage: (action:1138) Action signature verified for Downloads
DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0’
The client sees there is a download in the action and contacts the relay IN19INIEM001 for the download

At 00:15:14 +0530 -
DownloadsAvailable: false (action id 1138)
Relay says, “I don’t have the download, check back in a little bit”

At 00:15:21 +0530 -
Report posted successfully
At 00:15:22 +0530 -
Retry warning, attempt 2 succeeded for ForceNonexistence (/Library/Application Support/BigFix/BES Agent/__BESData/__report)
At 00:17:35 +0530 -
DownloadPing command received (ID=1138)
Client receives a ping, “Cheer up guy, I didn’t have the file before but I do now!”

DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0’
At 00:17:36 +0530 -
DownloadsAvailable: true (action id 1138)
Client doesn’t believe the relay so the client checks for the download, sure enough the download is available from the relay

DownloadsAvailable: checking for 'http://IN19INIEM001.company.tld:52311/bfmirror/downloads/1138/0’
DownloadsAvailable: true (action id 1138)
Client checks again for giggles

ActionLogMessage: (action:1138) Non-Distributed - DownloadsAvailable
ActionLogMessage: (action:1138) Submitting download request
ActionLogMessage: (action:1138) Download url: 'http://DC01HQIEM001.company.tld:52311/Uploads/5a488acc79a738a407353a1922868969530663df/X-Litepkg.tmp’
The client grabs the URL provided in the action and says, “This is my download URL”. If the file wasn’t on the relay or the client was set to download directly instead of using a relay, it would download it now but because the relay has the file it will take the hash of the file and ask the relay for that file

At 00:19:03 +0530 -
ActionLogMessage: (action:1138) Non-Distributed - DownloadsAvailable
ActionLogMessage: (action:1138) Action signature verified for Execution
ActionLogMessage: (action:1138) starting action
Everything’s set, lets run the action

At 00:19:03 +0530 - actionsite (http://DC01HQIEM001.company.tld:52311/cgi-bin/bfgather.exe/actionsite)
Command succeeded (Prefetch download manager collected file) prefetch X-Litepkg.tmp sha1:5a488acc79a738a407353a1922868969530663df size:43395863 http://DC01HQIEM001.company.tld:52311/Uploads/5a488acc79a738a407353a1922868969530663df/X-Litepkg.tmp sha256:02d2421d092509b09de001bfb45c02fcdcc9e47411d0c36152c130872b3d77c9 (action:1138)
I was able to download the file from the relay, let’s go!

The URL in the prefetch command is only used if the download isn’t available on the server. If this was a new windows patch the server/relay would say, “I don’t have that file, I’ll follow the URL to get the file!”.

Because this is an upload (likely from software distribution dashboard) the client asks for the file from the relay, the relay asks for it from the server, and the server grabs it from the Uploads directory and gives it to the relay who gives it to the client.

Let me know if you’d like further explanation for any part and I hope this was helpful!

3 Likes
6

Great explanation…