Dynamic Downloads

Usage and Config
1

Hello,

I’m interested in using the dynamic downloads feature.

I would like to be able to download files based on (something), I was thinking manual group membership (so its available to the client), or another tag/setting. However, I understand you cannot use relevance within the prefetch lookups.

I have created a sample task which works (I updated the DownloadWhitelist.txt)

begin prefetch block
add nohash prefetch item url=http://1.1.1.1/patch.d/manifest
collect prefetch items
add prefetch item {line 1 of download file "manifest"}
end prefetch block

This downloads manifest, which in turn downloads line one. All good.

My thinking is that I would need to change the first prefetch URL to be a dynamic CGI which will return a computer specific manifest, would that be right?

2

Alternative to that would be to generate a small script using relevance substitution to use powershell, wget, curl, etc to do the download of whatever you want.

3

Another alternative would be to upload the manifest file as a site file, with distribution to all clients. Then you reference the file as:

file "file.manifest" of folder "CustomSite_My_Custom_Site" of data folder of client

Since the file is distributed via site propagation, it’s already local to the client.

4

Bear in mind that this download is being performed by your root server, not the end client; if you made it some kind of dynamic CGI, the result would be based on what your server provided to the BES root server, not the client that’s running the action.

For that you’d need to use a ‘download now’ command so the client itself is sending the request (and loses the benefit of the root and relay caching and traffic shaping).

Also be advised that with ‘add nohash prefetch’, the file is downloaded and cached on the root and relays, at the time you take the action; if you have an action set to reapply on some interval, it will keep reusing the old download for as long as the action is open. If the hosted file changes, you’d have to stop & restart the action to see the new version of the file.

With the methods @JonL and @atlauren propose, when the file is updated your client will get the latest version, without having to restart the action.

5

We have started using this quite a bit over the last couple of years. It works great for packages that change frequently. We have several things that use this method now that use Restapi to upload the file to the site files for content that updates frequently, to minimize the frequency that you have to micromanage software distribution and task policies.

6

My galaxy brain moment was realizing that there’s nothing special about a “manifest file”. It’s a technique, not a file type. It can be in any format you like, containing whatever data you like. The point is that it’s a mechanism for storing data, reusable across many fixlets.

After experiments with data formats, I’ve settled on JSON for manifest files, and a glorious one-prefetch-block-to-rule-them-all. (Currently in testing.)

It will save massive amounts of tedious work in maintaining changes for products that change frequently, and customer-site fixlets that must be changed in lockstep.

2 Likes
7

Yeah the only thing i’d say is to keep the file size to a minimum since it’s going to send the file out to all clients. I wouldn’t use something like this for actual application files but a text file with prefetch data is fantastic for our use cases. We had some application files that were changing every couple of days, so we setup a task that runs when the install files update and re-populated the prefetch data in the text file/Manifest file (We call it a spec file just because that is the extension we use). We also started using it for Chrome/Edge/Firefox Chrome extension allow list deployment/compliance so it’s valuable even outside of dynamic downloads.

2 Likes