IEM AIX/Linux - Change download directory and run shell scripts

[lordtux]

Hi all,
Two questions.

I’m starting using IEM but I have a lot of doubts, I’ll be thankfull if you could help me.
Environment: AIX and LInux RedHat.

–Is possible to only download the packages? Like what is possible with the --downloadonly option on yum command?
–Is possible to change the location where the packages will be downloaded? I’m asking this because some servers have serious constraints of space, on AIX for example java packages can take more that 500MB in some servers, and is not simple or impossible to add more space in some servers. For example, If this is possible, with the resource of only download packages, I would mount a nfs shared folder on the servers, in this way we can have only one location on the network to store all package update.

Regarding scripts:
I see that IEM have a lot of possibilities, I wanted to automate some tasks in my enviroment, I looking to learning how to for example run shell script on the remote servers using IEM, gather some results and send this for other server on the network. I have some pdfs about IEM but I will be glad if you can point to me some additional resource that have more exactly information about how I can accomplish this. Since the network where the servers are is very strict, I have some problems to get scripts results from all servers and put one another where I can work with the scans result.

[jgstew]

run shell script on the remote servers using IEM, gather some results

You can run a script with BigFix, output the results to a file, and then read those results with Relevance with an Analysis Property.

You can also get most information through Relevance directly.

Related: BigFix Documentation & Resources

---

Is possible to only download the packages?

You could technically download things and then move them somewhere and do nothing else, but that would require that you make a copy of the Fixlet or Task and modify it to do this. It would be possible to automate it through the REST API.

Is possible to change the location where the packages will be downloaded?

Sort of. You could change where the client itself is installed so that it is running from a volume with more space, but that doesn’t sound like the issue.

It seems like you want to install updates from a network volume because none of the volumes on the server are sufficient. This is technically possible but would require you to rework everything yourself. I don’t think you could do this very easily with the built in Patching Fixlets without modification.

If you are willing to take on complete automation of everything yourself, then it should be technically possible with BigFix.

[lordtux]

@jgstew many thanks for the answers.
Thanks for the documentation, I’m started reading.

It seems like you want to install updates from a network volume because none of the volumes on the server are sufficient. This is technically possible but would require you to rework everything yourself. I don’t think you could do this very easily with the built in Patching Fixlets without modification.

For example, on my servers the BesClient is located on the /opt, but in some servers there is not enough space to install some patchs. So instead of the IEM download the packages to the /opt, I would like to decide where it can be put, on the /tmp/.packages for example and perform the install from there, or for example, include on script that mount on remote FS from another server:
mount AIX_server:/public /opt/BesClient/PackagestoInstall
This would avoid filling all free space on the client and we will have a cache from the packages that were downloaded, we will use this packages on cache to perform manually install on others servers when necessary, like in some situations with IEM agetn. AIX is not easy to install and manage packages like linux, where yum do all the work.
For all that I see until now, sees that I can accomplish this, modifying the code from the actual fixlet on the IEM that perform packages updates right?

On my case, this feature to only download and choose the path where download packages is the more important by now because we spend a lot of time to download all packages manually on the AIX.

[lordtux]

And other questions, regarding the scripts.
We have too the IEM Web reports, I wanted to create one script for run in all servers on scheduled times, gathers this results and sent to another server on the network to process them, but I was thinking if is possible collect this results as well and put them on the Web Reports, to generate for example one table with servers names, disks info, serial, logged users, last logged users, etc.
It is possible?

[jgstew]

Yes, that is possible in an Analysis and WebReport. Often times those items can be reported on through relevance directly without a need to run a script at all.

You could get the results from an Analysis directly using the REST API, or you could get the results from a WebReport.

[jgstew]

As far as I know there isn’t a way to choose where the BigFix client downloads items.

Are you saying that there isn’t even enough space to download a single patch?

You could set the BigFix client’s download cache to be small, which it is by default, and it should download a patch, install it, and then delete it and move on to the next one. As long as these servers have enough space to do at least that, then it should be fine.

Otherwise, it would take a lot of manual effort to mount a network share and install patches from there using BigFix because you would have to place the items into the network share yourself and write the code to install them yourself if and only if they are needed, or you would have to re-write the fixlets provided by IBM to use the network share.

[lordtux]

Are you saying that there isn’t even enough space to download a single patch?

Yes, saddly problem and some updates like Java for AIX for example can take more that 100MB.

You could set the BigFix client’s download cache to be small, which it is by default, and it should download a patch, install it, and then delete it and move on to the next one. As long as these servers have enough space to do at least that, then it should be fine.

In my case is not possible because we have exactly time frame, provided by customer to perform our operations and besides that, I made some tests and the IEM some times take a lot of time to download one single package.

Otherwise, it would take a lot of manual effort to mount a network share and install patches from there using BigFix because you would have to place the items into the network share yourself and write the code to install them yourself if and only if they are needed, or you would have to re-write the fixlets provided by IBM to use the network share.

In this case is preferrable to us to install all patchs manually instead using BigFix. Because my big problem with AIX patchs is the time involved in searching, filtering and uploading the packages to servers using NFS, for the Linux system we can use yum to only download package on the choosen folder, I wanted to do something similar on Aix.

What I doing now is using the links that you provided me and studying the Relevance Guid and other docuements, and after that I will take a look on the fixlet and try to modify the location where the packages are downloaded.

[jgstew]

IEM some times take a lot of time to download one single package

If the IEM/BigFix root server has a large enough cache, then each package will only be downloaded once and the rest of the downloads of that same package will come from the root cache. I generally recommend making the root server cache at least 300GB or more. Alternatively you could greatly increase the cache of your top level relays and that will help significantly.

The other issue is if you don’t have a relay to cache the content closer to the endpoints being updated. You not only want a cache at the top end to speed things up, but you also want a relay to cache the content in the same data center or WAN link or similar to greatly speed things up.

It should be that only the very first download of a package is slow and each subsequent download is fast, assuming a proper relay infrastructure.

the fixlet and try to modify the location where the packages are downloaded

This should be possible, but it will be complicated and hard to do. What if multiple endpoints are being updated at the same time, and they are both trying to download the same new file to the same network share? That will be a problem.

Your best bet may be to use the REST API to determine which fixlets are relevant on the endpoints you are trying to update, pull all of the prefetches from those fixlets, download them all to the network share using a single machine to “pre-cache” them. Then take all of the relevant fixlets and copy them into a baseline, but rewrite the parts that download the items to instead point at the network location, and do the same for the installer command to point at the network location.

While I think this is technically possible, this is a huge amount of work. I would expect it to take 6 months to figure out.

[lordtux]

If the IEM/BigFix root server has a large enough cache, then each package will only be downloaded once and the rest of the downloads of that same package will come from the root cache. I generally recommend making the root server cache at least 300GB or more. Alternatively you could greatly increase the cache of your top level relays and that will help significantly.

Hmmm, I need check the amount of space available on the root IEM server. Do you have a guide that teach how this cache works on the root server and how configure? Is possible, for example, enable cache for only a few packages or the bigfix will download all available?
The problem that I reported before related with slow download was only when IEM was acquiring the package from external source, from what I remember, from the root server to the clients, was fast.

This should be possible, but it will be complicated and hard to do. What if multiple endpoints are being updated at the same time, and they are both trying to download the same new file to the same network share? That will be a problem.

Well, in this case I would make some script to create subfolders with server name.

Your best bet may be to use the REST API to determine which fixlets are relevant on the endpoints you are trying to update, pull all of the prefetches from those fixlets, download them all to the network share using a single machine to “pre-cache” them. Then take all of the relevant fixlets and copy them into a baseline, but rewrite the parts that download the items to instead point at the network location, and do the same for the installer command to point at the network location.

Thanks for the tip, I’m digging the documentation to better understand the Relevant and Action language, while I try to accomplish this I will check what I can do using the cache. I’ll be thankful if you can answer my previous question.

[jgstew]

slow download was only when IEM was acquiring the package from external source

Can you provide an example? Even if a prefetch references an external source, it is actually downloaded once and only once on the root server and then sent to relays and clients, assuming that the root server cache is large enough to not roll over an loose the contents too quickly.

---

Related:

• https://bigfix.me/fixlet/details/515
• https://bigfix.me/analysis/details/2994717
• https://bigfix.me/analysis/details/25

[lordtux]

Can you provide an example? Even if a prefetch references an external source, it is actually downloaded once and only once on the root server and then sent to relays and clients, assuming that the root server cache is large enough to not roll over an loose the contents too quickly.

I understood now.
So, after all, what I understood until now is, the cache for the packages is make on each install on endpoint client right?
Is possible to setup a cache to all applicable updates? For example, I seeing now on the IEM console to one customer that there is 40 packages available to close some bugs, I would like to make them available now on the IEM server withou installing on the clients, is possible?

I was checking the Action script on some fixlets and I starting understand how the location where packages are being do put on the clients is get. And now other doubt raised, for these servers that have 40 packages available to install, I see 40 fixlets to install every package, each fixlet has its own code that check and download the package, how this fixlets are created? IBM create one fixlet to each update available?
Because I see now that if I wanted to modify the fixlet to change the download folder or add some other action, it would be a big work because instead of modify only one fixlet, I would need modify 40 fixlets, is that?

[jgstew]

modify 40 fixlets

Yes, you would have to modify all 40. This is why it is only feasible to do with Automation through the REST API.

I would like to make them available now on the IEM server withou installing on the clients

If you install the updates to a single client, then they will become available on the root server. You could also just use all of the prefetches from all of the fixlets in a single fixlet to do this. I believe there is also a pre-cache wizard that could possibly be used as well.

[jgstew]

This is related: Change Client Install Location on AIX

It brings up a potential solution to just install the bigfix client on a different volume on the AIX server if that is possible.

I would definitely bring these issues up to IBM. I would recommend filing an RFE and a PMR on this issue and reference these forum posts. They should definitely work to address this at the client level if possible.

[lordtux]

Thanks, I will read the post.