RHEL Patching <error> status

LouC 2023-09-25 18:07:09 UTC #1

I have this RHEL fixlet included in my baseline:
RHSA-2023:3263 - Git Security Update - Red Hat Enterprise Linux 7 (x86_64)

It was successfully installed on several machines, it failed on a few machines and on some machines it says error

Invalid action content: the action script contains a syntax error.
This action has been applied 1 time and will not be applied again.
This action has been retried 4317 times and will not be retried.

During the patch window the agent just sits there in a “waiting” status and then changes to “expired”. All the required RHEL tasks that i have in my baseline say “Not Relevant”

Delete RHEL 7 Package List File for Multiple-Package Baseline Installation
TROUBLESHOOTING: RHEL 7 Patching Deployment Logs - Cleanup
Import RPM-GPG-KEY-redhat-release - RHEL 7
Enable the Multiple-Package Baseline Installation feature - RHEL 7
Multiple-Package Baseline Installation - RHEL 7 - x86_64 - Server

I have the latest version of the RHSM Plug-in installed (v1.0.7.0). Any suggestions on what might be going on here?

JasonWalker 2023-09-25 19:34:53 UTC #2

RHEL patching uses a prefetch-plugin script that executes during the prefetch phase (before the action “executes”, to determine any prerequisite packages.
That script has returned an error. Check the /var/opt/BESClient/EDRDeployData directory for related logs.

LouC 2023-09-26 16:02:54 UTC #3

This is what i’m seeing in the logs. The folder exists but the file does not. Any thoughts as to why the file wasn’t created?

CRITICAL : Unexpected error!
CRITICAL : Exception type: <class ‘FileNotFoundError’>
CRITICAL : [Errno 2] No such file or directory: ‘/var/opt/BESClient/EDRDeployData//MultiPkgInstall.txt’

JasonWalker 2023-09-26 16:30:44 UTC #4

Just to check what you have in the Baseline … between Enable the Multiple-Package Baseline Installation feature - RHEL 7 and Multiple-Package Baseline Installation - RHEL 7 - x86_64 - Server, you should have included any patch fixlets for the RPMs you want to install - do you have other fixlets like RHSA-2023:3263 - Git Security Update - Red Hat Enterprise Linux 7 (x86_64) between those two components? Are any of those intermediate fixlets Relevant?

You may need to enable debug logging and engage Support to go through your logs, but I could conceive of a few possible problems (not sure these are problems, just things that come to mind).

If Import RPM-GPG-KEY-redhat-release - RHEL 7 is Relevant, it could be the case that your system needs to have that key imported before trying to action the Baseline. Should be safe enough to action this one separately. I could imagine that all of the intermediate fixlets could fail to resolved dependencies (during pre-action-execution, when the prefetch-plugin-script runs, if the yum/rpm/dnf utilities don’t yet trust the repo metadata because the GPG key has not been imported)
I believe you cannot have two overlapping baselines running. If you have open actions from two different RHEL baselines, it’s not guaranteed that one baseline runs to completion before another baseline starts. So it’s possible for the Delete RHEL 7 Package List File for Multiple-Package Baseline Installation component in one baseline to delete the list file while another Baseline is still running and expecting to use that file. Do you have overlapping open actions?
The Enable the Multiple-Package Baseline Installation feature task creates an (empty) MultiPkgInstall.txt file, is it possible something else is deleting it from the system between that component and the final component in the Baseline?

Another common problem, I don’t think it’s the case here, but by default prefetch-plugin scripts have to complete within 60 seconds or the BESClient terminates them and quits using them. But the rpm/yum/dnf dependency resolution often takes longer than that. From the Patching Support site, you may need to run 56 Change Timeout for Prefetch Plugins to set a longer timeout on your Linux clients. I don’t think this is the problem in your case, I think there’s a specific error message we would throw if the prefetch-plugin timeout had been reached, but it’s possibly worth checking anyway.

LouC 2023-09-26 19:42:15 UTC #5

I am seeing “Execute prefetch plug-in attempting to reuse plug-in which took too long earlier” in the bes client log. I’m going to increase the timeout and see what happens during our next attempt. Hopefully this is it. thanks!