RHEL Patching issue <error> status -- Missing required argument <hashalgorithm>= (SOLVED)

Usage and Config Patch
1

So the basic issue is that no matter which RHEL patch I deploy or what RHEL endpoint I deploy it to I am getting an action status of “error” surrounded by angle brackets (won’t display correctly on this page because they’re special html characters)

The environment:

  1. I currently have an active RHEL subscription from which I can download patches for my red hat boxes. It’s a developer’s subscription which allows me to download patches for RHEL server

  2. I have correctly configured the RHSM plugin in the BigFix Console. This means that I have:
    a. Configured the plugin on the dashboard
    b. Registered the system (the BF Server actually doing the download) in the acess.redhat.com site after loging in (I added it to the ‘registered systems’)
    c. I downloaded all the certificates (entitlement cert / identity cert) into the “RHSMProtocol\certs\cert_set_1\” folder on the BigFix server (where the RHSM plugin is installed)
    d. I’ve also enabled the GPG keys for the endpoints and I’ve extended the “plugin timeout"

The problem:
Every RHEL patch I try to send fails, but more than that: The failure registers in the BigFix console Action Status as “error”. However, I can see in the client log that it says “Ending Action” so I’m assuming it’s a failure. I’ve done some troubleshooting with no results:

a. Tested that my subscription gives me access to the specified patches. I did this two ways:
i. Run the RHSM plugin with the ‘–check-allrepos’ and ‘–check-baserepos’ argument (results were as I would have expected and are written in the RHSMPlugin.log file shared below)
ii. Registering the box itself with Red Hat and running a “yum update” directly from the computer. Once yum updated everything I went back into BigFix and the Fixlets that were relevant before no longer are relevant. So basically my subscription gives me access to the patches that BigFix is trying to install.
b. Re-installed the BigFix Client.
c. Created a brand new RHEL vm and installed nothing other than the BES Client
d. Tried with both a RHEL 6 and a RHEL 7 endpoint
e. (on edit): I found one of the packages this deployment was trying to download and was able to find it (and download it) in my ‘Entitled Repos’ by logging directly to redhat.com

And I’m sure I’m forgetting some other things I’ve tried because I’ve been troubleshooting this for a few days now.

So obviously my basic problem is that I can’t patch RHEL devices, but I don’t know why. I combed the logs, but the best I could find was this entry in the BESClient log:

ActionLogMessage: (action:4051) Missing required argument hashalgorithm=.

Note: hashalgorithm is surrounded by angle brackets.

I’ve searched for this everywhere, but I haven’t found an exact match and the one I did find here in the forum had the word ‘size’ instead of ‘hashalgorithm’ and it was resolved by upgrading the client (which I can’t do because I’m on the latest version).

Because I can’t attach text files here I’ve uploaded them to my Google Drive Public folder. Before copying them I cleared them all out from the endpoint so there would be no old data on there, then I re-ran the patch attempt:

https://drive.google.com/open?id=0Bz1wcOaAa2r8R3JRTHBrU3ZqX1E

(FYI: these are test systems in a lab so none of the data is really sensitive)

In the Shared folder are the following files:

• Client Log (20170618.log)
• The entire EDSDeployData folder (including the EDR_DeploymentResults.txt file)
• RHSM Plugin Log from the BES Server

Any help would be appreciated.

Regards,
M

2

@mxc0bbn

Does the BigFix server has Enhanced Security enabled? Try disabling Require SHA 256 Downloads if it is enabled.

https://www.ibm.com/support/knowledgecenter/en/SS6MCG_9.2.0/com.ibm.tivoli.tem.doc_9.2/Platform/Adm/c_scenarios_sha2_installation.html

3

OK, so you hit the nail RIGHT ON THE HEAD. As soon as I disabled Enhanced Security everything RHEL is running smooth as ice. GREAT CATCH!.

Now comes the question: What is causing this? SHA-1 is no longer secure so SHA256 is the way forward and you can only enable SHA256 when enhanced security is enabled.

I’m going to contact IBM on this, but in the meantime: Any ideas on what’s happening here?

Thank you,
M

4

After some additional testing I’d like to add something else I found:

So after disabling Enhanced Security and successfully testing the RHEL patches I decided to re-enable it to see if I could consistently duplicate the issue. This time the patching was successful even with Enhanced Security.

Normally this would be extremely strange, but there is a piece of the puzzle I didn’t mention because, frankly, I didn’t think it was relevant:

A few days ago my RedHat subscription expired and I renewed it…no big deal, but when the patches started failing immediately after that I thought the new subscription had generated different certificates for the system even though it was the same system (same hostname, vcpu count, OS, etc.) so I re-downloaded the certificates that put them in the RHSM plugin certs directory.

The cert names are the same and before overwriting them I opened them and spot check that they were pretty much the same cert…and it appeared to me that they were. However, that was the only other thing I did.

I guess it’s conceivable that since I enabled Enhanced Security after I brought down the certs the first time it ‘stamped’ them somehow and the new copies of the certs looked different so it was barfing up this “error” status.

I know those are a lot of assumptions, but I have no other way to explain why it all of a sudden stopped working and then after disabling and re-enabling enhanced security they work as normal again.

Hope this helps someone…

5

There is a download plugin which should be quite capable of doing Enhanced Security which could be out of date? I moved this to the patch team’s location so they can comment (though if you open a PMR they will see it)

If any certs were older (and were SHA1 based) then turning on Enhanced Security will disable them as well as it requires strong keys and SHA256 certificates.

6

Thanks Alan,

Actually that was one of the first things I did: update the plugin which was out of date…

At this point I’m still confused as to why they failed right after I renewed my Red Hat subscription…I said something above about the system ‘stamping’ or fingerprinting the cert files, but upon reflection…if that was true then the original certs that were already there should have worked after the renewal (as they appear to me that they were the same certs).

Anyway…I remain bewildered, but at least I know how to mitigate the issue…

7

@AlanM and @mxc0bbn

Facing the same issue, I traced the problem to its source. I have security configured to require SHA-256 downloads, and the RHSM plugin is the latest version.

Select a RHEL6 patch, whose task ID is 17143501 (need that info for later)

Take action on it, and it errors out:

Open the summary of the action results:

Find the offending action script error and note the file name parameter “EDR_PkgRequest”.

Decode the action script which leads to this file:

/var/opt/BESClient/EDRDeployData/EDR_PkgtRequest_17143501

The contents of the file:

url=RHSMProtocol://get.file/aHR0cHM6Ly9jZG4ucmVkaGF0LmNvbS9jb250ZW50L2Rpc3QvcmhlbC9zZXJ2ZXIvNi82U2VydmVyL3g4Nl82NC9vcy8=/Packages/rpcbind-0.2.0-13.el6_9.1.x86_64.rpm size=52576 sha1=366bf762fb990b311eee61a53e4b463a51a7a5b7

Note the absence of a SHA-256 hash. This violates the security configuration:

image.png698×464 35.2 KB

The BigFix Patch wiki, Using the Red Hat Subscription Management (RHSM) download plug-in, has this bit of information in it:

Note: The RHSMDownloadPlu​gin does not work when the Require SHA-256 Downloads option in the IBM BigFix Administration tool is enabled. When this option is enabled, all download verification use only the SHA-256 algorithm. However, there are certain Red Hat repository metadata from the vendor, which do not contain SHA-256 values for packages in the repository that are used by the plug-in.

Consider disabling the Require SHA-256 Downloads option to successfully deploy a patch. Security and package integrity is not compromised as another layer of checking and verification is done using the GPG signature of the package. For more information about the download option, see BigFix Platform Installation Guide at https://ibm.biz/BdspGv.

Given that only certain Red Hat packages are missing a SHA-256 hash, it seems likely that the timing of the renewal of the Red Hat subscription by mxc0bbn was merely a coincidence. (The Enhanced Security option is not relevant to this issue.)

Bottom line is that we have to globally allow SHA-1 downloads to successfully patch RHEL systems. I think an RFE is in order.