RHEL Patching Issue

(imported topic written by Niraj Kumar Mishra)

Dear All,

Yesterday We build two relays on RHEL 5 X86_64 bit machine. Today We are applying all relevant patches on these relay machines, but I patches are showing failed status with (Exit code=1). I have checked log and EDRDeployment log. Same exit code=1 I found in log as well. Can You please advise us that why all patches returning the exit code=1 in logs.

Log file message:

Line 767: Command succeeded (Exit Code=1) wait /bin/bash “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/InstallPackages.sh” -f “2012013602” -m “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_PackageMapping_2012013602” -r “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumResolveOutput_2012013602” -g “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012013602” -e “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools” -l “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/…/…/EDRDeployData/” “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_1.rpm” “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_2.rpm” “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_3.rpm” “/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_4.rpm” (group:271312,action:271313)

EDRDeployment log:

Install Failure: yum -C -y --noplugins localinstall -c ‘/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012013602’ - Error:

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi All,

Please help me regarding the same.

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Ben,

Can you please help me, I would be really thankful to you.

Thanks

Niraj

(imported comment written by cglimson)

Hi Niraj,

The Patch RHEL team is looking into this. Apologies for not acknowledging earlier.

Regards,

Malaine

(imported comment written by Niraj Kumar Mishra)

Hi Malaine,

I am really thankful to you for your kind response. I am looking forward for your help.

Thanks a lot !!!

Warm Regards

Niraj

(imported comment written by Preeya)

Hi Niraj,

On the relay endpoint, could you check the location " /var/opt/BESRelay/wwwrootbes/bfmirr/downloads/sha1" to verify that the downloads are available on the relay. Also, would it be possible to show a few more log entries (before the failure is actually reported) for EDR_DeploymentResults.txt , and the client logs?

Regards,

Priya

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

Thanks for your nice reply. I am giving you the requested logs.Please have a look and advice.

Thanks

Niraj

(imported comment written by Raja9109)

Try this fix -
https://access.redhat.com/site/solutions/31543

(imported comment written by Preeya)

Hi Niraj,

The EDR log shows these statements -

====================================================================================================

Public key for 2012013602_1.rpm is not installed

[Wed May 14 11:20:33 EDT 2014] 2012013602 ____ Failed to install the following packages:

[Wed May 14 11:20:33 EDT 2014] 2012013602 ____ libvorbis-1.1.2-3.el5_7.6.x86_64.rpm libvorbis-devel-1.1.2-3.el5_7.6.x86_64.rpm libvorbis-1.1.2-3.el5_7.6.i386.rpm libvorbis-devel-1.1.2-3.el5_7.6.i386.rpm

[Wed May 14 11:21:12 EDT 2014] 2012014602 Install Failure: yum -C -y --noplugins localinstall -c ‘/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012014602’ - Error:

====================================================================================================

The 1st line : “Public key for 2012013602_1.rpm is not installed” could be a reason fixlet action fails.

The IBM product documentation / IEM dev works explains how to resolve this in the FAQ section -

  1. http://pic.dhe.ibm.com/infocenter/tivihelp/v26r1/topic/com.ibm.tem.doc_9.0/Patch_Man/Patch_Man_RH/c_frequently_asked_questions.html

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/Patch%20Management%20-%20RedHat%20Enterprise%20Linux%20-%20FAQ

~ Priya

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

Excellent…This could be be the reason. I really appreciate your kind help.I will try out this and let you know. I am 100% sure that issue will be resolved.

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

I am extremely sorry to ask about another RHEL Patching issue in the same thread. But if you could help me, I would be really thankful to you.

Problem description: during the patching of RHEL server, I am getting below message in client log file.

Execute prefetch plug-in taking too long to execute.
I went through the below link to rectify the issue.

http://www-01.ibm.com/support/docview.wss?uid=swg21663775

I increased the
_BESClient_ActionManager_PrefetchPlugInTimeoutSecondssettin
**g **

to 300seconds .

But still patching are failed on RHEL.

Thanks in advance.

Thanks

Niraj

(imported comment written by Preeya)

Based on the current logs, could you point me the failed fixlets? Could you resend the logs with the failurers? Also did you restart the client after you did the Timeout setting ? There’s a line in the log file “execute prefetch plug-in’ didn’t complete within 60 seconds. Black listing plug-ins matching the sha1 hash of 'name of ‘bash’ until agent is restarted”

You would need to restart your BESClient before you start any new fixlet execution.

~Priya

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

Yes, after doing client timeout setting we restarted the besclient. I will share the recent logs from the failing machine that I had executed yesterday. For logs I need to ask linux sysadmin to share.

Mean time I would like to share the action info of one the failing fixlet:

// Note: The target packages can be found at https://rhn.redhat.com/rhn/errata/details/Packages.do?eid=23518. The dependencies for the target packages will not be listed on that page.

// Note: SHA1 and SHA256 are not the same values.

begin prefetch block

//Action Properties

parameter “sitefolder” = “{client folder of site whose (name of it as lowercase starts with “patches for rhel5 native tools”)}”

parameter “cwd” = “{client folder of the current site}”

parameter “fixletid” = “{origin fixlet id of action as string}”

parameter “yumVersionOld” = “{if (exists package “yum” whose (rpm version record of it >= rpm package version record “3.2.22-20”) of rpm) then “” else “-l”}”

//File Locations

parameter “EDR_RepodataManifest” = “{parameter “sitefolder”}/EDR_RepodataManifest”

parameter “EDR_YumConfig” = “{parameter “cwd”}/EDR_YumConfig_{parameter “fixletid”}”

parameter “EDR_YumRepos” = “{parameter “cwd”}/EDR_YumRepos_{parameter “fixletid”}”

parameter “EDR_YumFilelists” = “{parameter “cwd”}/EDR_YumFilelists_{parameter “fixletid”}”

parameter “EDR_RepodataSpec” = “{parameter “cwd”}/EDR_RepodataSpec_{parameter “fixletid”}”

parameter “EDR_PackageSpec” = “{parameter “cwd”}/EDR_PackageSpec_{parameter “fixletid”}”

parameter “EDR_PackageMapping” = “{parameter “cwd”}/EDR_PackageMapping_{parameter “fixletid”}”

//Log Locations

parameter “EDR_DeployDataDir” = “{parameter “cwd”}/…/…/EDRDeployData/”

parameter “EDR_YumResolveOutput” = “{parameter “cwd”}/EDR_YumResolveOutput_{parameter “fixletid”}”

parameter “t0” = “{if (exists package “file” whose (rpm version record of it < rpm package version record “4.17-28” AND architecture of it = “x86_64”) of rpm AND NOT exists package “file” whose (rpm version record of it >= rpm package version record “4.17-28” AND architecture of it = “x86_64”) of rpm) then “file-4.17-28.x86_64” else “”}”

parameter “packages” = “{parameter “t0”}”

if {not exists setting “_BESClient_RHEL_AllowYumDownloads” whose (value of it = “1”) of client}

Failed

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/SelectRepoFiles.sh” -m “{parameter “EDR_RepodataManifest”}” -s “{parameter “EDR_RepodataSpec”}” -r “{parameter “EDR_YumRepos”}”

add prefetch item {concatenation “;” of lines of file (parameter “EDR_RepodataSpec”)}

collect prefetch items

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/CreateYumConfig.sh” -d “{parameter “cwd”}” -f “{download folder}” -p “{parameter “sitefolder”}” -r “{parameter “EDR_YumRepos”}” -c “{parameter “EDR_YumConfig”}” {parameter “yumVersionOld”}

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/ResolveDependencies.sh” -f “{parameter “EDR_YumFilelists”}” -y “{parameter “EDR_YumResolveOutput”}” -o “{parameter “EDR_PackageSpec”}” -p “{parameter “fixletid”}” -m “{parameter “EDR_PackageMapping”}” {parameter “yumVersionOld”} -c “{parameter “EDR_YumConfig”}” “{parameter “packages”}”

if { (number of lines of file (parameter “EDR_YumFilelists”) | 0 ) > 0 }

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/SelectRepoFiles.sh” -d “filelists” -m “{parameter “EDR_RepodataManifest”}” -s “{parameter “EDR_RepodataSpec”}” -r “{parameter “EDR_YumRepos”}”

add prefetch item {concatenation “;” of lines of file (parameter “EDR_RepodataSpec”)}

collect prefetch items

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/CreateYumConfig.sh” -d “{parameter “cwd”}” -f “{download folder}” -p “{parameter “sitefolder”}” -r “{parameter “EDR_YumRepos”}” -c “{parameter “EDR_YumConfig”}” -x

execute prefetch plug-in “/bin/bash” “{parameter “sitefolder”}/ResolveDependencies.sh” -f “{parameter “EDR_YumFilelists”}” -y “{parameter “EDR_YumResolveOutput”}” -o “{parameter “EDR_PackageSpec”}” -p “{parameter “fixletid”}” -m “{parameter “EDR_PackageMapping”}” {parameter “yumVersionOld”} -c “{parameter “EDR_YumConfig”}” “{parameter “packages”}”

endif

add prefetch item {if (exists (file (parameter “EDR_PackageSpec”)) whose (number of lines of it > 0)) then concatenation " ; " of lines of file (parameter “EDR_PackageSpec”) else “”}

endif

end prefetch block

parameter “EDR_PackageList” = “{if (exists (file (parameter “EDR_PackageMapping”)) whose (number of lines of it > 0)) then concatenation " " of ((”%22" & download path (preceding text of first " " of it) & “%22”) of lines of file (parameter “EDR_PackageMapping”)) else (parameter “packages”)}"

wait /bin/bash “{folder (parameter “sitefolder”)}/InstallPackages.sh” -f “{parameter “fixletid”}” -m “{parameter “EDR_PackageMapping”}” -r “{parameter “EDR_YumResolveOutput”}” -g “{parameter “EDR_YumConfig”}” -e “{parameter “cwd”}” -l “{parameter “EDR_DeployDataDir”}” {parameter “EDR_PackageList”}

//Cleanup

delete “{parameter “EDR_YumConfig”}”

delete “{parameter “EDR_YumRepos”}”

delete “{parameter “EDR_RepodataSpec”}”

delete “{parameter “EDR_PackageSpec”}”

delete “{parameter “EDR_PackageMapping”}”

delete “{parameter “EDR_YumResolveOutput”}”

delete “{parameter “EDR_YumFilelists”}”

Once again thank you so much for your great support

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

I am attaching logs for the two failing machines. If you could please look into these logs and advise me, I would be really thankful to you.

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

Can you please help me.

Thanks

Niraj

(imported comment written by Preeya)

Hi Niraj,

The setting to be applied for the endpoint is “_BESClient_ActionManager_PrefetchPlugInTimeoutSeconds” , I see that both your clients have the field name set as "
_BESClient_ActionManager_PrefetchPlugInTimeoutSecondssettin
g".

Change the name and try running the fixlets again.

Thanks, Priya

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

I am sorry but as per below IBM technote setting name mentioned like “
_BESClient_ActionManager_PrefetchPlugInTimeoutSecondssettin
g
”.

http://www-01.ibm.com/support/docview.wss?uid=swg21663775

I am going to change the setting name as per your instruction and try to apply the baseline again and will let you know the result.

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

Still some of the patches are failing… I will share the recent log again

Thanks a lot.

Thanks

Niraj

(imported comment written by Niraj Kumar Mishra)

Hi Priya,

As per your suggestion, I did the changes and try to apply the baseline on failing machines. This time some of the patches got applied but still most of them are failing.

I am giving you the log info for one of the failing fixlet.

Line 582: Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

            Line 1894:    ItemizedDownloadsAvailable: true (action id 307796)



            Line 1895:    ActionLogMessage: (group:307323,action:307796) Non-Distributed - DownloadsAvailable



            Line 1896:    ActionLogMessage: (group:307323,action:307796) Submitting download request



            Line 1897:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-enhanced-7.0.109-7.2.el5.x86_64.rpm'



            Line 1898:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-minimal-7.0.109-7.2.el5.x86_64.rpm'



            Line 1899:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-common-7.0.109-7.2.el5.x86_64.rpm'



            Line 1908:    ActionLogMessage: (group:307323,action:307796) Execute prefetch plug-in taking too long to execute.



            Line 1911:    ActionLogMessage: (group:307323,action:307796) ending sub action



            Line 2297:    Not Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

Also I am attaching the log file as well. Can you please look into this and advise me.

Thanks and Regards

Niraj

(imported comment written by Preeya)

Hi Niraj,

Going through the logs, I didn’t see any failed fixlets. Fixlets in your log file are either fixed or non relevant. The EDR log doesn’t show any failed fixlets either. Which are the failed fixlets that you see?

Also for the baseline that you run - are these fixlets still relevant? Check the attachment : Non-Relevant fixlets.

Thanks,

Priya