Best Practices for not annoying the hell out of my co-workers -Reboots

(imported topic written by ken@gracenote91)

I would appreciate any thoughts on how to deploy a lot of patches without a lot of reboots.

After a lot of social and technical experimentation, I am currently deploying patches without configuring BigFix to force reboots via the Post-Action behavior tab.

Thus a bunch of patches can be applied without having to reboot.

However, I have found that I need to deploy the “Reboot Needed” patch to get people to actually reboot to clear those “Pending Restart” messages so that my reporting can be accurate.

The issue is that after the reboot, the the client then forces the reboot after every remaining patch that gets applied… instead of allowing multiple patches to be applied and then doing a single reboot.

I can understand logically why this is happening.

The net result is that users are being forced to do multiple reboots in a short timeframe. Compare this to using Windows Update where all the Patch Tuesdays patches could be applied without forcing reboots.

I am wondering if anyone has any best practices/workarounds that will allow many patches to be applied without forcing multiple reboots just to get the clients to report properly.

(imported comment written by BenKus)

Hey Ken,

How about putting all your patch Fixlets in a baseline each month and then when you take the baseline action, add a restart at the end… This is probably the most common approach and I think it works well…

Alternately, you can choose to avoid restarts when taking actions and instead create an action to force a restart at night between a specific time.

Ben

(imported comment written by SystemAdmin)

Ken,

This is a very timely topic for us. We have anguished over the same reboot issue for all of time. Do you currently have a corporate patching policy? Lately several MS patches have been putting servers and workstations in a state that applications and services are not working properly (and dbs have become corrupted). And who was everyone blaming for as the cause of this disarray…yup you guessed it …BigFix and Trend. Several of us have spent hours and hours (and days and days) troubleshooting only to find that one - engineers were not rebooting servers for days or weeks after patching - and two - we were not forcing reboots for workstations.

In all cases (which was numerous) a reboot resolved the issue for good. So, because of this we just finalized a real “patching” policy and now when patches go out to workstations - they will have a “force reboot” after 5 days - but will be warned to do so earlier. Engineers now have to reboot servers immediately after patching. We had to get tough - because it wasn’t getting done and we were expending so much energy on issues that are solved with a reboot. We had servers crashing because they were not rebooted for weeks after major patching. Just can’t have that.

Now that everyone understands the risks of not rebooting and has seen it with their own eyes - the new tough policy is being welcomed - as the engineers now have a policy to back them up with their user communities to get time to reboot. And rebooting the workstations will greatly reduce the HelpDesk call volume.

I wish you good fortune on your rebooting endeavor. I can ask if ok to share our patching policy - if that might help.

Cheers,

Mike

(imported comment written by ken@gracenote91)

Thanks for the info. YEs, I would love to see your patch policy/procedures as we are in the drafting phase now.

I basically took the same approach as you, but still run into a problem.

That is, for example, if I send out 15 patches to all workstations and don’t enforce a reboot with the patch but rather send out a “Pending Restart” fixlet with “apply 3x/ whenever relevant” (but expiring after 1 day), a workstation will reboot and then often ask to reboot multiple time in a row. It is like the first reboot does not clear all the stacked up “Pending restarts”. That, or else, some patches don’t apply before a previous pending reboot clears. Once that first pending restart clears, the second patch applies - putting it back into a Pending Restart status, which then kicks off the “Pending Restart” fixlet… the end result is the endusers are being prompted to reboot their workstations 3-6 or more times in a single day… and sending nasty-grams to me.

Ben: is there some rule of thumb that I can assume works all the time with Pending Restarts/Restart Needed ?

Ken

kadler@gracenote.com

(imported comment written by BenKus)

Hey Ken,

If you put that a computer should restart and reapply 3x, then the agent will obey you if the computer needs a restart for any reason (see: http://support.bigfix.com/cgi-bin/kbdirect.pl?id=159)… Note that some apps will put computers in a pending restart state often…

You should put a “wait 24 hours between runs” (or some value like that) or just remove the reapply behavior to avoid the multiple restarts… And to answer your question, a restart should remove all pending restart action statuses (as long as the action isn’t closed/expired) and restart needed Fixlets (you might need to wait a bit after restart for the agent to send this Fixlet as unrelevant).

Ben

(imported comment written by gjeremia91)

Here’s how I’m presently handling it, complicated by the fact that some users want notifications and not all patches need a reboot:

either:

build a baseline with a postaction (lastbaseline component, with includeingroupreleavence"=false). This component determines if we need a reboot, caused by BES Action. (you can also do a pre-action, that determines if you needed to reboot before you started patching…if you want).

when

setting "PATCH_2009_Q3"="{now}" on "{now}" for client if {if(name of operating system starts with "Win") then((exists key "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\BESPendingRestart" of it AND exists value "BESPendingRestart" of key "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\BESPendingRestart" of it) of registry OR (exists key "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunServicesOnce" of it AND exists value "BESPendingRestart" of key "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunServicesOnce" of it) of registry) else (true)} setting "PATCH_rebootneededbypatch"="1" on "{now}" for client else setting "PATCH_rebootneededbypatch"="0" on "{now}" for client endif true

Now I know, when my Q3 patches were applied, and if I need a reboot (caused by the patch…assuming there is no other pre-existing reboot condition caused by BES).

OR

You can do a similar thing in the action screen for deploying baselines too. I put it in the baseline itsef because my wizard is intended to reduce operations requirements (and mistakes).

Now, I have a separate fixlet that will prompt a user to reboot if they have both “reboot needed caused by bes” AND setting “PATCH_rebootneededbypatch” (I’d also check that the effective date of the setting is > 1 or 2 hours from “now”)

Now, just to cater for the fact that they may reboot on their own (and not use the fixlet), I have another fixlet that looks for “PATCH_rebootneededbypatch”=“1” but (reboot needed caused by BES)=false. This just sets “PATCH_rebootneededbypatch”=0.

I probably did a horrible job of explaining all that because I am in a rush today…

(imported comment written by ken@gracenote91)

Thanks so much for the thoughtfull response…

I need to reread it a couple of times and digest it. Right now, I am rathole-ing on redploying all my agents because re just did a fresh reinstall of the BES server . sigh.