Stuck in Pending Restart status even after reboot

[Gujumax]

For the first time, we used Server Automation to reboot a group of systems in sequential order. However, the job gets stuck on the first group because the status of the systems stays in PENDING RESTART or in “not reported” status, even though the server has rebooted and is back up. I’ve waited over an hour to and the status never changes. This prevents it from marking this group as complete in the automatin plan so that it can move to the next group for reboots. Is this a known issue? How can I fix this? We’re on v10.0.2.

Please see the screenshot (server uptime is 0 which means it rebooted). That was Group A and never moved on to Group B, C, D… We ended up stopping the task and taking another approach which was not ideal for restarting our systems.

This is our QA env where the BF root is a VM and also we have a single relay server that is also a VM located in the domestic region to handle about 600 systems including int’l regions. Is the case of simply adding more relays to address this issue?

[JasonWalker]

You probably need to add entries to the PendingFileRenameOperations exclusion list, so the systems are not flagged as ‘pending restart’ when there are entries there; then the systems would only flag as ‘pending restart’ based on BigFix actions, not because something else on the machine is flagged as Pending Restart.

Have a look at the this thread for details:

Pending restart status issue

With the setting _BESClient_ActionManager_PendingRestartExclusions=:; all entries in the Microsoft registry key HKLM\System\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations are ignored because every entry is a path containing a colon. Using this colon followed by semicolon value, this is a wildcard approach that some customers have used with this client setting.

To exclude everything in PendingFileRenameOperations, you could send an action for

setting "_BESClient_ActionManager_PendingRestartExclusions"=":;" on "{parameter "action issue date" of action}" for client

[Gujumax]

Thanks Jason. I’ll take a look at that.

One thing to point out is that this is after patching our systems, so I understand that it would be in a Pending Restart status. It doesn’t clear the status to Complete even after the reboot which is odd.

[JasonWalker]

Right, the thing is the BESClient will not clear any Pending Restart if there are any indications a restart is still needed. What we often see is that a driver, endpoint protection, or (occasionally) a Windows Update writes a new entry to PendingFileRenameOperations during/after the restart, when we expect the key to be clean.

The presence of the values in PendingFileRenameOperations causes the BESClient to detect a restart is needed, so none of the Pending Restart action statuses are cleared.

Ignoring the registry key by applying this client setting should allow the client to clear the Pending Restart action statuses after a reboot.

[Gujumax]

Gotcha. That makes sense. Muchas Gracias Jason.

[Gujumax]

Okay, so I pushed out the PendingRestartExclusion task as you suggested to one of the system that is in Pending Restart status, but the status didn’t change even when the Pending Restart is False. Any idea?

[JasonWalker]

It will need a reboot for that setting to take effect

[JackHanna]

We never had this issue before in BigFix 9. Also why is it only, or mainly, Server 2019. Server 2012R2 servers seem to not have this issue.
Why is every 2019 server doing this?

[JasonWalker]

I think you should check the registry keys referenced earlier in this thread to determine why the client thinks the server needs to be rebooted, and if you think it’s a false detection, exclude those values from the detection.

I’ve seen a number of antivirus products (and HP printing software) misuse those registry keys for persistence, to reload themselves on every reboot.

[JackHanna]

so… shouldn’t the job reflect the reboot status of the job? So that the job shows finished after it run and reboots the computer?
you can’t have it both ways.

[JackHanna]

[JasonWalker]

So unless you’ve updated the PendingFileRenameOperations exclusion list setting, that registry entry indicates a restart is still need.

[Mike]

This should help diagnose your environment… here is the relevance we use for a “Pending Restart - Reason” property:

if (pending restart | false) then (if (windows of operating system | false) then ((("Component Based Servicing: True") of keys "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending" of it; ("Windows Update Auto Update: " & (name of it as string | "")) of values of keys "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired" of it; ("Domain Join: " & (name of it as string | "")) of values whose (name of it = "JoinDomain" or name of it = "AvoidSpnSet") of keys "HKLM\SYSTEM\CurrentControlSet\Services\Netlogon" of it; ("Pending Computer Rename: " & (item 0 of it as string | "") & " to " & (item 1 of it as string | "")) of (values "ComputerName" of keys "ActiveComputerName" of it, values "ComputerName" of keys "ComputerName" of it) whose (item 0 of it != item 1 of it) of keys "HKLM\SYSTEM\CurrentControlSet\Control\ComputerName\" of it; ("Pending File Rename Operations: " & (concatenation of substrings separated by "\??\" of it as string | "")) of (substrings separated by "%00" of (it as string)) whose (it as string != "" and it as string != "%00") of values whose (name of it = "PendingFileRenameOperations") of keys "HKLM\System\CurrentControlSet\Control\Session Manager" of it; (("BigFix Action Registered: " & (it as string | "")) of (if (exists pending restart names | false) then pending restart names else nothing))) of registry) else (if (exists pending restart names) then (("BigFix Action Registered: " & (it as string | "")) of (if (exists pending restart names | false) then pending restart names else nothing)) else "Other Reason")) else ""

Printer drivers (commonly HP, triggered because of an RDP logon to the system) and a bug in Windows Defender always adding files whenever an attachment in an email is viewed are our most common found triggers.

[Sophia]

This is extremely useful, thanks for sharing it

[orbiton]

You can add the following analysis - https://bigfix.me/analysis/details/2998574
Also take a look into the following Linkedin Post - https://www.linkedin.com/pulse/bigfix-clear-those-pesky-pending-restart-flags-brad-sexton/