Windows security patch deployment strategy, timing, wake on lan etc

Usage and Config Patch
1

Hello all -

I was curious how others were going about their deployment strategy here? I’ve been trying to get an environment of about 500 machines (workstations and servers) up to date with windows security patches here and have been mostly successful, but there seems to be a number of machines that do not get patched and I don’t quite have it isolated down as to why, and which machines.

I’ve been trying to deploy the patches during early morning hours that wouldn’t be intrusive to the end users, by using wake on lan and wake from standby schedules. The servers are easy since they’re always up and available, but the workstations are proving challenging as many laptops are not always present… in addition, a couple of remote offices have about a third of their machines that aren’t getting hit by the patches so I assume the wake on lan isn’t working for every machine.

I think the best thing to do here is to troubleshoot wake on lan and why some of the machines aren’t waking. What’s the best way to go about identifying WHICH computers aren’t waking, and how can I troubleshoot them remotely? I’d like to be able to patch all machines after business hours ideally. We do have a baseline policy for Wake-on-Lan setting the following to all machines:

00-10447: BES Client Setting: Designate Wake-on-LAN Forwarders
00-10448: Enable Wake-from-Standby by Magic Packet - Windows XP/Vista/Win7/2008 and Mac OS
00-10449: PC Narcolepsy: Set System Unattended Sleep Timeout - Windows Vista/7

Other than that, is anyone deploying patches during business hours to troublesome machines? I have been itching to do that just to get them patched in the mean time, but I’d hate to invoke a restart or break some functionality during business hours as well. My past patching experience has been with wsus and I used to previously set a company patch day, where we’d roll out the patches during the middle of the business day and require a restart by end of day. I don’t think I can quite do the same here with bigfix and the new company, it needs to be as seamless as possible.

Would love to hear about how other people are handling this :slight_smile: Thanks

2

At your remote locations, is there no one you can call and work with? As you probably know, WOL requires specific BIOS settings be configured to accept the magic packet and you won’t be able to manage those remotely or confirm they’re correct without someone physically in front of the machine unless there is a bitchin inspector I’m unaware of that can check the BIOS settings for WOL settings from within the OS.

1 Like
3

there are employees but no one with direct technical expertise. I want to see how far I can get on my own first but may very well end up needing to cross train and collaborate with someone there. I figure a good starting point would be figuring out how to isolate which machines aren’t working with wake on lan?

4

Without collaboration I don’t think you’ll be able to because you’ll have to know when those machines are off and attempt to wake them up. You could potentially find one that doesn’t ping then try to wake it up then troubleshoot network connectivity and ensure your WOL packet is being sent there. Without touching the other side, I don’t know how you’ll determine if it got there.

5

I was hoping there was a clever way or a clever analyses that would tell me which machines aren’t configured correctly for it, but being that it’s bios level controls that would be pretty difficult I imagine. Almost sounds like I may need to generate a list of machines, check on them after hours, send a wake on lan, and manually maintain a list of machines that aren’t responding to it, and troubleshoot from there. Bummer.

Would still love to hear about other folks patching strategies and tactics, probably stand to learn from the experience of others!

6

The other option I’m trying to get my superiors to adopt is open actions that apply the patches to a machine whenever it’s available and produce a reboot request with an 8 hour deadline so people who turn off their machines habitually will get their machines patched and give ample time to address the necessary reboot without affecting their work because they can do it when they go to lunch or at the end of the day.

1 Like
7

You can check the BIOS settings from an Action, but it will likely require deploying a vendor-specific utility to do so. For instance I have one BIOS configuration utility from HP and another from Dell, that can query the current BIOS configuration and dump it to a file; then I have an analysis read the output from the file.

These same utilities can write new BIOS configurations as well, so we can (for instance) schedule a daily wake-up at 3am, enable Wake-on-LAN, etc.

Have a look at bigfix.me, I think there are some BIOS configuring Tasks there. If not, let me know and I’ll post something.

1 Like
8

@Entaille , @jmaple , @JasonWalker

You should be able to apply all OS patches during business hours, then separately schedule a reboot using a different task if there is a pending reboot, if the uptime of the OS is greater than X time, and other criteria.

If all systems are dual core or better CPUs with SSDs then you really don’t have to worry about disrupting users. One exception is running patches while a laptop is on battery, which does disrupt the user in a way by shortening their battery life, but this can be accounted for in relevance as well.

Designate All systems as Wake-on-LAN Forwarders.

If you are trying to establish a maintenance window, then you should create a “Scheduled wake from standby” about 15 minutes or so before the “Scheduled Wake On LAN”. The reason is that only some machines will be in standby and be awoken by this, but this will mean more machines are likely to be already awoken to help wake up other machines using “Scheduled Wake On LAN”, which requires at least 1 computer per subnet to be awake in order for it to wake the others on that subnet.

You typically need a task at the beginning of a patching baseline that is always relevant to prevent the system from sleeping and a task at the end that is always relevant to stop the process that is preventing the system from sleeping. These tasks should not make the baseline relevant. (make sure that box is unchecked) You also need another task that detects when the system is being prevented from sleeping too long and kills it.

An alternative option is to have an open action that applies an hour before the maintenance window and up to 3 hours or so after and applies to all endpoints that are apart of it. Then have another task that kills it at the end of the baseline, or after a timeout. As systems wake, they will run the task to keep them awake, whenever they happen to wake up, an hour before and during the window.

In the case of Dell systems, you can query BIOS settings with relevance using WMI & Dell Command Monitor. You can likely do this with other OEM systems, but I’ve always worked in Dell shops, so thats what I write relevance against. You can also configure the BIOS using Dell Command Configure with a task.

I already have the relevance to detect the WoL BIOS config for dell systems in one of those analyses.

1 Like
9

We have enabled the WOL feature in our environment and Also assigned 1 system in a same subnet as a WOL forwarder. But the system which we want to wake didn’t receive any WOL packets from the forwarder.

Is there any settings , do we need to change on the system itself