CrowdStrike Issue 2024-07-19

https://www.crowdstrike.com/blog/statement-on-windows-sensor-update/

The BigFix team is aware of widespread outages related to CrowdStrike products due to a content issue.

We are currently developing content to detect systems which are likely to be impacted (still online, but have the erroneous content present); and a Task to uninstall CrowdStrike products.

For systems that are already impacted and experiencing Blue-Screen issues / failure to boot, there is a workaround published by the CrowdStrike vendor at the above link. It involves entering Safe Mode and deleting a CrowdStrike content file. If BitLocker or other encryption is enabled on the machine, recovery keys need to be provided to enter Safe Mode. At this time the BigFix team does not see an opportunity to automate this workaround for mass deployment.

8 Likes

At this time the BigFix team has two pieces of community-supported content available. These are currently provided as-is with no support, and represent a “best effort” from the BigFix team working with a limited set of test systems. If you use this content we recommend thoroughly testing on limited systems first. While these do not come with official support we will provide a best-effort to answer questions here.

An Analysis and a Task are now available now on our GitHub page:

The Analysis can be used to detect CrowdStrike versions, and whether a potentially-vulnerable C-00000291-*.sys file is present. The corrected file was issued this morning at 05:27 UTC, only versions older than that should be affected.

The Task can be used to uninstall CrowdStrike. Note that the task has several prerequisites

  • You must first download the CSUninstall.exe utility from CrowdStrike, which requires a CrowdStrike support account; you must rename the file to match its sha1 value and place on your Root Server at wwwrootbes/bfmirror/downloads/sha1

  • If CrowdStrike updates their uninstaller so the sha1 of your CSUninstall.exe version does not match our expected hash 0904f9e46a007de7805958ad62564f9ebb143a05 you will need to update the size, sha1, and sha256 values in the Task.

  • When issuing the Uninstall action, you will need to know your CrowdStrike deployment’s unique Maintenance Code.

  • The Maintenance Code is entered as a Secure Parameter from the ‘Description’ tab of the Task; as it uses Secure Parameters, this Action cannot yet be issued from WebUI, it needs the Console.

  • As the Action uses a Secure Parameter, it cannot be Dynamically Targeted by Group or Property; specific computers must be selected when issuing the Action.

The team is aware this Uninstall action may not be helpful to those machines that are already impacted and blue-screening; those require manual intervention, in Safe Mode, and cannot be addressed through our automation.

For instructions on downloading content from GitHub, please see Tips: Downloading files from GitHub

10 Likes

Hi @JasonWalker, just FYI, the file might be there but IF there is newer version already downloaded, then issue is solved. CrowdStrike doesn’t apparently override the file when they release a new version but instead leave the old and produce a new which the app starts using, so the analysis as you have (just blindly detecting if the bad file is there) is a bit misleading…

Example machine which got the updated file/fix without bluescreening and needing for the old file to be manually deleted:

2 Likes

Thank you very much for posting this info. We hadn’t seen that behavior before. I should be able to update the Analysis shortly.

I have updated the Analysis. The CSFalconService - Corrupt Sys File(s) Detected property will only report True is the newest CS-0000291-*.sys file is older than 05:27 UTC. The CSFalconService - Corrupt File(s) Names property will continue to report the names of all CS-0000291-*.sys files that are older than 05:27 UTC, even if newer versions are also present; this may be useful if you wish to retrieve file samples or manually remove those older files.

13 Likes

Jason,

Thank you for the awesome publication and information you have given our customer. You help last night was greatly appreciated.

4 Likes