We know this is the case and we feel for it, but we also hope that the number of those systems is smaller and more manageable.
The issue is the people that automatically build JREs can’t responsibly do it on platforms that are EoL for long because they cannot patch them. Then we can’t generally test our stuff on platforms that are EoL for the same reasons. Our best best is to boot up an old VM we keep around for this purpose, test it, then shut it back down.
This is exactly the agreed approach, after some convincing and much light seeing that this is least work and uniform approach. Then deal with breakages if they do actually happen, which we think is minimal.
We are going to leave backups in place until some agreed to time, few weeks?? till reboots happen as that will be ultimate test.
BTW, I have taken the liberty to create some content and update the remediation tasks with some assistance from a colleague due to time restraints, as well as a roll back task. And an analysis to read the separate log from the force-fix option. Any interest?
yes, though you should put them on GitHub in addition to or instead of BigFix.Me just because it makes it easier to accept updates and fixes from anyone. Easier collaboration.
Also, all of the stuff we are posting around Logpresso and JAR replacement is actually being built using templates, so we need to integrate any changes into the templates themselves.
We are working very hard to refine the detection and mitigation stuff. Rollback will be something we consider in the future.
I may do this tomorrow then as Its 3am again heh… and yeah I really need to use github for this much more. So much more convenient but I have just been putting it off. Will make a crack at it tomorrow, feel its quite useful additions
Yeah this is the challenge, I had to do something today out of necessity but it actually came out really well.
I will detail it tomorrow as it is mainly enhancements to existing tasks and of course new tasks for rollback and separate analysis for mitigations, including analysing the mitigated files, locked files and numbers of archived zip files (now stored inside BPS-scan folder).
FYI, the Logpresso Scan Utility based detection of Log4j fixlet is now live in the BES Inventory and License site.
Thanks. I also see there is a corresponding analysis too. I deleted the analysis in favor of the one in the site to avoid seeing duplicates in webreports.
I think congratulations are in order - nice work by all involved to get to an official published solution in 2 weeks. 
@JasonWalker and myself took this extremely seriously. Jason was on this sooner than me, and I’m thankful for that. We deferred our vacations and put in over 200 hours worth of work combined in less than 2 weeks. We don’t want all of you to suffer over the holidays without stepping up. We also had help from our internal teams doing QA and refinement against as many of our supported and technically unsupported platforms as possible. The one that is in production supports all but 3 platforms and the one on GitHub that uses an preinstalled JRE/JDK should cover those if it detects JAVA. We are not done, though the lack of downloadable JRE on HP-UX does not seem to be solvable anytime soon.
We could not have done this without help and feedback from all of you. Through this very busy thread we found holes in all other approaches that we started down based upon other recommendations and solutions. Thanks to @nicksberger for some specific recommendations and investigations. Thanks to @Mario for doing initial work on a universal JRE solution we ended up shipping.
I also want to mention that @strawgate ‘s and others approach of looking at running processes / file handles is supplemental to this approach, in that it is more narrow and quicker and covers highest priority problems. The main difference is that it can be run more frequently with lower impact.
Also, a ton of thanks to the developers of the Logpresso scan utility. They have been producing multiple updates per day even on weekends, integrating feedback from myself, @JasonWalker, and many others, including passing on feedback from all of you. I’m not a Java developer, but I did look over all of the code at one point and was impressed how compact and straightforward it was. The Logpresso scan utility developer has specifically named HCL BigFix as the first official integration: https://github.com/logpresso/CVE-2021-44228-Scanner#tool-integrations
We don’t know if we will ship it in production but we do have mitigation options that use the same methods on my GitHub right now and we plan to investigate automatic roll back as well all based upon the exact same Logpresso scan utility capabilities. The difference between a full scan, full remediation, and full rollback seems to just be very slight modifications.
Please please please let us know if you come across any breaking applications from mitigation. Also let us know if you have deployed mitigation widely without issues. We think the community can help each other with the confidence to deploy mitigations in ways that we can only do collectively.
Happy Holidays to all. We really mean it.
You guys are definitely on the ball and I think there is a lot of people that owe you a well deserved drink. Thanks for the hard work and hope you catch a break.
To the other people caught in the line of fire with this around the globe that have also had their holiday plans thrown into disarray, hopefully you catch a day or 2 to breathe. I am hoping to today myself hehe.
Happy holidays!
It is exactly this kind of dedication, expertise, capability, and community support that has earned us as a BigFix customer since 2004. I cannot imagine how we would begin to access/remediate log4j without you all. Please accept a very sincere Thank you and Happy New Year!
We’ve made some big changes to the Community versions of the LogPresso Java-based Scan and Remediate Tasks, as well as adding a LogPresso-based “Remediation Rollback” task.
Updates are described in detail at Log4j CVE-2021-44228, CVE-2021-45046 Summary Page
The short version is
I’m trying to help integrate our file scan results (logpresso) with another system via REST API. Here is the query I’ve built, but I can’t get any results to appear for “log4j Scan - File Result Details”.
https://<server_and_port>/api/query?relevance=(name of it | “not found”,value of result from (bes property “log4j Scan - Last Scan Time”) of it as universal time as universal string | “not found”, (concatenation “|” of (ip addresses of it as string)) | “not found”, last report time of it as string | “not found”, concatenation “|” of values of results from (bes property “log4j Scan - File Result Details”) of it | “not found”) of bes computers whose(name of it=“C02C20E6MD6V”)
I choose a specific computer name where I know there are two file results. “log4j Scan - Last Scan Time” is showing results.
Is there something about the type of the result that is preventing showing results?
There’s nothing intrinsically “special” about these properties. I’d start by querying for only one property at a time to see which, if any, are breaking the query.
Are you getting an error response, or just an empty result?
One consideration, since we have both the “Community” version (from GitHub) and the “Offical” version (in the “BES Inventory and License” Site), we could have duplicate property names if you’ve imported the GitHub version. A definition like (bes property “log4j Scan - Last Scan Time”) could be ambiguous.
One “interesting quirk” about the creation class bes property <string> is that you will get a property but not necessarily the right one. Looking at my console where I have three copies of this property, a query like
(names of it, names of sites of source analyses of it) of bes property "log4j Scan - Last Scan Time"
yields
log4j Scan - Last Scan Time, BES Inventory and License Test
…I know there are three different copies of this property on mine, but bes property "X" only gives me one of them, and I can’t control which one it gives.
I can get all of them via
(names of it, names of sites of source analyses of it) of bes properties whose (name of it = "log4j Scan - Last Scan Time")
log4j Scan - Last Scan Time, BES Inventory and License
log4j Scan - Last Scan Time, BES Inventory and License Test
log4j Scan - Last Scan Time, Test
You might need to change your lookup pattern to something like
( concatenation "|" of values of results from (bes properties whose (name of site of source analysis of it = "BES Inventory and License" and name of it = "log4j Scan - Last Scan Time")) of it | "not found") of bes computers
Did anyone faced issue with huge memory consumption with log4j scanning, we have some input from other teams where it block 5GB of memory while scanning the system.
This process was running on those machines.
C:\Windows\Temp\log4j2-scan.exe
I haven’t seen other reports of such yet. Were you using one of the Fixlets (we haven’t updated the .EXE version in quite some time, focusing instead on the .JAR versions of the scanner)?
I wonder whether it was scanning a very large archive or something like that?
I’d first ensure you’re using the latest version, and the run with the --trace parameter.
We’re seeting Tenable/Nessus log4j plugins consume mass quantities of CPU and RAM. There’s at least one WMI query that uses a lot of CPU, and a PowerShell using up to ~3.5GB of RAM. Turning the agent’s performance settings to “low” doesn’t seem to have an impact on these subprocesses.
The BigFix team has published an update to the “BES Inventory and License” Site, enabling updated Scans, Mitigation, and Mitigation Rollback using the 2.7.1 version of Logpresso Scanner
The Logpresso Scanner with version 2.5.3 works for RHEL 6, but with version 2.7.1 doesn’t work.
JUMP log4j-tools]$ ./log4j2-scan-2.7.1-logpresso
./log4j2-scan-2.7.1-logpresso: /lib64/libc.so.6: version GLIBC_2.15' not found (required by ./log42-scan-2.7.1-logpresso) ./log4j2-scan-2.7.1-logpresso: /lib64/libc.so.6: versionGLIBC_2.14’ not found (required by ./log42-scan-2.7.1-logpresso)
Our content should be using the Java version of the scanner, mostly to get around glibc and c++ dependencies
Hi Jason, from my experience the Java at least from several places I found had glibc version requirements itself