Improving relevance efficiency

eg2428 · March 8, 2024, 2:05am

I’m trying to write a relevance statement to capture the results from the last time Adobe Remote Update Manager was run. My relevance works, but it’s so slow that most endpoints run into “Inspector Interrupted” errors.

On Macs, per Adobe, the log file is written to ~/Library/Logs/RemoteUpdateManager.log. If RUM is triggered through BigFix or our other management tools as root, then the log is written to /var/root/Library/Logs/RemoteUpdateManager.log.

My goal is to

figure out where RemoteUpdateManager.log is
find the most recently updated version of RemoteUpdateManager.log
return the last 100 lines of the most recent RemoteUpdateManager.log

It seems like RUM just appends log results to RemoteUpdateManager.log on Macs, so I’ve seen some endpoints with log files over 16,000 lines long. This is why I only want to return the last 100 lines.

First, to figure out where the RUM logs are, I cycle through the users on the machine (avoiding users with underscores in their name, since those are system accounts), and the potential location if ran as root:

Q: (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log"))
A: "RemoteUpdateManager.log" "" "RemoteUpdateManager" "" ""
A: "RemoteUpdateManager.log" "" "RemoteUpdateManager" "" ""
T: 359688
I: file

I can verify these files are different by looking at their pathnames:

Q: (pathname of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log"))
A: /users/EndpointAdmin/Library/Logs/RemoteUpdateManager.log
A: /var/root/Library/Logs/RemoteUpdateManager.log
T: 358051
I: string

So, on this endpoint, it looks like someone ran RUM manually, and it was also run as root at some point.

Now, to figure out the most recent version, I check the maximum of the modification times of these files:

Q: (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log")))
A: Thu, 07 Mar 2024 16:54:00 -0500
T: 357161
I: time

Then, to get the file that was most recently modified, I check which of the files matches the maximum modification time:

Q: (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log"))))
A: "RemoteUpdateManager.log" "" "RemoteUpdateManager" "" ""
T: 697745
I: file

Once I have the file, I can return its contents. I’m reading only what’s after the last “|” per line, just to make things easier to read.

Q: (following texts of lasts "|" of lines of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log"))))
A:  Launching the RemoteUpdateManager...
A:  RemoteUpdateManager version is : 3.1.0.3
A:  **************************************************

A: [results truncated for clarity]

A:  No new applicable Updates. Seems like all products are up-to-date.
A:  **************************************************
A:  Failed to unload the AcrobatUpdateHelper library cleanly.
A:  ##################################################
A:  Ending the RemoteUpdateManager Return Code (0)
A:  ##################################################
T: 720079
I: substring

Now, I’d like to get the last 100 lines, because the log files are massive. To do this, I check if each line number is > the total number of lines, minus 100:

Q: (following texts of lasts "|" of lines whose (line number of it > (number of lines of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log")))) - 100)) of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log"))))
A:  Starting UpdaterCore CheckForUpdate...
A:  Skipping CFU as UpdaterCore not found ...
A:  **************************************************
A:  Starting UpdaterCore DownloadUpdates...
A:  Starting HD DownloadUpdates...
A:  Using HUM library from ADC path(/Library/Application Support/Adobe/Adobe Desktop Common/HDBox/HUM.dylib) ...
A:  No updates found in HUM reponse
A:  Failed to enumerate folder (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads) while cleaning up downloaded HD updates ...
A:  Failed to cleanup some obselete update media downloaded at (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads)
A:  No new applicable Updates. Seems like all products are up-to-date.
A:  **************************************************
A:  Failed to unload the AcrobatUpdateHelper library cleanly.
A:  ##################################################
A:  Ending the RemoteUpdateManager Return Code (0)
A:  ##################################################
A:  ##################################################
A:  
A:  
A:  ##################################################
A:  ##################################################
A:  Launching the RemoteUpdateManager...
A:  RemoteUpdateManager version is : 3.1.0.3
A:  **************************************************
A:  Initializing AcrobatUpdateHelper...
A:  AcrobatUpdateHelper library initialized successfully
A:  **************************************************
A:  Starting ARM CheckForUpdate...
A:  ARM CheckForUpdates completed successfully for (com.adobe.ARMDCHelper), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.dc), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.2023), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.servicesupdater.dc), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.servicesupdater.2023), but no updates were found available
A:  No new updates are available for Acrobat/Reader
A:  **************************************************
A:  **************************************************
A:  Initializing UpdaterCore Library...
A:  Failed to find the UpdaterCore library at (/Library/Application Support/Adobe/Adobe Desktop Common/UWABox/UpdaterCore.framework/UpdaterCore) path
A:  FFC override file does not exist ... (/Library/Application Support/Adobe/AAMUpdater/1.0/AdobeUpdater.Overrides)
A:  **************************************************
A:  Starting UpdaterCore CheckForUpdate...
A:  Skipping CFU as UpdaterCore not found ...
A:  **************************************************
A:  Starting UpdaterCore DownloadUpdates...
A:  Starting HD DownloadUpdates...
A:  Using HUM library from ADC path(/Library/Application Support/Adobe/Adobe Desktop Common/HDBox/HUM.dylib) ...
A:  No updates found in HUM reponse
A:  Failed to enumerate folder (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads) while cleaning up downloaded HD updates ...
A:  Failed to cleanup some obselete update media downloaded at (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads)
A:  No new applicable Updates. Seems like all products are up-to-date.
A:  **************************************************
A:  Failed to unload the AcrobatUpdateHelper library cleanly.
A:  ##################################################
A:  Ending the RemoteUpdateManager Return Code (0)
A:  ##################################################
A:  ##################################################
A:  
A:  
A:  ##################################################
A:  ##################################################
A:  Launching the RemoteUpdateManager...
A:  RemoteUpdateManager version is : 3.1.0.3
A:  **************************************************
A:  Initializing AcrobatUpdateHelper...
A:  AcrobatUpdateHelper library initialized successfully
A:  **************************************************
A:  Starting ARM CheckForUpdate...
A:  ARM CheckForUpdates completed successfully for (com.adobe.ARMDCHelper), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.dc), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.2023), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.servicesupdater.dc), but no updates were found available
A:  ARM CheckForUpdates completed successfully for (com.adobe.acrobat.servicesupdater.2023), but no updates were found available
A:  No new updates are available for Acrobat/Reader
A:  **************************************************
A:  **************************************************
A:  Initializing UpdaterCore Library...
A:  Failed to find the UpdaterCore library at (/Library/Application Support/Adobe/Adobe Desktop Common/UWABox/UpdaterCore.framework/UpdaterCore) path
A:  FFC override file does not exist ... (/Library/Application Support/Adobe/AAMUpdater/1.0/AdobeUpdater.Overrides)
A:  **************************************************
A:  Starting UpdaterCore CheckForUpdate...
A:  Skipping CFU as UpdaterCore not found ...
A:  **************************************************
A:  Starting UpdaterCore DownloadUpdates...
A:  Starting HD DownloadUpdates...
A:  Using HUM library from ADC path(/Library/Application Support/Adobe/Adobe Desktop Common/HDBox/HUM.dylib) ...
A:  No updates found in HUM reponse
A:  Failed to enumerate folder (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads) while cleaning up downloaded HD updates ...
A:  Failed to cleanup some obselete update media downloaded at (/Users/EndpointAdmin/Library/Application Support/Adobe/RemoteUpdateManager_Downloads)
A:  No new applicable Updates. Seems like all products are up-to-date.
A:  **************************************************
A:  Failed to unload the AcrobatUpdateHelper library cleanly.
A:  ##################################################
A:  Ending the RemoteUpdateManager Return Code (0)
A:  ##################################################
A:  ##################################################
A:  
T: 5719107
I: substring

Cool! It works. But most of the time it’s way too slow (on one old endpoint, I saw it take 10 minutes in the QnA tool). I’m more surprised that I get any results at all, given how long it takes to run on most machines.

To add on top of this, I’m trying to get results from Windows machines as well. On Windows, the log file seem to get overwritten each time, so I don’t need to worry about only grabbing the last 100 lines. But I have the same issue of having to find the log file first, since it can be in %temp% or C:\Windows\Temp depending on who ran it.

Overall, I’m trying to use the follow relevance (evaluated below on a Windows machine):

Q: if (windows of operating system) then (following texts of lasts "|" of lines of (files (("C:\Users\" & it & "\AppData\Local\Temp\RemoteUpdateManager.log") of (names of users);"C:\Windows\Temp\RemoteUpdateManager.log") whose (modification time of it is (maximum of ((modification time of it ) of files (("C:\Users\" & it & "\AppData\Local\Temp\RemoteUpdateManager.log") of (names of users);"C:\Windows\Temp\RemoteUpdateManager.log")))))) else ((following texts of lasts "|" of lines whose (line number of it > (number of lines of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log")))) - 100)) of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users);"/var/root/Library/Logs/RemoteUpdateManager.log")) whose (modification time of it is (maximum of (modification time of it) of (files ((("/users/" & it & "/Library/Logs/RemoteUpdateManager.log") of names whose (it does not contain "_") of users); "/var/root/Library/Logs/RemoteUpdateManager.log")))))
A:  
A:  ##################################################
A:  ##################################################
A:  Launching the RemoteUpdateManager...
A:  RemoteUpdateManager version is : 3.1.0.3
A:  **************************************************
A:  Initializing AcrobatUpdateHelper...
A:  **************************************************
A:  Starting ARM CheckForUpdate...
A:  ARM CheckForUpdates failed for [AdobeARM]. ARM Exit Code = [713]
A:  **************************************************
A:  **************************************************
A:  Initializing UpdaterCore Library...
A:  UpdaterCore library initialized successfully.
A:  FFC override file does not exist ... (C:\Program Files (x86)\Common Files\Adobe\UpdaterResources\AdobeUpdater.Overrides)
A:  **************************************************
A:  Starting UpdaterCore CheckForUpdate...
A:  CheckForUpdates completed successfully.
A:  **************************************************
A:  Starting UpdaterCore DownloadUpdates...
A:  Starting HD DownloadUpdates...
A:  Using HUM library from ADC path(C:\Program Files (x86)\Common Files\Adobe\Adobe Desktop Common\HDBox\HUM.dll) ...
A:  No updates found in HUM reponse
A:  Failed to enumerate folder (C:\Windows\System32\config\systemprofile\AppData\Local\Adobe\RemoteUpdateManager_Downloads) while cleaning up downloaded HD updates ...
A:  Failed to cleanup some obselete update media downloaded at (C:\Windows\System32\config\systemprofile\AppData\Local\Adobe\RemoteUpdateManager_Downloads)
A:  No new applicable Updates. Seems like all products are up-to-date.
A:  **************************************************
A:  ##################################################
A:  Ending the RemoteUpdateManager Return Code (2)
A:  ##################################################
A:  ##################################################
A:  
T: 6.147 ms
I: plural substring

Is there any way to help this run more efficiently? I could evaluate all potential Windows, Mac, root, etc. filepaths and do away with the top level If-Then-Else statement, but that means I would have to grab the last 100 lines for Windows machines as well. That seems like the slowest part, so I’d prefer to only have to do that for Macs since I have to.

Thanks!

eg2428 · March 13, 2024, 6:03pm

Being under the assumption (perhaps incorrectly) that the biggest inefficiency is grabbing the last x lines of the file, I’ve submitted an idea to add this natively to Relevance:

https://bigfix-ideas.hcltechsw.com/ideas/BFLCM-I-248

Please vote if this is something that would interest you as well, thanks.

JasonWalker · March 13, 2024, 6:57pm

Part of the issue is this structure. Think of each whose() block as a nested loop…here, for every line of the file lines whose(), you are re-calculating the number of lines of the file. A file with 50 lines will calculate the number of lines fifty times… here I’ll retrieve only the ‘number of lines’ to count results for each query, to make the time differences more clear.

q: number of lines of file "c:\temp\shortfile.txt"
A: 50
T: 0.469 ms

// this will repeat the 'number of lines' calculation for each line in the file...
q: number of lines whose (line number of it > number of lines of file "c:\temp\shortfile.txt" - 2) of file "c:\temp\shortfile.txt"
A: 2
T: 14.037 ms

(I tried that same with a fifty thousand line file, but it hasn’t retrieved the answer in over ten minutes now)

A better structure may be the one posted by @atlauren at Last N lines of a file containing a string - #12 by atlauren


q: number of lines (integers in ((number of lines of it - 1),(number of lines of it))) of file "c:\temp\shortfile.txt"
A: 2
T: 0.861 ms

This only has to retrieve the number of lines in the file twice, and then retrieve just those lines by their line numbers explicitly.

There may even be a slightly faster version that only has to retrieve the number of lines once…

q: number of lines (integers in ((it - 1 ,it ) of number of lines of it)) of file "c:\temp\shortfile.txt"
A: 2
T: 0.671 ms

I think if you apply this to your query above you should get some much faster results.

This technique uses the ‘integers in’ inspector to give an explicit list of numbers, like

Q: integers in (1, 3)
A: 1
A: 2
A: 3

combining that with the lines (<line number>) of <file> inspector to retrieve those specific line numbers…