Very long site evaluation times

Usage and Config
1

I was working on a new fixlet and deploying iterations to my test device regularly, when it stopped responding all of a sudden. Sending a refresh did nothing, and the client logs were stuck on:

At 14:02:27 -0500 -
   Report posted successfully
At 14:02:38 -0500 - CustomSite_MyCompany_Print_Service (http://rootserver.com:52311/cgi-bin/bfgather.exe/CustomSite_MyCompany_Print_Service)
   BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 833.fxf
At 14:02:39 -0500 - CustomSite_MyCompany_Print_Service (http://rootserver.com:52311/cgi-bin/bfgather.exe/CustomSite_MyCompany_Print_Service)
   BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 835.fxf

They stayed stuck until 14:42, when the device started recognizing commands again:

At 14:02:27 -0500 -
   Report posted successfully
At 14:02:38 -0500 - CustomSite_MyCompany_Print_Service (http://rootserver.com:52311/cgi-bin/bfgather.exe/CustomSite_MyCompany_Print_Service)
   BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 833.fxf
At 14:02:39 -0500 - CustomSite_MyCompany_Print_Service (http://rootserver.com:52311/cgi-bin/bfgather.exe/CustomSite_MyCompany_Print_Service)
   BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 835.fxf
At 14:42:00 -0500 - 
   PollForCommands: Requesting commands
At 14:42:01 -0500 - 
   PollForCommands: commands to process: 4
At 14:42:48 -0500 - 
   ForceRefreshMV command received.  Version difference, gathering action site.

In besclientdebug.log, I see that during that time, the client was evaluating content in the KEV site, and finished evaluating the "Fixlet 835" in 1355 microseconds:

Thu, 04 Dec 2025 14:02:38 -0500 UnSideLined file: Fixlet 833.fxf 
Thu, 04 Dec 2025 14:02:38 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 833.fxf 
Thu, 04 Dec 2025 14:02:38 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.833:Background Evaluation 
Thu, 04 Dec 2025 14:02:38 -0500 DebugMessage BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 833.fxf 
Thu, 04 Dec 2025 14:02:38 -0500 SideLined file: Fixlet 833.fxf 
Thu, 04 Dec 2025 14:02:38 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 833.fxf: 1102 microseconds 
Thu, 04 Dec 2025 14:02:39 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 834.fxf 
Thu, 04 Dec 2025 14:02:39 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.834:Background Evaluation 
Thu, 04 Dec 2025 14:02:39 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 834.fxf: 436 microseconds 
Thu, 04 Dec 2025 14:02:39 -0500 UnSideLined file: Fixlet 835.fxf 
Thu, 04 Dec 2025 14:02:39 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 835.fxf 
Thu, 04 Dec 2025 14:02:39 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.835:Background Evaluation 
Thu, 04 Dec 2025 14:02:39 -0500 DebugMessage BackgroundAdviceEvaluation::FinishDataLoop side line file Fixlet 835.fxf 
Thu, 04 Dec 2025 14:02:39 -0500 SideLined file: Fixlet 835.fxf 
Thu, 04 Dec 2025 14:02:39 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 835.fxf: 1355 microseconds 
Thu, 04 Dec 2025 14:02:40 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 836.fxf 
Thu, 04 Dec 2025 14:02:40 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.836:Background Evaluation 
Thu, 04 Dec 2025 14:02:40 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 836.fxf: 1822 microseconds 
Thu, 04 Dec 2025 14:02:40 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 851.fxf 
Thu, 04 Dec 2025 14:02:40 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.851:Background Evaluation 
Thu, 04 Dec 2025 14:02:40 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 851.fxf: 1569 microseconds 
Thu, 04 Dec 2025 14:02:40 -0500 Evaluate file CustomSite_MyCompany_Print_Service/Fixlet 852.fxf 
Thu, 04 Dec 2025 14:02:40 -0500 DebugMessage  EvalLog CustomSite_MyCompany_Print_Service.852:Background Evaluation 
Thu, 04 Dec 2025 14:02:40 -0500 Complete file CustomSite_MyCompany_Print_Service/Fixlet 852.fxf: 730 microseconds 
Thu, 04 Dec 2025 14:02:43 -0500 Evaluation complete for site: CustomSite_MyCompany_Print_Service 
Thu, 04 Dec 2025 14:02:43 -0500 Evaluate file Known Exploited Vulnerabilities Content Pack/100_Utility_Tasks.fxf 
Thu, 04 Dec 2025 14:02:43 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.100_Utility_Tasks.fxf@00000000:Background Evaluation 
Thu, 04 Dec 2025 14:02:43 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.100:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.110:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.120:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.130:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 Complete file Known Exploited Vulnerabilities Content Pack/100_Utility_Tasks.fxf: 7474 microseconds 
Thu, 04 Dec 2025 14:02:44 -0500 Evaluate file Known Exploited Vulnerabilities Content Pack/130_Any_Version_of_MacOS.fxf 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.130_Any_Version_of_MacOS.fxf@00000000:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13100:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13130:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13190:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13210:Background Evaluation 
Thu, 04 Dec 2025 14:02:44 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13220:Background Evaluation 
Thu, 04 Dec 2025 14:03:03 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13240:Background Evaluation 
Thu, 04 Dec 2025 14:03:03 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13250:Background Evaluation 
Thu, 04 Dec 2025 14:03:03 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.13310:Background Evaluation 
Thu, 04 Dec 2025 14:03:03 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.14260:Background Evaluation 
Thu, 04 Dec 2025 14:03:03 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.14930:Background Evaluation 
Thu, 04 Dec 2025 14:03:20 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.14950:Background Evaluation 
Thu, 04 Dec 2025 14:03:31 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.14980:Background Evaluation 
Thu, 04 Dec 2025 14:03:43 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.14990:Background Evaluation 
Thu, 04 Dec 2025 14:04:04 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15000:Background Evaluation 
Thu, 04 Dec 2025 14:04:28 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15050:Background Evaluation 
Thu, 04 Dec 2025 14:05:08 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15060:Background Evaluation 
Thu, 04 Dec 2025 14:05:30 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15070:Background Evaluation 
Thu, 04 Dec 2025 14:05:33 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15080:Background Evaluation 
Thu, 04 Dec 2025 14:06:13 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15100:Background Evaluation 
Thu, 04 Dec 2025 14:06:26 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.15150:Background Evaluation 
Thu, 04 Dec 2025 14:07:12 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19340:Background Evaluation 
Thu, 04 Dec 2025 14:08:24 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19350:Background Evaluation 
Thu, 04 Dec 2025 14:10:09 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19360:Background Evaluation 
Thu, 04 Dec 2025 14:10:57 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19370:Background Evaluation 
Thu, 04 Dec 2025 14:11:43 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19380:Background Evaluation 
Thu, 04 Dec 2025 14:13:28 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19390:Background Evaluation 
Thu, 04 Dec 2025 14:14:12 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19400:Background Evaluation 
Thu, 04 Dec 2025 14:14:36 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19410:Background Evaluation 
Thu, 04 Dec 2025 14:15:00 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19420:Background Evaluation 
Thu, 04 Dec 2025 14:15:01 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19430:Background Evaluation 
Thu, 04 Dec 2025 14:15:11 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19450:Background Evaluation 
Thu, 04 Dec 2025 14:16:14 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19460:Background Evaluation 
Thu, 04 Dec 2025 14:16:40 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19470:Background Evaluation 
Thu, 04 Dec 2025 14:16:40 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19500:Background Evaluation 
Thu, 04 Dec 2025 14:17:17 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19510:Background Evaluation 
Thu, 04 Dec 2025 14:17:21 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19520:Background Evaluation 
Thu, 04 Dec 2025 14:18:04 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19540:Background Evaluation 
Thu, 04 Dec 2025 14:18:49 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19550:Background Evaluation 
Thu, 04 Dec 2025 14:20:53 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19560:Background Evaluation 
Thu, 04 Dec 2025 14:21:20 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19570:Background Evaluation 
Thu, 04 Dec 2025 14:22:04 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19580:Background Evaluation 
Thu, 04 Dec 2025 14:22:04 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19590:Background Evaluation 
Thu, 04 Dec 2025 14:22:05 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19600:Background Evaluation 
Thu, 04 Dec 2025 14:22:48 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19610:Background Evaluation 
Thu, 04 Dec 2025 14:23:13 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19620:Background Evaluation 
Thu, 04 Dec 2025 14:24:40 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19630:Background Evaluation 
Thu, 04 Dec 2025 14:25:23 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19640:Background Evaluation 
Thu, 04 Dec 2025 14:27:39 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19660:Background Evaluation 
Thu, 04 Dec 2025 14:28:23 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19670:Background Evaluation 
Thu, 04 Dec 2025 14:29:10 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19680:Background Evaluation 
Thu, 04 Dec 2025 14:29:34 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19690:Background Evaluation 
Thu, 04 Dec 2025 14:29:35 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19700:Background Evaluation 
Thu, 04 Dec 2025 14:30:19 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19710:Background Evaluation 
Thu, 04 Dec 2025 14:31:46 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19720:Background Evaluation 
Thu, 04 Dec 2025 14:32:29 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19730:Background Evaluation 
Thu, 04 Dec 2025 14:33:51 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19740:Background Evaluation 
Thu, 04 Dec 2025 14:34:36 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19750:Background Evaluation 
Thu, 04 Dec 2025 14:34:36 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19760:Background Evaluation 
Thu, 04 Dec 2025 14:34:36 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19780:Background Evaluation 
Thu, 04 Dec 2025 14:35:40 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19790:Background Evaluation 
Thu, 04 Dec 2025 14:37:06 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19800:Background Evaluation 
Thu, 04 Dec 2025 14:38:00 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19810:Background Evaluation 
Thu, 04 Dec 2025 14:38:04 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19820:Background Evaluation 
Thu, 04 Dec 2025 14:38:05 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19840:Background Evaluation 
Thu, 04 Dec 2025 14:38:05 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19850:Background Evaluation 
Thu, 04 Dec 2025 14:38:48 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19860:Background Evaluation 
Thu, 04 Dec 2025 14:39:56 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19880:Background Evaluation 
Thu, 04 Dec 2025 14:40:40 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19890:Background Evaluation 
Thu, 04 Dec 2025 14:41:24 -0500 DebugMessage  EvalLog Known Exploited Vulnerabilities Content Pack.19910:Background Evaluation 
Thu, 04 Dec 2025 14:42:00 -0500 DebugMessage PollForCommands: Requesting commands 
Thu, 04 Dec 2025 14:42:01 -0500 DebugMessage PollForCommands: commands to process: 4

We just recently enabled the KEV site, and I'm a bit concerned it's taking so long to evaluate. Is KEV not to blame, and just by chance we're seeing this behavior with this specific site?

We're running BigFix 11.0.5 (agent 204) on macOS 26.1

2

It can be difficult to say with certainty whether there's a problem here at all, or if there is one whether it's related to KEV.

The first question I'd ask is 'was it behaving that way before'? Next I'd go through the steps at Tip: Troubleshooting Client Reponsiveness which has what I think are some pretty good, general steps to go through.

It looks like you've probably done at least some of that already (kudos on both having the debug log enabled, and attaching the relevant snippet here!)

A couple of things worth observing in that. #1, is that the client doesn't seem to be receiving UDP notifications when there are new actions. At the end of that debug log, the client ran a Command Poll, where it queries the upstream Relay to ask if there's anything new. The relay returned that there are 4 new things (which might be new Actions, or Site Updates). The Relay should have sent notifications to the client in real-time when those things were getting updated, but the client didn't receive them; so the client will only detect new actions or site updates when it hits the Command Poll Interval.

Thu, 04 Dec 2025 14:42:00 -0500 DebugMessage PollForCommands: Requesting commands 
Thu, 04 Dec 2025 14:42:01 -0500 DebugMessage PollForCommands: commands to process: 4

I'd look at whether UDP on port 52311 is being blocked from the Relay to the Client; if so, then you might consider enabling Persistent Connections on both Relay and Client. If you need the client to react more quickly to new actions or content, you might also shorten the Command Poll Interval (but, setting it too short is also not adviseable, as that can add workload on the Relays as well as interrupting the client's background evaluations if its too short; I'd generally set it to not less than one hour for general machines, and maybe 15 minutes for my own machine that I'm using to test and need the actions to run faster)

Something else worth noting is that in all of this evaluation time, nothing new became Relevant; so there may not have been a need for the client to post a new report, depending on your Analysis properties and whatever evaluation and minimum reporting intervals are configured.
I probably should ask the dev team why the evaluation time is listed for some fixlets while others only say 'Background Evaluation' without a timing. If I had to guess right now I'd think maybe the ones with reported times are "new/updated" content that are doing a first-time evaluation, while the others are unchanged content that the client has evaluated before and is now just repeating in the background loop, but don't hold me to that.

Basically, I'd say this log looks like normal operation for a client that may be outside of the corporate network and not receiving UDP messages.

1 Like
3

Hi Jason,

Thanks for taking a look. UDP 52311 is open, and the client can receive notifications. Or at least, it could. I found that the client was not responding to refreshes sent from the console, and was not returning queries sent from the webUI. However, it used to be able to, and other clients on the same subnet are able to refresh/respond to queries fine.

After a reboot, it seems to be working again:

ForceRefresh command received.  Version difference, gathering action site.

I'm unsure where/when the issue occurred, or where to start debugging (the agent, macOS, security software, network?). I'll try deploying some of the same recent content to other machines and see if they start acting up as well.

But thanks for confirming the KEV site is not to blame, and it looks like it was just a coincidence.

4

Yeah, all of those things :slight_smile:

Since other machines on the subnet are getting notications, I'd say probably not the network / firewall / switches.

Since it started getting notifications after reboot, I think it unlikely to be mac firewall or security software.

...maybe the UDP port was already in use by another application when besclient started, so the client couldn't open the listening port? I'm not sure what we would log for that, but I'd expect something in the besclient log at startup time if that's the case. This was a common thing that would occur on Microsoft DNS servers, where their resolution to a DNS vulnerability / performance enhancement was to have their DNS server open a few thousand UDP ports at startup, which made it much more likely to conflict with 52311 for BigFix.

5

Related or not, this reminds me of a warning we see in the BESConsole, and have been seeing for quite some time:

At 12/4/2025 8:52:32 AM
in site "BES Support"
in file "SecurityVulnerabilityWarning.BESDomain"
in condition
in relevance "((exists current console user whose (master flag of it)) OR (NOT exists shared variable( "SecurityVulnerabilityWarning.ojo", "2014-06-30" ))) AND (exists fixlets (1790; 1791; 1792; 1793) whose (applicable computer count of it > 0) of current bes site) AND (not evaluation of bes license)"

	Long domain relevance evaluation (50 ms)

The referred-to relevance and domain file are all packaged HCL/BigFix content.

So... what gives? :person_shrugging:

6

Well, that's ...odd.

I don't understand why that should have a slow evaluation, all of those lookups should be really fast.

Just reading through it...it should display a warning is any of those four fixlets are relevant. The four fixlets don't even exist anymore. Looks like it's from 2014-06-30.

Without looking it up...if it's bad enough to give a console pop-up warning then it would affect BigFix itself....four fixlets may relate to root, relay, web reports, and webui...from 2014... I'd have to guess OpenSSL Heartbleed maybe?

It would have popped-up a warning to all master operators; and, to all other users too, until a master op acknowledges the message (which would create that dashboard variable for "SecurityVulnerabilityWarning.ojo"); once a master-op acknowledges the message, non-master-ops would stop seeing the warning but master ops still would.

Anyway, that's my guess at archaeology for today. I'll ask the Platform team to look at removing that BESDomain file.