Anyone seen this message before in relay log files and how to:
I tried adding:
[Software\BigFix\Enterprise Server\PostResults]
BufferDirectoryMaxSize = 10737418240
BufferDirectoryMaxCount = 15000
to the besrelay.config file on linux and bouncing the relay service but it makes zero difference.
Normally we’d set these using an Action to apply client settings
_BESRelay_UploadManager_BufferDirectoryMaxSize
and
_BESRelay_UploadManager_BufferDirectoryMaxCount
Once the client accepts them they may well end up in the besrelay.config but I’m not sure they’d be in PostResults, I’d expect to see something about UploadManager. It may take me a while to add a Linux relay to my deployment though. If the setting doesn’t appear to be taking effect in your environment try applying it as a client setting .
If the Relay never clears the files itself, it likely means that an upstream relay is stuck in which case you should also be seeing BackOff messages in the besrelay.log. If the relay is clearing content but just keeps getting overloaded with new client uploads, the default 1 GB BufferDirectoryMaxSize may be too small (it’s often too small when using Inventory or other add-ons that process a lot of Uploads).
The directories in question on a Linux relay are /var/opt/BESRelay/FillDBData/BufferDir/ and /var/opt/BESRelay/FillDBData/BufferDir/ForwardingBufferDir/ , check those two paths for files, check the oldest files with ls -lt, and ensure that the oldest files are getting purged every few seconds as that file gets forwarded up to the parent relay
I did send an action to my AWS relays to add a zero to the _BESRelay_UploadManager_BufferDirectoryMaxSize setting to a value of 10737418240
I still see this in the relay log very often:
Thu, 26 Sep 2024 17:50:39 +0000 - /cgi-bin/bfenterprise/postfile.exe (1558161152) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
The log message now does reflect the new size but I am still exceeding it? a 10 times increase.
Here are the directories you mentioned:
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/TemporaryDir/
total 0
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/ForwardingBufferDir/
total 0
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/
total 28
-rw------- 1 root root 3176 Sep 26 17:53 0000000000c47571
-rw------- 1 root root 3465 Sep 26 17:53 0000000000c47572
-rw------- 1 root root 5406 Sep 26 17:53 0000000000c47573
-rw------- 1 root root 3204 Sep 26 17:53 0000000000c47574
-rw------- 1 root root 3196 Sep 26 17:53 0000000000c47575
-rw------- 1 root root 3177 Sep 26 17:53 0000000000c47576
drwxr-xr-x 2 root root 6 Sep 26 17:53 ForwardingBufferDir
drwxr-xr-x 2 root root 6 Sep 26 17:53 TemporaryDir
Do we know for sure what this message is referring to when it comes to relay settings?
Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
I’ve done a bit of digging and haven’t found a reference for that message meaning anything outside of the FillDBData BufferDir yet. Since this is a postfile operation I supposed it’s possible it’s referring to the UploadManager directory - can you check /var/opt/BESRelay/UploadManagerData and /var/opt/BESRelay/UploadManagerData/BufferDir ?
Is 1558161152 a Computer ID for a computer in your deployment? I’m afraid I haven’t found a great reference on that log message yet but …
I’m suspecting it’s a Computer ID.
I’m suspecting it’s the ID of a computer that might be trying to upload a single very large file. Maybe a captured OS image for OS Deployment? That’s all I could think of that we’d reasonably expect to exceed 10 GB in size. You’d have to configure those settings to be larger than the expected captured image, on every Relay in the chain from the client to the root server if you’re using OS Deployment.
The directory on the Relay might be empty or nearly empty - I think it could be rejecting a client ArchiveManager upload because the upload is too large, and rejecting it before it ever shows up on the Relay itself.
Where this all started was failures of “archive now” action script from BF inventory when it goes to send up the scan data. I will have to look at those numbers but I suspect like you did that it is a computer id of some sorts.
We don’t do image deployments with BF and certainly not coming from an endpoint up to the root. It is mostly BFI data.
Thu, 26 Sep 2024 20:34:54 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:54 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:55 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:55 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:55 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:56 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:57 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:57 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:57 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:57 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:58 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:59 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:34:59 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:35:00 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:35:00 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:35:00 +0000 - /cgi-bin/bfenterprise/postfile.exe (805304064) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:35:01 +0000 - /cgi-bin/bfenterprise/postfile.exe (794814208) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
Thu, 26 Sep 2024 20:35:02 +0000 - /cgi-bin/bfenterprise/postfile.exe (1558161152) - Overloaded buffer directory reached, maxsize=10737418240 bytes, maxcount=10000
So, you are thinking that the numbers inside the () are computer IDs trying to send over 10GB of data up to the relay?
Unfortunately, those numbers inside the () are not computer IDs.
I think you’ll probably have to get Support to go through your logs with you. That was all I could guess.
Did you see a lot of usage on /var/opt/BESRelay/UploadManagerData and /var/opt/BESRelay/UploadManagerData/BufferDir ? That’s the last spot I’d think to check.
I don’t think any of these BFI scans will be that large, it’s probably just the number of them causing an issue. I was only speculating that OS Deployment could be one possible reason for huge uploads from a client.
Ya, got cases open for this.
Here are the directories you mentioned:
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/TemporaryDir/
total 0
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/ForwardingBufferDir/
total 0
[root@ip-10-150-33-106 bin]# ll /var/opt/BESRelay/FillDBData/BufferDir/
total 28
-rw------- 1 root root 3176 Sep 26 17:53 0000000000c47571
-rw------- 1 root root 3465 Sep 26 17:53 0000000000c47572
-rw------- 1 root root 5406 Sep 26 17:53 0000000000c47573
-rw------- 1 root root 3204 Sep 26 17:53 0000000000c47574
-rw------- 1 root root 3196 Sep 26 17:53 0000000000c47575
-rw------- 1 root root 3177 Sep 26 17:53 0000000000c47576
drwxr-xr-x 2 root root 6 Sep 26 17:53 ForwardingBufferDir
drwxr-xr-x 2 root root 6 Sep 26 17:53 TemporaryDir
The FillDBData/BufferDir is constantly changing, which I would expect.
Those are the first two directories we looked at, earlier (is this a new screenshot? 'cause those are the same files?)
The two directories I want to check aside from those two FillDBData directories, are the UploadManagerData directories…
These are for the UPLOAD Manager.
It is hitting either one of those settings.
Please change the MazSize setting to be double that setting and it is the total size of the DIRECTORY not of a single file.
Lets work this in the case please Joe.