FillDB Buffer Size Issue

Hello,

There are 30000 Endpoints in our network, Most of the systems are not reporting to the root server.
I Went through IEM Server Diagnostics for the troubleshoots and got the below issue :

I followed the below link for the referance but not able to resolve it.
“FillDB Bufferdir File Counts”

Followed the below instruction & created key : “PostResults” and specify the REG_DWORD Values as shown in below fig.
BufferDirectoryMaxSize

Component: PostResults
Registry Hive: HKLM\Software\BigFix\Enterprise Server\PostResults
Type: DWORD
Version:
Platform: Windows
Default: 3145728
Requires Restart: No
Description: It defines the maximum size of the bufferdir, in bytes. If the bufferdir has reached the defined size, PostResults will reject submissions.

Even for the testing purpose we gave 50 MB in “BufferDirectoryMaxSize”, But whatever we’are setting it up the "FillDB File Size Limit " is more then 120%.

Please Advice.
Any help is appreciated.

Thanks,
Prabhu

1 Like

This is normal if the system is under sudden unexpected load, but bad if it is constant.

This should mean that the Storage on your root server (and/or root db server if separated) is too slow to keep up with the reports coming in. You should reduce the max buffer to something closer to the default to signal to the clients downstream to back off and stop sending reports.

You should set your client’s minimum report interval to 2min, adjust the analysis reporting interval to 5 min, and put the FillDB volume and your SQL storage on SSDs.

Check your root server’s CPU & Memory usage to make sure they are in reasonable shape.

Put faster NICs and networking between your root and your top level relays, and try to have as few as possible clients reporting directly into your root server. ( Use relay affiliation groups and automatic relay selection, but your root server should not be in a relay affiliation group that any client will use. )

I’m hopeful that the IBM Developers will chime in with more suggestions and recommendations around performance and configuration. Definitely something I’d like to see a lot more documentation on.

1 Like

What James stated is exactly what I would look at. Increasing the space is not the solution.

When I have seen this occur it is because of disk performance. Look at things like the disk queue lengths. If the queue us high, then you need to look at ways to improve the disks. If you have the DB, transactions logs and IEM application on the same drive, you need to split this. I recommend 1 disk for the IEM server, 1 for the DB and 1 for the DB transaction logs. You can put the SQL application on the same drive as the IEM server or put it on another drive too.

Also check out https://www.ibm.com/developerworks/community/wikis/home?lang=en#/wiki/Tivoli%20Endpoint%20Manager/page/Performance%20Configurations to see if there are any other things you can do.

2 Likes

To add to @TheTick 's comment:

Our volumes are divided up like this:

2 SSDs in RAID1 - Windows Volume
6 HDDs in RAID5 - \BigFix Enterprise\BES Server\wwwrootbes   (bulk storage)
2 SSDs in RAID1 - \BigFix Enterprise\BES Server\*   (FillDB and others)
2 SSDs in RAID1 - SQL DBs
2 SSDs in RAID1 - SQL Transaction Logs
2 SSDs in RAID1 - SQL TempDB

We try to install & update all software through BigFix/IEM, so for us, \BigFix Enterprise\BES Server\wwwrootbes is very large.

We have a similar number of endpoints, but our main performance issue seems to be due to the number of simultaneous console operators we have and the very decentralized nature of our IT structure. A recent patch for the console we are testing seems to be helping.

Update: I would recommend PCI-E based SSDs like the Intel P3700 for SQL & FillDB with frequent backups.

3 Likes