Yes. We believe this issue is a direct result of a rare but particularly troublesome Microsoft bug. Basically our agent writes out a file using standard API calls and Windows claims it was written properly with no errors, but it actually was written full of null files (or other bogus data). Thus when the agent restarts and reads the file, it is corrupted and the agent can’t verify the security of the operators and shuts off. We are working closely with Microsoft on this and we are able to reproduce the problem for them (outside of BigFix Agent) and they are working on a fix for this.
In the meantime, we have done exactly what you suggested:
changed the code to write out the files in different ways that Microsoft thinks will avoid the issue.
we created the “BES Client Helper Service” that can be used for troubleshooting scenarios just like this one.
Note that we have only had a few reports of this and we believe it is a very rare issue. The BES Client Helper Service will be able to identify if this issue occurs and fix it automatically so it should give you some visibility into the problem as well…
One problem with this Task to fix this problem is that all the computers must be reporting into BigFix for you to be able to target the computers “Catch 22”. This is a little vbscript which I have put in a Group Policy for Windows startup, this helps reduce the admin work and gets all your computers reporting back into BigFix.
To dig a bit deeper, it seems that this problem still appears. At first i thought it only had to do with the upgrade i did to 7.1, but now this comes back on some computers. This is what it said in the log file:
At 10:55:11 +0100 -
Starting client version 7.1.1.315
At 10:55:12 +0100 -
FIPS mode disabled by default.
Error loading revocation store
Client shutdown (Error loading revocation store)
After this I deleted the _revocation file and started the client again, and it worked.
I have not tested the helper service on it, if it happens again on that computer I will test the helper service.
*edit I went over the clients, I make a excel file from comparing the computers reported in AD for last 15 days to the Bigfix database. And get a list of computers not in bigfix. there where a total of 5 computers that had this fault out of about 2000 clients.
After applying my vbscript into a windows startup group policy, within a week I now have over 180 computers that are now reported into BigFix. This would indicate that approx. 10% of our fleet where not reporting into BigFix due to the revocations file issue.
Ben… NO!! This folder is restricted to Administrators only. I also ran another script before implementing this one that sent me emails of computers that the service was installed but not running, I manually connected to these computers and then check the log files and they all had the revocations fault (I did this for fault finding). Once I implemented the script which deletes the revocations file, all where fixed.
I’m eagerly anticipating the fix for the __revocations corruption. It is a significant issue for us. We have dozens of agents with that problem. I’m going to try the helper service until then.
We too had a rather significant issue with the revocations corruption. Luckily it appeared to all have happened to a specific set of imaged PCs - and we caught it about 2 weeks after it started (this was months ago). We utilized our Novell Zenworks backend to correct the revocations issue. However, I’m sure we didn’t catch them all and there are systems out there no longer reporting - that we may never really know about until they get reimaged. Usually we utilize BF to bailout Novell needs - but this time I had to grovel to my Novell guy and ask for his assistance
I am excited about utilizing the helper service and also eagerly awaiting the fix. Thank you BF gang for following up on the issue and finding a resolution!
Is there any reason the Helper Service (Task) cannot be utilized on Vista?
Note that we have implemented some changes in BigFix 7.2 agents that work around the Windows filesystem issues and we don’t expect that this issue will be in 7.2 agents. We continue to request Microsoft to fix the filesystem issue in a next release or patch release.
