Citrix PVS Creating New Client ID

[mbartosh]

We have a Citrix PVS (provisioning services) environment. In this environment, the servers are restarted every night, and when they are restart a new image is applied. We have defined what is called a write cache drive which is supposed to be persistent and does not get affected by the nightly reimage. We put the Bigfix agent on the write cache drive which we call the W: drive. This was supposed to prevent the Bigfix agent from getting a new client ID every night, but it is not working. We are still getting a new ID every night. I notice that all of the client logs are in place so the entire client is not being removed and re-added. However, the KeyStorage folder is getting a new date every night.

Getting a new client ID every night causes BFI capacity scan to not work correctly. We end up with a lot of machines with no scan data because of the new ID.

[JasonWalker]

This is the scenario that Client identity match should handle. See https://help.hcltechsw.com/bigfix/9.5/platform/Platform/Installation/t_preserving_bundling_when_clientreinstalled.html

I’d start by just setting ClientIdentityMatch=100 via BESAdmin and see whether that’s enough, given your write cache drive scenario. If that’s not enough to resolve, you may have to go through the rest of the steps to preserve the client ID.

[mbartosh]

We are already using the ClientIdentityMatch=100. As far as the rest of it, that seems like a lot of work to go through every night. I guess it could be scripted, but it seems like Citrix would have a better answers since they have the persistent drive. I think we are missing something on the Citrix side.

[ageorgiev]

I have been discussing this topic with PMs for a few years now and potential solution that we were seeking is to add another level of ClientIdentityMatch (let’s say 500) where the only attributes that need to be matched between machines on registration are just the basics (Machine name, IP address, UUID, etc). ClientIdentityMatch = 100 only works if reg keys are retained; it needs the old cert file to be retained and even with those in place I still couldn’t get it scripted to work… This kind on-demand provisioning services are not new or unique to Citrix and in my view, BigFix ought to have/offer solution for them and unfortunately it doesn’t at the moment!

The worst thing about it though is not just the fact they report with new Client ID (you can work around it if that was the only issue by creating aggressive BES Client Remover schedule based on let’s say “IP Address” or “Computer Name” depends on your environment of course and what would be an unique way to identify duplicates post PVS reset) but instead it creates downstream problems on BigFix Compliance, BigFix Inventory, etc! I had a case with BFI team going back and forth for an year before eventually being classified as “Platform problem” where I couldn’t get the VM Managers data mapped to the client data for those devices just because the BFI upload data is stamped by ComputerID and things do not match up!

[mbartosh]

Our Citrix provisioning services (PVS) team created a persistent drive on each Citrix server, also know as a write cache drive. Then they added commands to save and restore the Bigfix registry hive. Below are the command used. This procedure seems to have worked the same client ID has remained the same for 7 days.

Restart Commands

Net stop besclient
sc config besclient start=demand
reg export “HKLM\SOFTWARE\WOW6432Node\Bigfix” “w:\temp\Bigfix.reg” /Y The backup, Bigfix.reg, has to be saved on the W: drive.

Startup Commands

reg import "c:\temp\Bigfix.reg"
sc config besclient start=auto
net start besclient

[mbartosh]

ageogiev,

Sadly it appears your are correct that there is no solution. I thought we had it licked with the shutdown and startup script restoring the original Bigfix registry hive, and using a persistent partition. The odd think is that the ID seems to be retained for about 6 hours, and then I see this in the BES Client log: Scheduling client reset; Computer id changed to 551744474

The odd thing is that sometimes the client ID does not change. I think understanding the scheduling reset message is going to be key.

[Kapax]

mbartosh,
have you been able to avoid the duplication?
I’m facing your same situation and your mention of the 6 hours makes me think of an issue with the default registration period.

Please let me know if you had any progress on the topic.
Andrea

[ssakunala]

Hi Andrea,
There is another post with similar issue and there is a KB article with a proposed solution.

Any official doc to address duplicate computer caused by Citrix "Citrix Provisioning Services"?

HCL should allocate resources to build the lab environment to simulate and test different virtualization technologies in today and moving forward to tomorrow world. This helps to deal with different issues and how the BigFix agent behaves especially dealing with duplicate computer matter which is a big issue in virtualization environment. My technical and logical sense is that regardless how the same virtual server is restored with preinstalled agent from snapshot or system restore method, …

https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0100334

[Kapax]

Thx Som for this answer!
In the post that you refer I see that apparently on August @mbartosh did no longer have the issue (without any explanatory reasons)… that’s also why I was trying to understand if that was remaining true and he no longer had the issue.

Thx
Andrea

[mbartosh]

I monitor the issue everyday because I don’t know when it is going to break again. After having what we thought was a permanent solution, it broke after running smoothly for about three months. We have currently been running without an issue now for about two months.

The problem is that the Bigfix client ID is generated from PDAPI rather than having a static client ID like SCCM. PDAPI seems to be very sensitive to any changes in the Windows OS. Citrix PVS can preserve the machine identity when PDAPI changes. For example, we had a problem with LAPS which was changing PDAPI, but the Citrix PVS identity remained unchanged.

Here is the solution we are using.

Our Citrix PVS team created a persistent drive. We call it the W: drive. The BES Client is installed on the W: drive.

Next we have a shutdown and startup script which saves and then restores the Bigfix registry.

Shutdown Script
shutdown.bat - note that the registry is saved to the W: drive since it is persistent.

Net stop besclient

sc config besclient start=demand

reg export “HKLM\SOFTWARE\WOW6432Node\Bigfix” “w:\Bigfix.reg” /Y

Startup Script
Startup.bat

echo %date% %time% bigfix.reg > c:\bigfix\startuplog.txt

reg import “w:\Bigfix.reg”

echo %date% %time% BESCLIENT >> c:\bigfix\startuplog.txt

sc config besclient start=auto

echo %date% %time% Before Timeout >> c:\bigfix\startuplog.txt

timeout /T 500 /NOBREAK

echo %date% %time% After Timeout >> c:\bigfix\startuplog.txt

net start besclient

[Kapax]

Thx Mike,
indeed the approach you are doing is a full restore of BigFix client registry environment…
btw: the persistent drive keeps another version of the client ID via a file in the KeyStorage folder (the client certificate) whose common name is generated after the Client ID itself.

In the environment where I have been working, I was receiving issues in the decryption of that certificate (that I understand is the point that requires the delay to allow initialization to finish), that was then triggering a client reset and a new certificate generation. So far, regardless of the delay, I’m still getting issues in that part of the initialization process.

P.S.: I’m not sure if (why) it makes sense to change besclient service start mode (demand/auto) since afterall you are only starting/stopping the client manually.

Andrea

[mbartosh]

As I said, the method Bigfix uses for checking the machine identity is very sensitive, and prone to creating a new Bigfix ID. I would check the Windows System event log and see if there is a clue there as to why Bigfix is creating a new ID. In my case, I was getting a LAPS error in the event log. Once we removed LAPS, Bigfix stopped creating a new ID.