I have this problem mostly solved, I just wanted to document it here, which I wish I would have started doing much earlier in the process to have an ongoing record of my progress over time.
Chrome uses something called PepperFlash to provide Adobe Flash internally without requiring Flash to be installed. This allows Chrome to update Flash independently of other copies of Adobe Flash on the system, and Chrome’s update mechanism tends to work very well.
The 64bit version of Chrome requires the 64bit version of PepperFlash. Adobe Flash content will not function in the Chrome Browser if the 64bit version of Chrome attempts to use the 32bit version of PepperFlash.
The bug with Chrome that we experienced involves switching from 32bit Chrome to 64bit Chrome. Chrome will look for the latest version of PepperFlash to use in both the Current User Profile folders as well as the location Chrome is installed in ProgramFiles if it is installed per-machine using the MSI. If Chrome finds a version of PepperFlash in the User Profile equal to or newer to the version in ProgramFiles, it will use the version in the User Profile. The issue is that it will attempt to use it even if it is the 32bit version of PepperFlash instead of the 64bit version that is required in this case for 64bit Chrome. The same bug would likely happen if you switched from 64bit Chrome to 32bit Chrome, but this is not a typical case.
This bug was not discovered in testing because on systems where we tested the 64bit version of Chrome did not have a previous 32bit version of PepperFlash in the user folder.
In response to this issue, I created many relevance queries to help detect the issue, which could then be used with Fixlets/Tasks to address the problem.
- https://bigfix.me/relevance/details/3006207
- https://bigfix.me/relevance/details/3006206
- https://bigfix.me/relevance/details/3006208
- https://bigfix.me/relevance/details/3006220
- https://bigfix.me/relevance/details/3006209
- https://bigfix.me/relevance/details/3006211
- https://bigfix.me/relevance/details/3006212
- https://bigfix.me/relevance/details/3006213
One of the challenges I faced addressing this problem is that it is surprisingly hard to differentiate 64bit Chrome and 32bit Chrome.
One method to discover 64bit Chrome is that only the 32bit version seems to have the file nacl_irt_x86_32.nexe so this relevance should be TRUE for 64bit Chrome installs: https://bigfix.me/relevance/details/3006210
An even better method to discover if 64bit Chrome is installed was to look at the PepperFlash manifest.json file which declares the architecture of PepperFlash: https://bigfix.me/relevance/details/3006214
It would be nice if there was a relevance inspector for PE File Headers. It would also be nice if there were iterators for some of the file version blocks and other similar file items.
This is another option to try to detect a x64 version of chrome: https://bigfix.me/relevance/details/3006308
The next challenge is that Chrome can have multiple versions side by side in ProgramFiles, so I need to find only the newest one.
This provides the newest version number: https://bigfix.me/relevance/details/3006215
This will give the actual folder object with the newest version: https://bigfix.me/relevance/details/3006216
Once I knew the newest version of Chrome in ProgramFiles, I need to determine if it has 64bit PepperFlash: https://bigfix.me/relevance/details/3006217
The issue only happens if there is a version of PepperFlash in the User Profile that is equal to or greater than the one in ProgramFiles but 32bit instead of the 64bit required in this case. This relevance would tell me if there was a version of PepperFlash in the User folder that was equal to or newer than the one in ProgramFiles: https://bigfix.me/relevance/details/3006218
It turns out that this isn’t sufficient because that relevance didn’t check if the PepperFlash in the UserFolder was 32bit or not. If I delete PepperFlash that is equal to or newer than the one in ProgramFiles without checking if it is the 32bit version that wouldn’t work, then it would actually end up also deleting any that were 64bit. This doesn’t sound like an issue at first, but it turns out that Chrome will actually dynamically update PepperFlash when Chrome is running and it will do this by storing the correct version in the UserFolder. This means if I didn’t do the right check, then my task that would delete the wrong PepperFlash would actually also delete the correct one in the future when Chrome updates it later.
This resulted in this relevance to detect when there is a 32bit version of PepperFlash in the UserFolder that is equal or newer than the one in ProgramFiles: https://bigfix.me/relevance/details/3006223
Which helps detect the problem when used in combination with this relevance to detect 64bit PepperFlash in ProgramFiles: https://bigfix.me/relevance/details/3006222
Then this is all combined into a single statement when it comes to actually deleting the problematic folders: https://bigfix.me/relevance/details/3006221
concatenation "%0d%0a" of ("rmdir /S /Q %22"&it&"%22") of pathnames of items 0 of it whose( item 1 of it <= version of file "pepflashplayer.dll" of item 0 of it ) of (it, ( maximum of versions of files "pepflashplayer.dll" of folders "PepperFlash" whose(exists values whose(it as string = "x64") of keys "x-ppapi-arch" of jsons of files "manifest.json" of it) of items 1 of ( ( maximum of (name of it as version) of folders of folders "Google\Chrome\Application" of folders whose(name of it starts with "Program Files") of folders "C:" ) , ( folders of folders "Google\Chrome\Application" of folders whose(name of it starts with "Program Files") of folders "C:" ) ) whose(item 0 of it = name of item 1 of it as version) ) ) of folders whose(exists values whose(it as string = "ia32") of keys "x-ppapi-arch" of jsons of files "manifest.json" of it) of folders "AppData\Local\Google\Chrome\User Data\PepperFlash" of folders of folders "C:\Users"
I wrote all of this relevance in 2 days, but I already have trouble parsing this last relevance statement. I really need to add more in-line relevance comments to keep it all straight. It really becomes hard to manage when the statements become this complex.
This should be the final product: https://bigfix.me/fixlet/details/9294
I need to do some more testing, but this appears to solve the issue.
Related:
- https://bugs.chromium.org/p/chromium/issues/detail?id=546017 (matching issue)
- https://bugs.chromium.org/p/chromium/issues/detail?id=458894 (this is the cause)
- https://chromium.googlesource.com/chromium/src.git/+/0bc0349c7d8ed530bf82b656e95cc1cbbb59f424
- https://bugs.chromium.org/p/chromium/issues/detail?id=454131
- https://bugs.chromium.org/p/chromium/issues/detail?id=572131