Data Import Failure - Duplicate Key Exists

I recently installed the latest BFI patch, now running 10.0.2.0. Currently using SQL Server 2012.

BigFix Inventory
Version: 10.0.2.0-20201002-2013

Data Imports are failing with the following error.

2020-10-29 19:51:07 (+0:00:00.360) ERROR: Sequel::DatabaseError: DBNAME: temadb - NativeException: com.microsoft.sqlserver.jdbc.SQLServerException: Cannot insert duplicate key row in object ‘sam.file_facts_scd’ with unique index ‘sam_file_facts_scd_computer_id_full_path_sha1_valid_to_index’.

Is there a way to resolve this error?

1 Like

Also upgraded to 10.0.2.0 this week and am seeing the same error causing failure in our import logs. We have a support case opened for this. Should be getting attention on it soon.

2 Likes

Had the same problem and it seems HCL are aware and working on the issue. The workaround is to change the following Advance Settings and then the import seems to work:

split_ms_merge -> true
raw_data_api_history_keep_days -> 1

1 Like

@ageorgiev I see an ALL CAPS warning when attempting to modify the “split_ms_merge” value.

_Split SQL query for inserting software facts_
_Splits the SQL query that is used for inserting software facts into multiple queries to reduce usage of TEMPDB in a large-scale environment. Splitting the query lengthens the import. The parameter is applicable only on Windows. WARNING: USE THIS PARAMETER ONLY WHEN YOU ARE REQUESTED BY HCL SUPPORT_ 

Is there a simple process to reverse this if I enable the setting. Any cause for concern any than longer than normal imports?

Hello,

This setting switches between two implementation. When set to “true” is causes to use the sample pattern on MS SQL as on the DB2 all the time.

There is possible to switch back. But I would not advice this per the known issue. Reversal might work when you will keep raw_data_api_history_keep_days to 0 (no history of the File Facts).

We are recommending using this option on large MS SQL deployments and also receive positive feedback that duration if also shorten. Please compare your result taking into account processed computers in Full Scan result once you switch to it.

1 Like

The settings are always in Management -> Advanced Server Settings, so yes, you can change them back - the only limitation is that you can’t change those settings if the import is currently running. You need to wait for it to finish/fail, change them back.
If you want to be on the safe side open a Support case and work through it. I just shared a workaround I got off HCL since we run our environment MS SQL but of course it comes with a risk, so decide for yourself whether you are willing to try them.

1 Like

@ageorgiev I appreciate it! As soon as I modified the advanced settings as you explained, the import completed successfully.

Good advice on the call to HCL when changing those settings. Hopefully we’ll see an official HCL fix and publishes an update to resolve the issue.

I have faced similar issue today. So, changing the below setting has any impact in long run ?
split_ms_merge → true
raw_data_api_history_keep_days → 1

I found a Knowledge article for this issue along with a workaround in the HCL Support site.

https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0084887

Hopefully, this will be fixed in the next release.

2 Likes