BigFix Insights ETL using live DB

I realize that documentation highly suggests using a offline copy, but curious if anyone has more real world usage how critical that is?

We have under 12,000 devices and a dedicated SQL server that is sufficiently performant, and can increase specs if required.

We also use DSA, which seems to also not be recommended as an option.

It’s not saying much, but I have a Dev instance that’s been ETLing for several years off the live DB, but it’s virtually unused environment as it’s just for testing with under 20 clients. We’re looking to finally setup the Prod instance.

With the most recent Insights, it will leverage reads on the secondary if you build an availability group.

If you want to test “going direct” with your eyes open and monitors enabled, the impact may be manageable in terms of application and server resources. Insights tend to drive large transactions, versus the core platform, so lock wait and transaction impact should be monitored.

Feel free to contact me directly if you would like to discuss.

1 Like

Thanks for the input. I believe your reference to availability group, you’re suggesting a SQL cluster? And pointing insights to use a SQL instance that isn’t the currently active server?

We have a fairly simple infrastructure where our DB server is a single dedicated instance hosted on a single server doing nothing else, and then we have a separate server for the root service + webreports + webui. For DSA, we have a similar setup (dedicated SQL instance and then a separate server for root + webreports).

Correct on the cluster, though reads on the secondary are managed “automatically” by the SQL instance. This is intended as an alternative to DSA, for HADR (SQL based replication has benefits).

Given this path is “not for you”, going direct would require some careful monitoring. Happy to provide consultation here if it helps.

1 Like