I'd be curious to know as well. I don't think we're doing anything specific to prevent that from working, but we don't provide "support" as in "we don't publish a procedure on this and probably can't help with it if it doesn't work". But, especially if you have an environment on which you can test, I'd sure like to hear your results.
Which certificates is your scanner complaining about?
The certificates that should be used by Browsers, the Console, or API interfaces - the Root Server's REST API certificate, the Web Reports and WebUI certificates - can all be replaced by custom certificates issued by your CA, with existing documented procedures.
The certificates presented by Relays, or by Clients (in PeerNest) mode, are issued internally by the "Client Certificate Authority". This chains up to a CA that we install as part of your root server, and should not be trusted by anything aside from the BigFix client itself. The BES clients are configured to trust this CA as part of the content in the masthead.afxm / actionsite.afxm; but this is often flagged by security scanners, that by default want every certificate to be trusted by every browser.
It's debatable whether you really should do that. Really, there's no reason why you should want your browser to trust certificates that are issued on-the-fly by the BigFix server. In your case, it sounds like you have a tightly controlled internal Certificate Authority, with it's own team to manage it...how comfortable would they be, if you as a BigFix administrator can now suddenly issue new certificates, that are going to be automatically trusted by every browser and device across the enterprise, because you're going to chain your 'Client Certificate Authority" to be issued by their enterprise root CA? It's certainly something you all should consider carefully.
For my part, my usual guidance is "issue the individual certificates, for the root server, for WebUI, for Web Reports, etc. from your Enterprise CA". You have client applications that should be establishing their trust from the Enterprise CA to those components.
But don't set the BigFix root server's issuing authorities to be trusted by everything else in the network. That actually reduces your security. Right now your browsers don't trust the "Client Certificate Authority" -- and they shouldnt'. I think it's much better to either risk-accept that result from the vulnerability scanner, or at best import the BigFix certificate authorities just into your scanner so that it stops flagging, but no other devices start trusting the BigFix authority.
(Recent versions of BigFix introduced a BESAdmin option that outputs our certificate chain, so you can import that to be trusted by your scanner. I think that should help in your case).
If, after careful consideration, you still want the BigFix server to issue certificates, for every device in your network, that chain up to & are trusted by your enterprise authority, we did deliver that ability as well in, in version 11.0.4. That's the "Bring your own CA" feature, described at BigFix 11.0 Patch 4 is now available!
Even with "Bring your own CA", though, I don't know whether cross-signing works as expected. I just haven't had an opportunity to try it out.