BigFix Explorer slows down API responses

[fermt]

I am troubleshooting a performance issue with a brand new BigFix Integration that relies on the BigFix REST API.

The response time of the API is critical for this application to be suitable for usage. During our initial testing we realized that the API was taking around 15 seconds to return results back to the integration. Upon further investigation, I realized that queries using the /api/query endpoint were the ones taking longer than other endpoints( For example /api/computers was faster).

Then I ran the same relevance queries against /api/query on another BigFix environment and the response times were way faster(1-2 secs).

I ended up uninstalling the BigFix Explorer service on the environment where responses were very slow and that “solved” the problem, the BigFix API started returning results faster.

There is no much documentation or information on how to resolve these type of problems with BigFix Explorer neither if this is a known situation. So I am posting here to see if someone is dealing or has dealt with this in their own environments.

This my third time trying to use BigFix Explorer and the more I tried to use it the more I find problems with it.

My previous posts:

How to install BigFix Explorer on a custom path?

That. We never install applications on the C:\ drive and the Fixlet to install the BigFix Explorer service doesn’t allow to specify a custom installation path.

So... BigFix Explorer... what is it for?

In my case, we heavily rely on the stability and performance of the BigFix REST API and our automation send continuos and sometimes expensive queries against it. We have seen slow responses when the automation hits the api during a prod deployment of patching actions. We have programmatically throttled down the number of continuous calls to avoid requests to get delayed. Even after using a dedicated Web Reports for this we continue seeing performance issues. We are in the progress of renewing …

Bigfix root API /query response missing 'count'

Yes, I opened case #CS1065347 for the BESExplorer service randomly crashing. I ended up stopping the service after it crashed again just after a couple of days I had closed the case. The other “Limitation” is that when I run queries like the following: (((value of results from(bes properties "CustomProperty") of it as string)| ("")) %26 "%252522" )of bes computers whose (name of it contains "Hostname") Where I dynamically add doble quotes to the results to later build a json object using ses…

[ageorgiev]

@Bigfix root API /query response missing 'count'

(((value of results from(bes properties "CustomProperty") of it as string)| ("")) %26 "%252522" )of bes computers whose (name of it contains "Hostname")

Do you mind posting the entire query you have? The reason I am asking is if the above is how you have structured it, while it is the simplest form, it is not optimized at all and it will not scale up well. You add 4-5 custom properties to the same structure and it will completely explain performance deterioration (and this will not be Explorer-specific but it will be the same via WR QNA; WR-based API; etc.

Use something sets-bases like this, it doesn’t look pretty but it scales tramendously well. I’ve gotten it to a query pulling 50-60 properties across 20k endpoints in a single go and it runs less than a minute!

(name of item 0 of it|"Missing Name", ((year of it as string & "-" & month of it as two digits as string & "-" & day_of_month of it as two digits as string) of date (universal time zone) of it & " " & (two digit hour of it as string & ":" & two digit minute of it as string & ":" & two digit second of it as string) of time (universal time zone) of it) of last report time of item 0 of it|"Unknown", (if (size of item 1 of it >= 1) then (concatenation ";" of values of results (item 0 of it, elements of item 1 of it)) else ("N/A")), (if (size of item 2 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 2 of it)) else ("N/A")), (if (size of item 3 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 3 of it)) else ("N/A")), (if (size of item 4 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 4 of it)) else ("N/A")), (if (size of item 5 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 5 of it)) else ("N/A")), (if (size of item 6 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 6 of it)) else ("N/A")), (if (size of item 7 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 7 of it)) else ("N/A")), (if (size of item 8 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 8 of it)) else ("N/A")), (if (size of item 9 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 9 of it)) else ("N/A")), (if (size of item 10 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 10 of it)) else ("N/A")), (if (size of item 11 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 11 of it)) else ("N/A")), (if (size of item 12 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 12 of it)) else ("N/A")), (if (size of item 13 of it >= 1) then ((if it = "" then "" else it) of concatenation ";" of values of results (item 0 of it, elements of item 13 of it)) else ("N/A"))) of (elements of item 0 of it,item 1 of it,item 2 of it,item 3 of it,item 4 of it,item 5 of it,item 6 of it,item 7 of it,item 8 of it,item 9 of it,item 10 of it,item 11 of it,item 12 of it,item 13 of it) of (set of bes computers whose (agent type of it = "Native"), set of bes properties whose (name of it as lowercase = ("Custom1") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom2") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom3") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom4") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom5") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom6") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom7") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom8") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom9") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom10") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom11") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom12") as lowercase), set of bes properties whose (name of it as lowercase = ("Custom13") as lowercase))

That said, I don’t have specific Explorer as I still find WR-based API to be better suited (you can have multiple WR servers under the hood and control which the API use but it adds redundancy with Explorer, no redundancy). Last time I checked even the API logging wasn’t there which is strange (I get why it is not there for the root server but if Explorer is stand-alone surely it can be logging whatever requests it gets).

[fermt]

I am not even trying something that complicated, I am simply getting the current date/time:

now

It seems like a lot of time is used when re-directing the requests to the BigFix Explorer Service and return the results back. The query itself runs in ms.

[ageorgiev]

That’s strange then for sure. I would open a support case and see if the behaviour can be tracked and if identified as an issue/bug to possibly get it raised as a KB.