API Server Side Verbose Logging

Customizations
1

I have not been able to find a way to increase the verbosity of the server side API logging. There are many questions about this but no clear answers, is there a way to do this?

As an example, I burned several hours trying to figure out why my FixletWithActions POST to the fixlet endpoint was failing with an http 400. The documentation is pretty much non-existent. When creating a fixlet with an action via the API this seems to be the class to use. I create an instance of the class along with and action and action script, everything serializes correctly (except the mimetype on the script, but even that aside) I submit to the fixlet endpoint and there is no clear indication of why it fails except http 400.

After some time I try to submit the same xml replacing FixletWithActions with Fixlet and it worksā€¦ this is clearly and issue with the XSD, it should serialize as a Fixlet. Again even this aside, I would expect to have some logging on the server side to view the error saying class not supported or something to that effect.

The BESRelay.log and server_audit.log show the connections, but nothing indicating error.

So, as many others have asked before, is there server side API debug logging available?

Thank you,
Patrick

2 Likes
2

Iā€™m not sure if the connection logs will catch the api requests. We are also in the research on how to track our API Requests as we are seeing at some times the BigFix REST API stops returning answers.

_BESRelay_HTTPServer_HttpLogDirectoryPath

    Default: none
    If enabled, writes out a connection log to this directory. Connection logs can get quite big (gigabytes per day), and may cause hard disk performance problems. The log directory needs to exist and will not be created automatically.
    This setting requires that the Root Server or Relay service be restarted.
    Example: c:\logs\httpoutfolder
1 Like
3

Thank you, I will let you know how it goes.

4

Unfortunately these logs do not contain any useful information to help. It would be great if BigFix could expose something that would allow HTTP 400 errors to have more context.

2 Likes
5

Sounds like a good candidate for a BigFix Idea (formally RFE).

1 Like
6

Iā€™m actually not sure what this is. Do you mean SourcedFixletAction?

Can you provide example generated XML that you would be submitting through the REST API? (redact anything sensitive)

Can you provide the URL you are sending the REST request to, but with the FQDN obscured? (everything after the first / ? Example: https://-FQDN-:52311/api/actions

I assume you are doing a POST, not a GET.

Are you trying to create an action, or a fixlet?

My general recommendation is to create something similar to your desired outcome in the BigFix Windows Console, then export it as XML from there and use that as a guide to how the XML should look when doing it from the REST API. I often do exactly this, then turn that into a template. In my case I use the Mustache templating language. See an example here which I exported from an example and replaced key parts with template variables: jgstew-recipes/BigFix/FixletDebugger-Win.bes.mustache at main Ā· jgstew/jgstew-recipes Ā· GitHub

I also have a fully generic template for Fixlets or Tasks.

I also have a helper function I created that infers default values for some of the template variables automatically so I donā€™t have to figure them out myself when generating new content.

The other tip I would give, is to automatically generate the XML using automated methods that you desire to submit to the REST API and instead save it to a local file, then import it into the BigFix Windows Console directly. This will give much better feedback on potential issues with the XML itself. Once you have automatically generated XML that imports into the console directly correctly and functions as intended when tested, THEN you can be fairly confident that you have XML that should work with the REST API directly correctly. The main catch to this is that I donā€™t think it is possible to use something like SourcedFixletAction XML through a console import (though I donā€™t actually know for sure) because that is something kind of special in that it isnā€™t a piece of BigFix Content XML itself, but rather an API shortcut to generating an Action in an ā€œeasierā€ way.

I donā€™t know this for certain, but my guess is that the import API would accept anything that you could import directly into the console: https://developer.bigfix.com/rest-api/api/import.html

I donā€™t think there is a log for this purpose. I think the main feedback is the response the API gives, which is not always as helpful as I would hope.

1 Like
7

I just read over the BES.XSD since it had been a while since I had looked at it to figure out what FixletWithActions is and now I understand what the issue is in this specific case.

FixletWithActions is NOT a thing that can be submitted to the BigFix REST API at all, it isnā€™t actually a real thing what so ever. It is just a pointer within the XSD itself to the ā€œTypeā€ and schema for Fixlets and Tasks.

There are a few different things you can send to the BigFix REST API as far as real content XML that is the same as if you would export it from the console. You can see these in the XSD as anything that starts with <xs:element name= at the top level at the very start of the XSD within this key:

  <xs:element name="BES">
    <xs:complexType>
      <xs:choice maxOccurs="unbounded">

So what the heck is FixletWithActions ???

Well, in BigFix, technically, there is almost 0 difference between a Fixlet and a Task as far as the actual XML is concerned internally. There are very slight differences in terms of defaults and default behaviors within a baseline, but as far as how they are represented internally, no difference, which is what this actually is saying here:

  <xs:element name="BES">
    <xs:complexType>
      <xs:choice maxOccurs="unbounded">

        <xs:element name="Fixlet" type="FixletWithActions" />
        <xs:element name="Task" type="FixletWithActions" />

What this means is that both a Fixlet and a Task are defined by the schema as laid out within the named ā€œtypeā€ FixletWithActions within the XSD but FixletWithActions only exists within the XSD itself as a pointer to that complex type. If you want to create a Fixlet OR a Task, then you must generate XML which has all the required elements within the FixletWithActions but the actual XML would be something like:

<BES>
    <Fixlet>
        `the required stuff according to FixletWithActions`
    </Fixlet>
</BES>

or

<BES>
    <Task>
        `the required stuff according to FixletWithActions`
    </Task>
</BES>
If you really want all the details, expand this

So both Fixlet and Task are of type FixletWithActions

which has the following schema:

  <xs:complexType name="FixletWithActions">
    <xs:complexContent>
      <xs:extension base="Fixlet">
        <xs:sequence>
          <xs:element name="DefaultAction" type="FixletAction" minOccurs="0" />
          <xs:element name="Action" type="FixletAction" minOccurs="0" maxOccurs="unbounded" />
        </xs:sequence>
      </xs:extension>
    </xs:complexContent>
  </xs:complexType>

Notice the <xs:extension base="Fixlet"> element?

This means that FixletWithActions has all of the schema of type Fixlet as well.

This is where things get even more confusing. Why is there a Fixlet type that Fixlet itself extends?

The reason is that when BigFix first started everything was a Fixlet and there were ONLY Fixlets. It was the only content within BigFix. As all other content was added to BigFix, that content extended the base type, which was Fixlet. So in a sense, a Fixlet is a very specific instance of the Fixlet type, but a Task and a Baseline and most other BigFix content is also an extension of this Fixlet type. In fact, as Iā€™m typing this, Iā€™m realizing this might cause a circular reference within the schema itself.

So the full reality of the schema of a Fixlet is basically something like:

Fixlet (element)
-> Fixlet type
    DefaultAction element (0 or 1)
    -> FixletAction type
         various elements
         -> ActionScript type
         -> ActionSuccessCriteria type
         -> ActionSettings type
         -> ActionSettingsLocks type
    Action element (0 or more)
    -> FixletAction type
         various elements
         -> ActionScript type
         -> ActionSuccessCriteria type
         -> ActionSettings type
         -> ActionSettingsLocks type
1 Like
8

Sorry for the delayed response I was out of the office, and thank you for the very detailed responses James!

The point of this post was to figure out if there is a good way to figure out these issues on the server side and it seems that there isnā€™t. All of the above points will certainly be helpful to work around this issue.

Exporting the action / fixlet etc. as XML is a really great point, we are new to BigFix and I have not heard of this yet, thank you very much.

As you can imagine I am not generating the xml from templatesā€¦ that is the point of using the XSD to produce classes that I can serialize in and out of. When this is done the class fixlet does not serialize properly as it drops the actionā€¦ unless fixletwithactions is used. I have a work around for this already it was simply used as an example that I would expect to see something like ā€˜unable to parse fixletwithactions as the type is not validā€™.

I am not sure how many people are using the xsds to generate their classes but there are certainly some rough edges, perhaps the usage is extremely low?