Rest API To Initiate Server Automation Plan

We are beginning to play with Server Automation in BigFix for patching systems. This use case applies to RedHat Linux systems sitting on VMWare, with VMWare Management Extender creating a Snapshot prior to patching the system.

Overall, I am happy with performance for single-system patching processes but would like help for determining how to make a REST Api call to invoke an Automation plan against a particular endpoint. The conditions are:

10 Step Automation plan
All steps associated endpoints are based on Two Steps:
101 - ESXi Hypervisor Virtual Machine endpoint
104 - Server
These two endpoints have the same exact names (generally) but simply one represents the OS and one the container.

Moving forward I know our shop will more easily use this product without interaction with the GUI and through streamlined processes, and my research tells me that REST Api holds a lot of gold in that arena. So given the limited scope of the problem, is this something that could be accomplished fairly simply?

Thanks!

1 Like

Hey mxalvis - you could check out the details of the SA REST API here:

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/Server%20Automation%20REST%20API

Good day Paul! We spoke at Dillard’s about Server Automation and I’m glad to see you’re here to help.

So from what I’ve gathered and tried, I’ve deployed Rest API (Task ID 108) to a Windows system but I am not sure how this plays in to the process. This machine we’ll call WINDOWS1. The BESServer we’ll call BES1

From my experience with playing with the API and what I parsed in the provided link, I tried to GET the Automation Plan via the following URL

https://BES1.dillards.com:52311/api/serverautomation/plan/custom/rhel/11629
unknown request method

So, my question now would be how does the Windows machine play into the picture for trying to use the API to evoke this plan against a machine and what do I need to fix in my REST call?

From the link referencing the Plan calls
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/Plan
it appears the Post method is along the lines of what I want to achieve, but I am not quite sure from where to initiate and how to best populate the correct information.

1 Like

You should not include machine names and BESServer names or URLs. I would redact those from your post.

Oh hey Matt! :smile:

So did you have a chance to check out the demo application that’s shipped with the REST API deployment?:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/The%20Demo%20Application

This should give you an idea of how to interact with the server, but in short, it’s a 2 step process:

  1. Make a GET request to the server to indicate which existing plan you want to execute.
    > The server will respond to you with an XML document that is an execution skeleton of the plan

  2. On this skeleton, you fill in the desired parameter values and targets for each step and then POST it back.
    > When the server gets this ‘filled in’ skeleton, it has everything it needs to execute the plan (and does so).

Ordinarily, you could use a tool like curl to interact with a REST API, but because there’s a little bit of processing to be done between parts (1) and (2) above, you could use something like node and make AJAX calls to interact with the API. Of course, you could use whatever client side technology you like, which I suppose is part of the appeal of interacting via REST.

One alternative you might consider if for example you don’t have a Windows machine to deploy the REST API, is extracting the XML from a previously run instance of the plan (i.e. a plan action - you can view these in the console at either All Content > Actions or for a smaller number of actions to look through: All Content > Sites > Master Action Site > Actions).

Once you know the ID of the plan action, you could use the platform REST API (see here) to retrieve it. This contains the full definition of the plan action that is processed by our plan engine, so you could also consider just updating this and using the platform REST API (see here) to create (or more correctly, import) a new action that will get processed by the plan engine (i.e. executed).

Now this method might be a little less pleasant to work with because the document you get returned here is essentially an XML document (plan XML) within another XML document (a standard BES action definition), but it does remove the requirement for the SA REST API deployment, so just something to consider. In either case though, there will be some XML manipulation to be done (to simulate the selection of targets and parameter values like you would do in the UI normally)

… probably one other thing I should have said here about the steps (1) and (2) in my post above.

It’s not mandatory to do both (1) and (2) every time you want to execute a plan.

If you do part (1) just once to get the execution skeleton back (and your underlying plan definition doesn’t change in the meantime), all you really need to do is update that and then do part (2).

You could do part (2) multiple times with different parameter and target values if you wish.

That looks like a great method. My only concern is for the Non-Windows option, it must be populated from a previously run Action and you are not able to pull from a Plan that has been created and, say, never run before? OR is it that if I want that functionality I must run an “application” method as described with sample application installed on the Windows instance?

Only concern at this time is the overhead of R&D associated to a full application (like the Java example provided) vs the command line oriented method you’ve described which my fellow *NIX admins would be far more comfortable manipulating. If it had to be run against an Action I’m sure we could make it work, but I am only trying to be forward thinking in streamlining the processes for those less savvy with BigFix itself in the future

That’s correct - if you wanted to run a plan that had never been run before, you would need to go down the SA REST API route (as opposed to the alternative platform REST API route I describe).

And I fully get that you might not want to provide a full blown web front end for using the REST API - totally understandable - we really just put that demo application in there so folks could see how it could be used (and even use something like firebug to see the HTTP traffic back and forth between the application and the REST API for reference).

But I don’t think you’ll be able to get away from the necessity to provide parameter and target values, and that would go not just for SA plan actions, but for any action you’re trying to create. Every action can potentially be different, and for an SA plan, that is just taken a step further due to the fact that every step in the plan can be targeted at different machines potentially.

The easiest scenario to programmatically support is the execution of a plan that was run before, exactly like it was run before (by that I mean it targets the same computers and uses the same parameter values). In this case, you should just be able to create a “copy” of the existing plan action using the platform REST API’s functions I linked to above. This would be useful let’s say for repetitive tasks that are done on the same machine(s).

But in pretty much any other case where you’re deviating from that, you’ll need to provide the action (and again, this goes for any action, not just an SA plan action) with the variables it needs to know what it’s doing (parameters, if it takes any) and where it’s doing it (one or more targets). By going the REST route the client needs to perform the job of the UI in terms of soliciting that input.

Paul,

I have performed REST Pull of an Server Automation action I want to duplicate. I have figured out how to pull the data via Perl and repopulate the Server names with the new appropriate server names. At that point, if I understand correctly (and there are not “execution time” settings I need modify) I believe it is ready to perform the “RESTAPI Import” but I fail to comprehend the syntax provided in the link. An example of performing this POST method would be much obliged.

I notice the reference to “BES.XSD” but fail to understand where this/these files should be located / referenced in the POST method to accomplish “Do this action now against these endpoints.” All in all, I have the XML file and do not quite get how to perform the import against it.

Thanks again for all your help!

Sounds like some good progress Matt.

The XSD files I believe you can find under /opt/BESServer/Reference, but you should still be able to make the call without actually having your hands on them. They are there for your own reference though if you want to look at them.

So when you exported it, you probably would end up with something like this (in this example, I’m running a 3 step plan against a fictitious MYBESSERVER host. In each step, I’m targeting the host only by name, but you could equally be targeting by name and ID if you wanted to):

<?xml version="1.0" encoding="UTF-8"?>
<BES xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BES.xsd">
    <SingleAction>
        <Title>Simple Plan</Title>
        <Relevance>false</Relevance>
        <ActionScript MIMEType="application/x-sh">#!/plan/xml
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;plan xmlns="http://iemfsa.tivoli.ibm.com/AutomationPlanAction" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0" source-plan-id="2494" prefetch="false" pending-restart="PausePlan" source-plan-site-url="http://MYBESSERVER:52311/cgi-bin/bfgather.exe/actionsite"&gt; 
&lt;plan-steps&gt;

&lt;step id="101"&gt;
  &lt;fixlet fixlet-id="34" site-url="http://MYBESSERVER:52311/cgi-bin/bfgather.exe/actionsite" name="Simple%20Task" action-name="Action1"  modification-time="Tue, 05 Aug 2014 16:46:22 +0000" &gt;
    &lt;relevance&gt;&lt;![CDATA[true]]&gt;&lt;/relevance&gt;
    &lt;action-script mime-type="application/x-Fixlet-Windows-Shell"&gt;&lt;![CDATA[// Enter your action script here
parameter "x" = "y"]]&gt;&lt;/action-script&gt;
&lt;success  type="RunToCompletion"/&gt;
  &lt;/fixlet&gt;
  &lt;targets&gt;MYBESSERVER
&lt;/targets&gt;
&lt;/step&gt;

&lt;step id="102"&gt;
  &lt;fixlet fixlet-id="34" site-url="http://MYBESSERVER:52311/cgi-bin/bfgather.exe/actionsite" name="Simple%20Task" action-name="Action1"  modification-time="Tue, 05 Aug 2014 16:46:22 +0000" &gt;
    &lt;relevance&gt;&lt;![CDATA[true]]&gt;&lt;/relevance&gt;
    &lt;action-script mime-type="application/x-Fixlet-Windows-Shell"&gt;&lt;![CDATA[// Enter your action script here
parameter "x" = "y"]]&gt;&lt;/action-script&gt;
&lt;success  type="RunToCompletion"/&gt;
  &lt;/fixlet&gt;
  &lt;targets&gt;MYBESSERVER
&lt;/targets&gt;
&lt;/step&gt;

&lt;step id="103"&gt;
  &lt;fixlet fixlet-id="34" site-url="http://MYBESSERVER:52311/cgi-bin/bfgather.exe/actionsite" name="Simple%20Task" action-name="Action1"  modification-time="Tue, 05 Aug 2014 16:46:22 +0000" &gt;
    &lt;relevance&gt;&lt;![CDATA[true]]&gt;&lt;/relevance&gt;
    &lt;action-script mime-type="application/x-Fixlet-Windows-Shell"&gt;&lt;![CDATA[// Enter your action script here
parameter "x" = "y"]]&gt;&lt;/action-script&gt;
&lt;success  type="RunToCompletion"/&gt;
  &lt;/fixlet&gt;
  &lt;targets&gt;MYBESSERVER
&lt;/targets&gt;
&lt;/step&gt;

&lt;/plan-steps&gt;
&lt;execution-order&gt;
   &lt;step id="101" depends=""&gt;
      &lt;on-failure action="StopPlan" targets="IncludeFailed" threshold="0"/&gt;
   &lt;/step&gt;
   &lt;step id="102" depends="101"&gt;
      &lt;on-failure action="StopPlan" targets="IncludeFailed" threshold="0"/&gt;
   &lt;/step&gt;
   &lt;step id="103" depends="102"&gt;
      &lt;on-failure action="StopPlan" targets="IncludeFailed" threshold="0"/&gt;
   &lt;/step&gt;
&lt;/execution-order&gt;
&lt;plan-settings&gt;
    &lt;boolean-setting name="notification-include-summary"&gt;&lt;![CDATA[true]]&gt;&lt;/boolean-setting&gt;
    &lt;boolean-setting name="notification-include-recipient"&gt;&lt;![CDATA[true]]&gt;&lt;/boolean-setting&gt;
&lt;/plan-settings&gt;

&lt;/plan&gt;</ActionScript>
        <SuccessCriteria Option="RunToCompletion"></SuccessCriteria>
        <Settings>
            ...
        </Settings>
        <SettingsLocks>
            ...
        </SettingsLocks>
        <IsUrgent>false</IsUrgent>
    </SingleAction>
</BES>

The contents of the <ActionScript> element is what we’re interested in - this is the plan action XML content. As I mentioned, you’re dealing with an XML document inside another XML document, so that’s why you see all the “&lt;” and “&gt;” all over the place.

But, it sounds like you’ve already got to grips with this part and you now just want to get it back into the system.

Actually, you could probably use the actions REST resource to create the action, that might even be better.

Try a HTTP POST to the URL below, specifying the ContentType header as application/xml and writing your updated XML document to the body of the request (that means the whole document, not just the part inside <ActionScript>. It should be in the same format as the example I’ve given above, which is basically the same format as you originally got it back from REST to begin with) :

https://<bes-server>:52311/api/actions

I’m definitely not a perl guy, but I’ve had a look around and I think something like this might do it for you (though not actually tested):

use LWP::UserAgent;

my $ua = LWP::UserAgent->new;

my $server_endpoint = "https://<bes-server>:52311/api/actions";

# set custom HTTP request header fields
my $req = HTTP::Request->new(POST => $server_endpoint);
$req->header('content-type' => 'application/xml');

# add POST data to HTTP request body
my $post_data = "<updated-xml-content>";
$req->content($post_data);

my $resp = $ua->request($req);
if ($resp->is_success) {
    my $message = $resp->decoded_content;
    print "Received reply: $message\n";
}
else {
    print "HTTP POST error code: ", $resp->code, "\n";
    print "HTTP POST error message: ", $resp->message, "\n";
}

Please let me know how you get on with this.

Paul, I’ve come back to this with a bit more insight, and have pursued the ‘SA REST API’ method due to the requirement of having manually ran a SA Plan to use traditional BigFix REST API and snatch an action. I’ve figured out how to grab the ‘skeleton’ but just want some clarification in “filling in the blanks.” I am using bash and curl as I myself am not a perl guy either.

Example: Get my RHEL Patching Server Automation Plan “Skeleton”

curl -fk -u ${user}:${pw} -X GET 'https://${server_fqdn}:9081/serverautomation/plan/custom/RHEL/11629'

For each step, I see the following block

  <step sequence="101" name="Power%20Off%20VM%20(soft)">
      <!-- <target-set>
        <group name="myGroup" id="123" />
        <computer name="nameOnlyTarget" />
        <computer name="nameAndIDTarget" id="123" />
      </target-set> -->
  </step>

I have tested manually updating the skeleton with the necessary machine computer Ids and hostnames, with “Server” objects/ids being placed in the correct tasks and “ESXi Hypervisor Virtual Machine” objects/ids the same. I posted this back via SA REST API, and it works like a champ!

However, for a purely programmatical approach, what would be the best way to be able to determine which steps are for “servers” and which are for “vms,” preferably without having to leverage the step name or without the bash script having to have the understanding of which tasks are geared for Servers or VMs. (IE, I know 101 is VM, 102 is Server, etc…)

Hi Matt,

Just from the skeleton itself, no, not really - as far as the REST API is concerned, a step is a step is a step.

However, I’ve been playing around with some relevance that might be able to identify the plan step IDs of those steps that are targeted at the proxy agent representations of the VMs. Try this out and see if it gives you back the correct step IDs for your plan:

===================

(
   preceding texts of first "%22%3E" of following texts of last "%3Cstep id=%22" of preceding texts of it
 )
 of substrings "%3Cmime-field name=%22vmware virtualization management command%22%3E%3C![CDATA[true]]%3E%3C/mime-field%3E" of script of actions of bes fixlet 
 whose
 (
   name of site of it = "ActionSite" 
  and
   id of it = 2644
 )

===================

(where 2644 is the ID of your plan definition fixlet)

This expression is based on the fact that (from what I can see at least) the virtual endpoint targeted actions contain a MIME field called “vmware virtualization management command” and is set to a value of “true”.

(for the purposes of being able to get the expression formatted in this post, I had to replace angle brackets in the strings above with “%3C” (<) and “%3E” (>). Otherwise it was showing up strangely.)

Please let me know if this is of any use to you Matt.

Cheers,
Paul.

Paul, this appears to be very close to what we need. However, it misses my “custom” snapshot task and also misses task 805 : “Warning: Outdated VMware Tools Version Detected.” (a non-custom task.)

Maybe I can leverage a specific Relevance clause that is “universal” across VMware commands, possibly having to custom copy the out-of-the-box commands I am using. “Power On VM”, “Power Off VM”

This is good stuff! I’ll play more with this aspect of things.

Hey Matt,

I feel like there should be a more elegant way to combine these together into a single expression, but this should do what you need:

elements of 
 (
   set of 
   (
     (
       preceding texts of first "%22%3E" of following texts of last "%3Cstep id=%22" of preceding texts of it
     )
     of substrings "%3Cmime-field name=%22vmware virtualization management command%22%3E%3C![CDATA[true]]%3E%3C/mime-field%3E" of it
   )
   + set of 
   (
     (
       preceding texts of first "%22%3E" of following texts of last "%3Cstep id=%22" of preceding texts of it
     )
     of substrings "%3Cmime-field name=%22x-fixlet-domain_attributes%22%3E%3C![CDATA[SLM Virtual Endpoint Management Setup]]%3E%3C/mime-field%3E" of it
   )
 )
 of script of action of bes fixlet 
 whose
 (
   name of site of it = "ActionSite" 
  and
   id of it = 2644
 )

The only ones it won’t cover are your custom ones… for those, I would say do an export of an existing fixlet on the Virtual Endpoint Manager site (“Power On VM” or “Power Off VM” would be good examples) and in there you’ll see something like this:

<MIMEField>
    <Name>vmware virtualization management command</Name>
    <Value>true</Value>
</MIMEField>

(actually, I’ve just gone and listed it here, so you can just copy the text above! :smile:)

What you can do then though is export the custom fixlet you’ve already created, open the resulting *.bes file in a text editor, and paste in the MIME field shown above - just add it directly above or below the other MIME field that should already be in your custom fixlet (the actual time will be different though):

<MIMEField>
     <Name>x-fixlet-modification-time</Name>
    <Value>Wed, 22 Aug 2012 23:57:10 +0000</Value>
</MIMEField>

then save it and re-import the resulting file (and add it to your plan).

You should only have to do this once if you create your other (future) custom VM fixlets by creating a copy of this one and editing the copy to create your new custom content.

Anyway, see how this goes and let me know how you get on with it.

-Paul.

Paul, Session relevance tester tells me that query would be great for this. I am now having issues running the query via REST API where expression cannot be parsed. I believe it would be due to the quotes and how they get rendered over HTTP. I am trying to determine how to escape them if that is indeed the issue.

curl -k -u alvism --data-urlencode 'relevance=elements of(set of((preceding texts of first "%22%3E" of following texts of last "%3Cstep id=%22" of preceding texts of it)of substrings "%3Cmime-field name=%22vmware virtualization management command%22%3E%3C![CDATA[true]]%3E%3C/mime-field%3E" of it)+ set of((preceding texts of first "%22%3E" of following texts of last "%3Cstep id=%22" of preceding texts of it     )     of substrings "%3Cmime-field name=%22x-fixlet-domain_attributes%22%3E%3C![CDATA[SLM Virtual Endpoint Management Setup]]%3E%3C/mime-field%3E" of it))of script of action of bes fixlet whose (id of it = 11629)' 'https://${besserver}:52311/api/query'
Enter host password for user 'alvism':

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Query Resource="elements of(set of((preceding texts of first &quot;&quot;&gt;&quot; of following texts of last &quot;&lt;step id=&quot;&quot; of preceding texts of it)of substrings &quot;&lt;mime-field name=&quot;vmware virtualization management command&quot;&gt;&lt;![CDATA[true]]&gt;&lt;/mime-field&gt;&quot; of it)+ set of((preceding texts of first &quot;&quot;&gt;&quot; of following texts of last &quot;&lt;step id=&quot;&quot; of preceding texts of it     )     of substrings &quot;&lt;mime-field name=&quot;x-fixlet-domain_attributes&quot;&gt;&lt;![CDATA[SLM Virtual Endpoint Management Setup]]&gt;&lt;/mime-field&gt;&quot; of it))of script of action of bes fixlet whose (id of it = 11629)">
                <Result></Result>
                <Error>This expression could not be parsed.</Error>
        </Query>
</BESAPI>
1 Like

Hey Matt - sorry for the delay in getting back to you here… just back today after the holidays.

Try this:

$ curl -k -u alvism --data-urlencode relevance=‘elements of(set of((preceding texts of first “%2522>” of following texts of last “<step id=%2522” of preceding texts of it)of substrings “<mime-field name=%2522vmware virtualization management command%2522>” of it)+ set of((preceding texts of first “%2522>” of following texts of last “<step id=%2522” of preceding texts of it ) of substrings “” of it))of script of action of bes fixlet whose (id of it = 11629)’ ‘https://${besserver}:52311/api/query’

It’s actually the % character in the %XX that needed escaping it looks like.
Here’s what I got running it for my own setup:

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BESAPI.xsd">
	<Query Resource="elements of(set of((preceding texts of first &quot;%22&gt;&quot; of following texts of last &quot;&lt;step id=%22&quot; of preceding texts of it)of substrings &quot;&lt;mime-field name=%22vmware virtualization management command%22&gt;&lt;![CDATA[true]]&gt;&lt;/mime-field&gt;&quot; of it)+ set of((preceding texts of first &quot;%22&gt;&quot; of following texts of last &quot;&lt;step id=%22&quot; of preceding texts of it     )     of substrings &quot;&lt;mime-field name=%22x-fixlet-domain_attributes%22&gt;&lt;![CDATA[SLM Virtual Endpoint Management Setup]]&gt;&lt;/mime-field&gt;&quot; of it))of script of action of bes fixlet whose (id of it = 2644)">
		<Result>
			<Answer type="string">101</Answer>
			<Answer type="string">104</Answer>
		</Result>
		<Evaluation>
			<Time>103.571ms</Time>
			<Plurality>Plural</Plurality>
		</Evaluation>
	</Query>
</BESAPI>

hi paul
can u give xml validation fo these two steps

@madhumk you can find the XSD schema here:
https://developer.bigfix.com/rest-api/schema_files.html