Testing out the new YAML inspector

The new YAML inspector in 11.0.5 looked interesting, so I decided to play with it a bit. I was hoping it would be useful like the XPATH inspector for XML document has been. When testing it on several actual yml files, the results weren’t as clean as I’d hoped they would be. I thought I would post an example and see what the hive mind here comes up with. (Full disclosure, I’m not a yml expert, but do bump into them occasionally.)

Here is a simple, but realistic example that I’ve made generic:

app:
  user:
    authentication:
      ldapProviderConfig:
        connectTimeout: 10000
        socketTimeout: 10000
  security:
    tls:
      configuration:
        version: 1.2
        keyStoreType: PKCS12
    trust:
      StorePath: ../security/cacerts
  location:
    country: US
    address: 123 test ave
    city: Big City
    state: NY
    zip: 12345
   

QNA results:

q: (names of it, values of it) of keys of yaml of file "c:\temp\test.yml"

A: app, user:%0a authentication:%0a ldapProviderConfig:%0a connectTimeout: 10000%0a socketTimeout: 10000%0asecurity:%0a tls:%0a configuration:%0a version: 1.2%0a keyStoreType: PKCS12%0a trust:%0a StorePath: ../security/cacerts%0alocation:%0a country: US%0a address: 123 test ave%0a city: Big City%0a state: NY%0a zip: 12345

T: 0.913 ms

I: plural ( string, yaml value )

It would be great if the inspector handled the carriage returns. The embedded returns and varying length of spaces make it more challenging to use.

It would be useful for the inspector to understand the hierarchy similar to how the XPATH inspector does. The current result is all run together.

Maybe someone can help me make this more useful.

Look like the behaviour changes when inspecting the values of the values of keys, so I guess the depth of the yaml is also a factor (I’m also not a yaml expert )

Q: keys of ((values of it) of keys of yaml of file  "C:\temp\myyaml.yml")
A: user : authentication:%0a  ldapProviderConfig:%0a    connectTimeout: 10000%0a    socketTimeout: 10000
A: security : tls:%0a  configuration:%0a    version: 1.2%0a    keyStoreType: PKCS12%0atrust:%0a  StorePath: ../security/cacerts
A: location : country: US%0aaddress: 123 test ave%0acity: Big City%0astate: NY%0azip: 12345
T: 22.303 ms
I: plural yaml key
Q: (names of it, values of it) of keys of values of keys of ((values of it) of keys of yaml of file  "C:\temp\myyaml.yml")
A: authentication, ldapProviderConfig:%0a  connectTimeout: 10000%0a  socketTimeout: 10000A: tls, configuration:%0a  version: 1.2%0a  keyStoreType: PKCS12
A: trust, StorePath: ../security/cacerts
A: country, USA: address, 123 test ave
A: city, Big City
A: state, NY
A: zip, 12345
T: 21.283 ms
I: plural ( string, yaml value )

Yeah, for simple key and value pairs, the inspector works great. In more complex and deep yml files that we encounter regularly, the inspector doesn’t handle as gracefully. My request to HCL is to improve upon a good idea of the YAML inspector to make it a great and practically useful idea.

Hmm. I'd certainly be willing to roll up enhancement requests, but I'm not entirely sure what I should ask for them to do in a case like this. Part of the issue is that in YAML newlines and indentation matters. Checking the documentation at https://developer.bigfix.com/relevance/search/?query=yaml it looks like we base some of the behaviors on what 'jq' or 'yq' utilities would do -- which hopefully makes the inspectors more familiar / usable for people with experience in those utilities, which are commonly used in things like Helm charts or GitHub Actions for parsing yaml and json.

Consider this simplified example of a github action

name: example-action
description: example GitHub Action

inputs:

  tagVersion:
    required: true
    description: This tag is used for creating the release version and the folder name.

runs:
  using: "composite"
  steps:
    - name: Echo Tag Version
      shell: bash -i {0}
      run: |
        echo " TagVersion ${{ inputs.tagVersion }}"

    - name: Stop Flow if release exists
      shell: bash
      run: |
        export GIT_DIR=${{ github.workspace }}/sourcefolder/.git
        git fetch origin --prune --prune-tags
        git fetch --tags
        if git tag -l | grep -q "^{{ inputs.tagVersion }}$"; then
         echo "Release ${{ inputs.tagVersion }} already exists. Stopping workflow."
           exit 1
        fi

Here, an entire bash script is included as a YAML value for one of the subkeys. Within that value, both the newlines and the indentation of each line is important and should be preserved if this is processed any further. In other contexts, any time a 'value' itself contains subkeys, preserving the newlines and indentation allows pipeline the value into another iteration.

At the top level, there are four 'keys'; "name", "description", "inputs", and "runs". "inputs" and "runs" each have subkeys or multiline values:

q: keys of yaml of file "c:\temp\github\action.yaml"

A: name : example-action
A: description : example GitHub Action
A: inputs : tagVersion:%0a  required: true%0a  description: This tag is used for creating the release version and the folder name.
A: runs : using: composite%0asteps:%0a  - name: Echo Tag Version%0a    shell: bash -i {0}%0a    run: "echo \" TagVersion ${{ inputs.tagVersion }}\"\n"%0a  - name: Stop Flow if release exists%0a    shell: bash%0a    run: "export GIT_DIR=${{ github.workspace }}/sourcefolder/.git\ngit fetch origin --prune --prune-tags\ngit fetch --tags\nif git tag -l | grep -q \"^{{ inputs.tagVersion }}$\"; then\n echo \"Release ${{ inputs.tagVersion }} already exists. Stopping workflow.\"\n   exit 1\nfi"

The 'inputs' key exists and contains subvalues. We can retrieve that directly -

q: values of keys "inputs" of yaml of file "c:\temp\github\action.yaml"
A: tagVersion:%0a  required: true%0a  description: This tag is used for creating the release version and the folder name.

Since we represent the 'value' as text including the the embedded newlines and spacing, we can also pipeline into a second 'yaml' inspection.
We can iterate the 'values of keys of values of keys' similar to JSON processing:

q: names of keys of values of keys "inputs" of yaml of file "c:\temp\github\action.yaml"
A: tagVersion

q: values of keys "required" of values of keys "tagVersion" of values of keys "inputs" of yaml of file "c:\temp\github\action.yaml"
A: true

We can also use a 'path' notation, again very similar to jq or yq, including array indexing to find the second 'step' (using 0-indexed arrays, the first step is step 0 and the second is step 1 )

q: paths "$['runs']['steps'][1]" of yaml of file "c:\temp\github\action.yaml"
A: name: Stop Flow if release exists%0ashell: bash%0arun: "export GIT_DIR=${{ github.workspace }}/sourcefolder/.git\ngit fetch origin --prune --prune-tags\ngit fetch --tags\nif git tag -l | grep -q \"^{{ inputs.tagVersion }}$\"; then\n echo \"Release ${{ inputs.tagVersion }} already exists. Stopping workflow.\"\n   exit 1\nfi"

We can also get the direct content of the multi-line bash script, either by retrieving 'value of key of ' , or by changing the 'path' to directly reference the key. If we were using this relevance to create a bash script via 'createfile' or 'appendfile', this is what we'd want in the output - including the embedded newline characters -

q: values of keys "run" of paths "$['runs']['steps'][1]" of yaml of file "c:\temp\github\action.yaml"

A: "export GIT_DIR=${{ github.workspace }}/sourcefolder/.git\ngit fetch origin --prune --prune-tags\ngit fetch --tags\nif git tag -l | grep -q \"^{{ inputs.tagVersion }}$\"; then\n echo \"Release ${{ inputs.tagVersion }} already exists. Stopping workflow.\"\n   exit 1\nfi"

q: paths "$['runs']['steps'][1]['run']" of yaml of file "c:\temp\github\action.yaml"

A: "export GIT_DIR=${{ github.workspace }}/sourcefolder/.git\ngit fetch origin --prune --prune-tags\ngit fetch --tags\nif git tag -l | grep -q \"^{{ inputs.tagVersion }}$\"; then\n echo \"Release ${{ inputs.tagVersion }} already exists. Stopping workflow.\"\n   exit 1\nfi"

and...I just noticed this, but apparently the inspector does recognize the difference between the multiline bash script as a value (see it's embedding \n instead of %0a for newlines in that section) versus additional subkeys/values. That's actually more than I expected.

1 Like