Need to find and compare values on endpoint

Created this analysis.

if exists (match (regex "NO") of ((parenthesized part 2 of it) of (matches (regex "^([^,]+),(.*)$") of (lines whose (it as string contains "Clock Synchronization . NTP")of file "c:\_RHEL_GSD.txt" as string)))) then "Non Compliant" else "Compliant"

Any service not necessary for the functioning of server should not be running >> Defaults
** Clock Synchronization . NTP >> NO**
** /etc/ssh/ssh_config to control remote access >> NO**
** Screensaver-password based Locking of inactive GUI sessions >> NA**
** PASS_MAX_DAYS >> NO**
** PASS_MIN_LEN >> NO**
** Minimum Password Age >> YES**
** Log as much information about the system through syslog. >>**
** General Steps to be followed during installation >> YES**
** Disk Partitioning and Mounting >> NO**
** Disable IPv6 >> NO**
** Add nodev Option to Non-Root Local Partitions >> NO**
** Restrict device ownership to root only >> NO**
** Disable USB Devices (AKA Hotplugger) >> NO**
** Disable GNOME Automounting >> YES**

But not getting proper results.
Kindly suggest.

You haven’t given me enough info to help you.

Can you explain in detail what the goal you are trying to achieve is?
What does this relevance do?

Can you provide an example of what the proper results should be?
Can you provide what the output you are getting is? (not clear which you provided)
Can you provide some example contents of c:\_RHEL_GSD.txt to know what is the starting point?

Hi James,

Thanks for your response.

I am executing some commands on RHEL machines and getting output generated in file.
That file has multiple checks returned as a result of commands executed on machine.

Now i want to compare those output values and if found ‘NO’ then it is Non Compliant nad if ‘YES’ then Complaint else ‘N/A’

Analysis which i have created is only returning everything as Compliant wherein on server i can see multiple deviations.

Analysis created :-

if exists (match (regex "NO") of ((parenthesized part 2 of it) of (matches (regex "^([^,]+),(.*)$") of (lines whose (it as string contains "Clock Synchronization . NTP")of file "c:\_RHEL_GSD.txt" as string)))) then "Non Compliant" else "Compliant"

So need your help to get this fixed.

SAMPLE OUTPUT :-

IM Version 1.0

HOSTNAME, xxxxxxx

IP, x.x.x.x

DATE, Thu Apr 12 12:34:16 IST 2018

Any service not necessary for the functioning of server should not be running >> Defaults
   Clock Synchronization . NTP      >>       NO
   /etc/ssh/ssh_config to control remote access      >>      NO
   Screensaver-password  based Locking of inactive GUI sessions      >>      NA
   PASS_MAX_DAYS      >>      NO
   PASS_MIN_LEN      >>      NO
   Minimum Password Age      >>      YES
   Log as much information about the system through syslog.      >>
   General Steps to be followed during installation       >>      YES
   Disk Partitioning and Mounting       >>      NO
   Disable IPv6       >>      NO
   Add nodev Option to Non-Root Local Partitions       >>      NO
   Restrict device ownership to root only       >>      NO
   Disable USB Devices (AKA Hotplugger)       >>      NO
   Disable GNOME Automounting       >>      YES
   Verify Permissions on passwd shadow group and gshadow Files       >>      YES
   Add .nodev. Option To Appropriate Partitions In /etc/fstab       >>      NO
   Add .nosuid. and .nodev. Option For Removable Media In /etc/fstab      >>      NO
   Restrict Programs from Dangerous Execution Patterns      >>      NO
   Disable User-Mounted Removable File Systems      >>      NO
   Disabling Group and Outside File Access Permissions      >>      NO
   Lockout Accounts After 3 Failures     >>      NO
   Restrict Root Logins to System Console     >>      NO
   Limit Access to the Root Account from su     >>      NO
   Webmin should be disabled     >>      YES
   Remove .rhosts Support In PAM Configuration Files     >>      YES
   /etc/passwd     >>      YES
   /etc/shadow     >>      YES
   /etc/shadow     >>      YES
   Checking the User Passwords with sha512 encryptions     >>      YES

----- END OF FILE -----

Sanpshot from QnA as reference.

Value seems Non-Compliant but still my analysis returns this as Compliant :frowning:

…you understand that the only check your analysis is examining is the NTP service line, right? You’d need to look at every check in your analysis.

Every line data is required to be checked and taken out in output file as Compliant or Non Compliant

Right, but I see at least 29 checks there.

Are you creating 29 different properties that would each have a compliant / non-complaint result, or one property that would have a value of non-compliant if any of the checks fail?

The way you are looking at the NTP line, your compliance/non-compliance is only reflecting the NTP check. And because you are looking for ‘existence of “NO”’, you’ll get a result of Compliant for any type of error condition - like the file itself doesn’t exist, or the line is not found in the file, or your regex is incorrect, “Compliant” is your default result.

If you want a single result, it may be as simple as
if exists lines whose (it as string ends with ">> NO**") of file "C:\whatever.txt" then "Non-Compliant" else "Compliant"

1 Like

Wouldn’t it be easier to look for running applications (https://developer.bigfix.com/relevance/reference/application.html#running-application-string-application) not contained by set of (“service a”;“service b”) etc?

That way, you have the definitive list of services you WANT to be running, and can look for anything unexpected.

You could even do running applications whose (it is not contained by set of (“app a”;“app b”)) and get the full list of applications that are running that don’t match your criteria.

This way, you don’t have to run an action to pull data into a file, then inspect that file for evidence, and can do everything directly in relevance.

1 Like

Your regex is looking for

(<anything not a comma>)<comma>(<anything>)

but that isn’t what is in the line you are attempting to test (but it does match the format of the HOSTNAME, IP and DATE lines)

This is the right idea, but not correct for the file format, which was hard to tell in the forum post.

I agree with this idea. This might not be the right inspector for RHEL, but you could query this directly from a RHEL system in a few different ways. You can just return the list of services / processes running so that you can use that info for debug and analysis elsewhere, but you can also just directly look to see if things are missing on the system with relevance.

I’m being a bit pedantic, but just for the sake of vocabulary so we can be on the same page. Analyses have Properties, Properties have Relevance. if exists (match (regex "NO") of is Relevance, which in this case is being used in a Property and the Property used Within an Analysis.

This part of the relevance is causing it to only check a single line of the file, not the whole file.

In general, I would recommend making many properties, not just what you want as the final result. Make properties which give you all the raw data, then create a property that filters that raw data to just the results you want (lines containing “NO” for instance) then write another property that turns that into a real result like “compliant” or “not compliant” so that it is easy to audit. The issue is if you try to build up all the logic at once in relevance, then it is impossible to tell what went wrong and where.

It sounds like what you ultimately want in the end is that if the file contains “>>” followed by “NO” anywhere in the file, then it should be considered non-compliant.

In that case, I would start with the results of:

lines containing ">>" of files "c:\_RHEL_GSD.txt"

Then look at:

(it as trimmed string) of following texts of firsts ">>" of lines containing ">>" of files "c:\_RHEL_GSD.txt"

Then see:

(multiplicity of it, it) of unique values of (it as trimmed string) of following texts of firsts ">>" of lines containing ">>" of files "c:\_RHEL_GSD.txt"

Then:

not exists unique values whose(it contains "NO") of (it as trimmed string) of following texts of firsts ">>" of lines containing ">>" of files "c:\_RHEL_GSD.txt"

This should give you TRUE if compliant, and FALSE if not compliant. (you could reverse this just by removing the NOT in front)

Once this value returns what you want, then you can adapt it into returning words like “Compliant” or “Not-Compliant” instead of True/False, but you really need to build up to that and not start there.

::: ¡Very Important! :::

You should make sure your custom analysis properties are NOT set to “Every Report” and should instead be much less often. I would make most of them once every 6 hours or less often, and just one or two that you care about set to something like once an hour, but honestly, there isn’t a need to have the client run through the relevance that often in a day. When you are making changes to the analysis, all the clients should update all of the properties right away, it just when you are done, you don’t want clients aggressively evaluating properties that rarely ever change, as it will slow down the evaluation of all other properties and everything else the client does if you have too many over time evaluating too often.

1 Like

One quick question.

I was able to sort Compliant / Non Compliant values but if i have some output other than Yes / No then how to adjust them in this ?

q: if (not exists unique values whose(it contains “NO”) of (it as trimmed string) of following texts of firsts “>>” of lines containing “Any service not necessary for the functioning of server should not be running” of file “c:_RHEL_GSD.txt”) then “Compliant” else "Non-Compliant"
A: Compliant
T: 7.060 ms

q: if (not exists unique values whose(it contains “NO”) of (it as trimmed string) of following texts of firsts “>>” of lines containing “Clock Synchronization . NTP” of file “c:_RHEL_GSD.txt”) then “Compliant” else "Non-Compliant"
A: Non-Compliant
T: 7.169 ms

q: if (not exists unique values whose(it contains “NO”) of (it as trimmed string) of following texts of firsts “>>” of lines containing “/etc/ssh/ssh_config to control remote access” of file “c:_RHEL_GSD.txt”) then “Compliant” else "Non-Compliant"
A: Non-Compliant
T: 7.062 ms

q: if (not exists unique values whose(it contains “NO”) of (it as trimmed string) of following texts of firsts “>>” of lines containing “Screensaver-password based Locking of inactive GUI sessions” of file “c:_RHEL_GSD.txt”) then “Compliant” else "Non-Compliant"
A: Compliant
T: 7.804 ms

q: if (not exists unique values whose(it contains “NO”) of (it as trimmed string) of following texts of firsts “>>” of lines containing “PASS_MAX_DAYS” of file “c:_RHEL_GSD.txt”) then “Compliant” else "Non-Compliant"
A: Non-Compliant
T: 7.291 ms

Any service not necessary for the functioning of server should not be running >> Defaults
Clock Synchronization . NTP >> NO
/etc/ssh/ssh_config to control remote access >> NO
Screensaver-password based Locking of inactive GUI sessions >> NA
PASS_MAX_DAYS >> NO
PASS_MIN_LEN >> NO
Minimum Password Age >> YES
Log as much information about the system through syslog. >>

Yes, i need separate checks created for output generated in that file.

Well, it looks like your latest example has different content for the file (no asterisks in there).

If you really want a separate Analysis Property for each check, you can do something like the following…and create a separate property for each individual check:

q: (if it contains "NA" then "NA" else if it contains "YES" then "Compliant" else if it contains "NO" then "Not Compliant" else "Unknown Value") of following texts of firsts ">>" of lines whose (preceding text of first ">>" of it starts with "Screensaver-password based Locking") of file "d:\temp\test.txt"
A: NA

But as a suggestion, I think you might consider making one property for Non-Compliant checks, one property for Compliant checks, and one property for Non-Applicable checks. Then the property results would show which lines match each criteria. Using your second example, this would be something like

Non-Compliant Checks

q: preceding texts of firsts ">>" of lines whose (following text of firsts ">>" of it contains "NO") of files "d:\temp\test.txt"
A: Clock Synchronization . NTP 
A: /etc/ssh/ssh_config to control remote access 
A: PASS_MAX_DAYS 
A: PASS_MIN_LEN
T: 0.683 ms

Compliant Checks

q: preceding texts of firsts ">>" of lines whose (following text of firsts ">>" of it contains "YES") of files "d:\temp\test.txt"
A: Minimum Password Age 
T: 0.631 ms

Non-Applicable Checks

q: preceding texts of firsts ">>" of lines whose (following text of firsts ">>" of it contains "NA") of files "d:\temp\test.txt"
A: Screensaver-password based Locking of inactive GUI sessions 
T: 0.639 ms

I definitely agree with jgstew’s comment about making properties that gather source data so you know what you have to work with; I would also add to this that IBM’s WebUI now has the Query function, which can be extremely useful in testing relevance across a subset of computers.

I frequently find that what I thought was a bulletproof relevance statement, even after having tested it on my 10+ test machines, invariably run into some issues when run against several thousand, not to mention tens of thousand, production endpoints. There’s always some anomaly or unexpected issue, and WebUI / Query can help reduce the rework needed to get a property right.

1 Like