Checking order of lines in a file (Linux)

[bdowell]

I need to check /etc/pam.d/system-auth and check the order of the lines to confirm that a line that contains text1 comes before a line that contains text2

I’m a bit lost on this one, but greatly appreciate any assistance anyone can lend!

[Aram]

Here’s a random example that can perhaps give you some ideas:

Q: lines of file "R:\foo.txt"
A: This line is number 1 - A
A: This line is number 2 - B
A: This line is number 3 - C
A: This line is number 4 - D
T: 0.389 ms
I: plural file line

Q: line number of line whose (it as string contains “C”) of file "R:\foo.txt"
A: 3
T: 0.413 ms
I: singular integer

Q: (line number of line whose (it as string contains “A”) of it < line number of line whose (it as string contains “C”) of it) of file "R:\foo.txt"
A: True
T: 0.628 ms
I: singular boolean

[bdowell]

Thanks very much @Aram I think that is about what I was expecting (to look for line numbers and compare those)

I think I can get it working from there…

[bdowell]

I thought I had it, but… it looks like I fouled something up…

So what I had was this:

(line number of line whose (it as string as lowercase contains “text 1”) < line number of line whose (it as string as lowercase contains “text 2”) of it) of file “foo.txt”

Maybe I fouled up on the placement of the parens, but I was expecting to see true or false for whether or not text 1 came in the file before text 2. (For this test it doesn’t really matter if neither text is found, but if either is found, need to make sure the entries are right)

[Aram]

That is quite close…you are missing an ‘of it’ after the first line reference:

(line number of line whose (it as string as lowercase contains “text 1”) of it < line number of line whose (it as string as lowercase contains “text 2”) of it) of file “foo.txt”

[bdowell]

@Aram thanks again, that should have cleared that up nicely, much appreciated!!!

[bdowell]

Darn it, I thought this was done, but…

I’m getting an error returned…

When I float over the error, I’m seeing the following:

Singular expression refers to non-unique object

It seems like it is trying to give me some results as I’m seeing some spots where the <error> is shown that it shows two lines, one that says “False” right above that error message.

[bdowell]

Revisiting, it seems like what I probably need to do here is look for more specifics on what I’m looking to check as it seems that my current search may be finding entries that were meant as comments in the conf file.

I may be able to clear that up a bit more myself. Will update if I can as I get it figured out.

[bdowell]

Following up on this one again. I think what I’m seeing is that there can be more than one occurence of the values that I’m hunting for, so I get the confusion as to it being multiple entries.

So… just to confirm, can I add in a something like “first line” testing here?

So is something like this possible:

(line number of first line whose (it as string contains “A”) of it < line number of first line whose (it as string contains “C”) of it) of file "R:\foo.txt"

[Aram]

As you’ve identified, the error you’re seeing (Singular expression refers to non-unique object) suggests that there is more than one line that meets the defined criteria). For reference to common relevance error messages, their meaning, and suggested approaches, please see Common Relevance Error Messages.

For the scenario you describe above, can you be sure that the first instance of each line is the one you want to compare against? Are there other criteria we can use to positively identify the desired file lines?

For an example of ‘first instance’ comparison, perhaps something like the following:

(minimum of line numbers of lines whose (it as string contains "A") of it < minimum of line numbers of lines whose (it as string contains "C") of it) of file "foo.txt"

[bdowell]

I’m pretty sure for this one it is just a case of making sure the first parameter is coming before the second parameter the first time it occurs in the file. (As described the SA that is creating the master config that I should be looking for)

[bdowell]

Got a little more help from the System Admin on this one that should get things narrowed down a bit more.

Pesudo code style, I’m looking for a line that contains (starts with, but then again may have spaces that I’m not accounting for) “auth”… Once I see that line, I’m looking to see if value “words_words.1” is in the line. Then I get to look for another line that would start with “auth”, look to see if the line contains “words_words.2”. I need to confirm that words_words.1 was in a lower line number than the line that contains words_words.2

Except, it seems that just looking for the values in the lines was coming back with more tha one line that it could match in the file. (reading through samples of the files provided by the System Admin don’t seem to show multiple occurences of these values, but oh well, I guess BigFix is finding them).

Anyway, from checking the samples, it looks like I could use a starts with to help narrow things down.

So sample would be something like this:

#
# /etc/pam.d/common-auth - auth settings...
#
# this file...

auth  [ additional info ]   words_words.1
#
#
auth  otherword   words_words.2
#
#
auth  anotherword   words_words.3
#
# end of file

So I’m looking to confirm that things fall in the right order once the auth keywords occur.

[bdowell]

So here’s what I was hoping would work, but doesn’t (contains a relevance error, which I somewhat expected)

(line number of line whose (it as string as lowercase starts with "auth" and contains "value_value.2") of it > line number of line whose (it as string as lowercase starts with "auth" and contains "value_value.1") of it) of file "/etc/pam.d/system-auth"

Switching to:

(line number of line whose (it as string as lowercase starts with "auth" and it as string as lowercase contains "value number 1") of it > line number of line whose (it as string as lowercase starts with "auth" and it as string as lowercase contains "value number 2") of it) of file "/etc/pam.d/system-auth"

I think fixed that error, and will hopefully figure things out for me.

[bdowell]

Yay, I’m now getting a True or False (as expected) as to whether the lines are in the right order in the file. Thanks again all, especially @Aram for the patience and suggestions. Definitely very helpful and offering up more options on how to gather information from the systems I’m monitoring.

[bdowell]

heh, so close… almost done with this particular analysis, but… needed to expand slightly on above, and don’t quite have things right for the last check that I need:

I have to check for existence of a key/value to see if it is there. If it is there, then I need to confirm that if the line is there, it comes after another key value.

So… I was trying to use “If … then … else” and don’t have it working yet.

Relevance is like this:

if ((line of file whose (it as string as lowercase starts with "auth") and it as string as lowercase contains "value that shouldn't be in the file or must be later in the file") of file "/etc/pam.d/system-auth") then ((line number of line whose (it as string as lowercase starts with "auth" and it as string as lowercase contains "value that must come first") of it < line number of line whose (it as string as lowercase starts with "auth" and it as string as lowercase contains "value that shouldn't be in the file or must be later in the file") of it) of file "/etc/pam.d/system-auth") else "Not in the file"

[bdowell]

Got what I needed. Broke it down a little more. Returning two properties… One to tell me if the line that I would be looking for exists, the second to check the order of the lines (without worrying about checking for the existence of the line first)

A bit less elegant, but sufficient for what I needed.

Again, thanks all!

[JasonWalker]

To be sure I’m clear…

• Must contain words_words.1
• May contain words_words.2, but only if it comes after words_words.1
  …is that right?

[JasonWalker]

If I’m reading that right, we can make use of the pipe “|” as an error handler…
<error> | "alternative" --> “alternative”

We can also use “it as trimmed string starts with” when checking a line, to account for leading spaces.

Through careful use of plural “minima” and singular “minimum” (with pipe error handling) I think we can build up the query this way -

q: lines of file "d:\temp\system-auth.txt"
A: #
A: # /etc/pam.d/common-auth - auth settings...
A: #
A: # this file...
A: 
A: auth  [ additional info ]   words_words.1
A: #
A: #
A: auth  otherword   words_words.2
A: #
A: #
A: auth  anotherword   words_words.3
A: #
A: # end of file
T: 0.483 ms
I: plural file line

// Example - If we look for line and use the singular "maximum" for a word that is not present, we get an error
q: (maximum of line number of lines whose (it contains "words_words.4") of it) of file "d:\temp\system-auth.txt"
E: Singular expression refers to nonexistent object.

// Example - Trap the error, and if the line doesn't exist just return the total number of lines (so any other line number is less than it)
q: (maximum of line number of lines whose (it contains "words_words.4") of it | number of lines of it + 1) of file "d:\temp\system-auth.txt"
A: 15
T: 0.711 ms
I: singular integer

// Now we look for existing of words_words.1 - using plural "minima" so a missing line is not an error, but gives a "false" for the expression.
// Compare its line number to the minimum of line numbers of lines containing words_words.2.  If words_words.2 is missing, there is an error that is trapped and replaced by the number of lines.  We increment to number of lines + 1, in case words_words.1 is on the last line of the file this still should give True.
// Within the whose() clause, "item 0 of it" refers tot he minima line numbers containing words_words.1, and "item 1 of it" refers to the original file.
q: (exists (minima of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.1" ) of it, it) whose (item 0 of it <  minimum of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.2") of item 1 of it | number of lines of item 1 of it + 1) ) of file "d:\temp\system-auth.txt"
A: True
T: 0.827 ms
I: singular boolean

// Other tests - is words_words.2 before words_words.1 (should give False)
q: (exists (minima of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.2" ) of it, it) whose (item 0 of it <  minimum of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.1") of item 1 of it | number of lines of item 1 of it + 1) ) of file "d:\temp\system-auth.txt"
A: False
T: 0.829 ms
I: singular boolean

// Other tests - is words_words.1 before (missing) words_words.5 (should give True)
q: (exists (minima of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.1" ) of it, it) whose (item 0 of it <  minimum of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.5") of item 1 of it | number of lines of item 1 of it + 1) ) of file "d:\temp\system-auth.txt"
A: True
T: 1.144 ms
I: singular boolean

// Other tests - is (missing) words_words.5 before words_words.1 (should give False)
q: (exists (minima of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.5" ) of it, it) whose (item 0 of it <  minimum of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.1") of item 1 of it | number of lines of item 1 of it + 1) ) of file "d:\temp\system-auth.txt"
A: False
T: 0.567 ms
I: singular boolean

…and just after posting I simplified it a bit further -

q: exists (minima of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.1" ) of it, minimum of line numbers of lines whose (it as trimmed string starts with "auth" and it contains "words_words.2") of it | number of lines of it + 1) whose (item 0 of it <  item 1 of it) of file "d:\temp\system-auth.txt"
A: True
T: 0.839 ms
I: singular boolean

[bdowell]

Thanks again Jason for your help on this and these great examples to look at and learn from. Much appreciated!