I have the below files in C:\temp folder:
test1_20210123165009.txt
test3_20210125205107.txt
test2_20210124184449.txt
test4_20210126224733.txt
How to sort the above files in below order using BigFix relevance:
test4_20210126224733.txt
test3_20210125205107.txt
test2_20210124184449.txt
test1_20210123165009.txt
Reverse sorting is another tricky one, but can be accomplished as well.
Sorting, reverse sorting, data summation by groups, all of these tend to be more of a Reporting need than a âprocess on the endpointâ type of need. In the reporting layer, you can use javascript and other tools that have inbuilt and easy methods for sorting, but since you asked, I am certainly one to rise to a challenge to show how it might be done at the BigFix agent level.
Could you please share your use case for this query?
It was not clear if you wanted to sort by the integer after the word test, or the part between the underscore and the dot. Either is doable with the techniques I will show below. I chose the slightly less wordy âtest#â prefix for this demo.
Starting with a simple filter to limit this to only the files starting with âtestâ
q: names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
A: test1_20210123165009.txt
A: test2_20210124184449.txt
A: test3_20210125205107.txt
A: test4_20210126224733.txt
You probably noticed that Windows name sorts the files by default as part of our query.
Next up, build the reverse sorted portions as a string with separators, then use a reversing trick to reverse the entire string.
https://help.hcltechsw.com/bigfix/9.2/platform/Platform/Relevance/c_reversing_a_string.html
Q: concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
A: test1|test2|test3|test4
T: 9.571 ms
I: singular string
q: concatenation of characters (lengths of (following texts of (positions of it))) of concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
A: 4tset|3tset|2tset|1tset
T: 6.699 ms
I: singular string
Then we break it apart at the pipe and reverse each bit once again
q: (concatenation of characters (lengths of (following texts of (positions of it))) of it) of substrings separated by "|" of concatenation of characters (lengths of (following texts of (positions of it))) of concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
A: test4
A: test3
A: test2
A: test1
Now we have the exact sort we were searching for. We will use that in a minute.
Next we need to build a tuple to compare our sort with first and then stick that into a set (for efficiency)
q: (preceding text of first "_" of it, it) of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
A: test1, test1_20210123165009.txt
A: test2, test2_20210124184449.txt
A: test3, test3_20210125205107.txt
A: test4, test4_20210126224733.txt
T: 5.821 ms
I: plural ( substring, string )
q: set of (it as string) of (preceding text of first "_" of it, it) of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"
E: This expression evaluates to an unrepresentable object of type "string set"
T: 2.883 ms
I: singular string set
Now we will put both parts together in one giant tuple, with the reverse sorted items as the first element and the set as the second one. This will allow us to retain the careful ordering we did in our prior step. If I was to Set the first part, it would undo our reverse sort, as sets dedupe and alpha sort when they are created.
q: (item 0 of it, elements of item 1 of it) of (((concatenation of characters (lengths of (following texts of (positions of it))) of it) of substrings separated by "|" of concatenation of characters (lengths of (following texts of (positions of it))) of concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test" ), (set of (it as string) of (preceding text of first "_" of it, it) of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"))
A: test4, ( test1, test1_20210123165009.txt )
A: test4, ( test2, test2_20210124184449.txt )
A: test4, ( test3, test3_20210125205107.txt )
A: test4, ( test4, test4_20210126224733.txt )
A: test3, ( test1, test1_20210123165009.txt )
A: test3, ( test2, test2_20210124184449.txt )
A: test3, ( test3, test3_20210125205107.txt )
A: test3, ( test4, test4_20210126224733.txt )
A: test2, ( test1, test1_20210123165009.txt )
A: test2, ( test2, test2_20210124184449.txt )
A: test2, ( test3, test3_20210125205107.txt )
A: test2, ( test4, test4_20210126224733.txt )
A: test1, ( test1, test1_20210123165009.txt )
A: test1, ( test2, test2_20210124184449.txt )
A: test1, ( test3, test3_20210125205107.txt )
A: test1, ( test4, test4_20210126224733.txt )
By now you probably see we are headed towards the filter technique from the prior puzzler.
q: (item 0 of it, elements of item 1 of it) whose (tuple string item 0 of item 1 of it = item 0 of it) of (((concatenation of characters (lengths of (following texts of (positions of it))) of it) of substrings separated by "|" of concatenation of characters (lengths of (following texts of (positions of it))) of concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test" ), (set of (it as string) of (preceding text of first "_" of it, it) of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"))
A: test4, ( test4, test4_20210126224733.txt )
A: test3, ( test3, test3_20210125205107.txt )
A: test2, ( test2, test2_20210124184449.txt )
A: test1, ( test1, test1_20210123165009.txt )
And finally we strip off the unneeded parts of the tuple where our sorting happened to get one possible solution.
Is this your card, @KeepLearning ?
q: tuple string items 1 of items 1 of (item 0 of it, elements of item 1 of it) whose (tuple string item 0 of item 1 of it = item 0 of it) of (((concatenation of characters (lengths of (following texts of (positions of it))) of it) of substrings separated by "|" of concatenation of characters (lengths of (following texts of (positions of it))) of concatenation "|" of preceding texts of firsts "_" of names of files whose (name of it as lowercase starts with "test") of folder "c:\test" ), (set of (it as string) of (preceding text of first "_" of it, it) of names of files whose (name of it as lowercase starts with "test") of folder "c:\test"))
A: test4_20210126224733.txt
A: test3_20210125205107.txt
A: test2_20210124184449.txt
A: test1_20210123165009.txtHi @brolly33, thanks a lot for the reply and solutionâŚ
However, apologies for the question which is wrongly framedâŚ
Below is the example:
Need to get top 5 .txt files from the below list:
Abc_20210124184449.txt
Abc_20210121225468.txt
Abc_20210123165009.txt
Abc_20210125205107.txt
Abc_20210126224733.txt
Abc_20210120342156.txt
Abc_20210115612456.txt
Top 5 .txt files output:
Abc_20210126224733.txt
Abc_20210125205107.txt
Abc_20210124184449.txt
Abc_20210123165009.txt
Abc_20210121225468.txt
I too am interested in the use case here. Why does the order of the results returned matter? This context would be quite helpful to understand.
From your example above, it looks like you just want the files from the last 5 days (or the 5 most recent files).
What is it that you are trying to achieve with this relevance?
Iâm interested in the use-case as well, but I also love a good relevance puzzle. Does this meet what youâre looking for?
For sample data, Iâll look to the BES Client Logs. These are named for each day so they should sort in the same kind of order youâre looking for.
q: names of files whose (name of it ends with ".log") of folders "__Global\Logs" of data folder of client
A: 20210119.log
A: 20210121.log
A: 20210122.log
A: 20210125.log
A: 20210126.log
A: 20210127.log
A: 20210128.log
A: 20210129.log
T: 16.119 ms
I: plural string
While mine already happen to be sorted smallest to largest, Iâd use the âunique valuesâ inspector to ensure they are sorted alphabetically by name, and then combine them all with the âcomma-spaceâ separator. Combining them with a comma and a space allows me to use the âtuple string itemâ inspector to retrieve each one later.
q: concatenation ", " of unique values of names of files whose (name of it ends with ".log") of folders "__Global\Logs" of data folder of client
A: 20210119.log, 20210121.log, 20210122.log, 20210125.log, 20210126.log, 20210127.log, 20210128.log, 20210129.log
T: 14.771 ms
I: singular string
I can find out how many files there are using the ânumber of tuple string itemsâ inspector. In my case I have 8 logs (which would be indexed as tuple string item 0 through tuple string item 7 )
q: number of tuple string items of concatenation ", " of unique values of names of files whose (name of it ends with ".log") of folders "__Global\Logs" of data folder of client
A: 8
T: 13.262 ms
I: singular integer
Since these are sorted, âtuple string item 0â would be the first one, âtuple string item 1 of itâ would be the second, and âtuple string item (number of tuple string items of it - 1)â would be the last one.
In Relevance we can count (forward or backward) with the integers in (<start>, <finish>) construct:
q: integers in (0, 5)
A: 0
A: 1
A: 2
A: 3
A: 4
A: 5
T: 12.227 ms
I: plural integer
q: integers in (5, 0)
A: 5
A: 4
A: 3
A: 2
A: 1
A: 0
T: 12.171 ms
I: plural integer
Now itâs a matter of returning the last five tuple strings:
q: (tuple string items (integers in (number of tuple string items of it - 1, number of tuple string items of it - 5)) of it) of concatenation ", " of unique values of names of files whose (name of it ends with ".log") of folders "__Global\Logs" of data folder of client
A: 20210129.log
A: 20210128.log
A: 20210127.log
A: 20210126.log
A: 20210125.log
T: 11.651 ms
I: plural stringThere are at least of use cases I can think of - âfind errors in the last 5 daysâ of logsâ, or maybe âdelete all logs except for the last 5 daysâ of themâ
@JasonWalker @Aram
Getting the 5 latest makes sense to me to answer a what was recently touched type of question.
I too am puzzled at the âwhyâ behind needing a reverse sort in relevance. Looking forward to that response from @KeepLearning.
To the exact query above, itâs just applying the same techniques for filtering for top 5, reversing, trimming and reversing a second time, then filtering and trimming off the sort at the end.
Much easier to do in JavaScript at the reporting layer instead of trying to get the client to do it locally with relevance.
q: files whose (name of it as lowercase ends with ".txt") of folder "c:\test"
A: "Abc_20210115612456.txt" "" "" "" ""
A: "Abc_20210120342156.txt" "" "" "" ""
A: "Abc_20210121225468.txt" "" "" "" ""
A: "Abc_20210123165009.txt" "" "" "" ""
A: "Abc_20210124184449.txt" "" "" "" ""
A: "Abc_20210125205107.txt" "" "" "" ""
A: "Abc_20210126224733.txt" "" "" "" ""
A: "DLV cscript command.txt" "" "" "" ""
A: "dlv.txt" "" "" "" ""
A: "Fake.txt" "" "" "" ""
A: "one more file 5.txt" "" "" "" ""
A: "test3_20210125205107.txt" "" "" "" ""
A: "test4_20210126224733.txt" "" "" "" ""
q: tuple string items 1 of items 1 of (item 0 of it, elements of item 1 of it) whose (item 0 of it = tuple string item 0 of item 1 of it) of (( (concatenation of characters (lengths of (following texts of (positions of it))) of it) of (tuple string items (0;1;2;3;4) of it) of concatenation of characters (lengths of (following texts of (positions of it))) of (it as string) of concatenation " ," of (following text of first "_" of preceding text of last "." of name of it|"") of files whose (name of it as lowercase ends with ".txt" and name of it as lowercase starts with "abc_") of folder "c:\test"),(set of (it as string) of (following text of first "_" of preceding text of last "." of name of it|"", name of it) of files whose (name of it as lowercase ends with ".txt" and name of it as lowercase starts with "abc_") of folder "c:\test"))
A: Abc_20210126224733.txt
A: Abc_20210125205107.txt
A: Abc_20210124184449.txt
A: Abc_20210123165009.txt
A: Abc_20210121225468.txtThanks @JasonWalker @brolly33 for a clear solution!
This is really helpful.
The use case is to keep latest 3-4 files and kind of backup the rest.