(imported topic written by Bill.Ehardt)
I thought this would be simple… but I can’t seem to find the easy way to do it. I’m calculating space in GB, but I want to limit to 2 decimal places.
Turn this: 4.93857574462891 into 4.94
How can I do this easily knowing space could go from under 1 to over 1000?
(imported comment written by amelgares)
Multiply your float by “1.00” using the number of zeros you want in the result.
Example:
Q: product
of
(
“3.2592”
as
floating point
;
“1.000”
as
floating point)
A: 3.259
Q: product
of
(
“3.2592”
as
floating point
;
“1.00”
as
floating point)
A: 3.26
(imported comment written by NoahSalzman)
This is overly complex but should work.
q:
(relative significance place ((
number of
characters
of
(
it
as
floating point
as
integer
as
string))
as
integer
+
1
)
of
(
it
as
floating point))
of
“423.93857574462891”
A:
423.94
q:
(relative significance place ((
number of
characters
of
(
it
as
floating point
as
integer
as
string))
as
integer
+
1
)
of
(
it
as
floating point))
of
“4.93857574462891”
A:
4.94
(imported comment written by Bill.Ehardt)
Awesome ty! I think for what I need the first is a bit easier (though both can give a bit other than 2 decimal places at times)
(imported comment written by MattPeterson)
I hate to bring this up from the dead, but I will anyway…
I’m trying to convert a number with lots of decimal points into one with fewer. The suggests above work, for the most part, accept when dealing with rounding up.
The problem I’m having is trying to round a number like 5.999 to two decimal places. This should logically should be rounded to an even 6, but instead it’s being rounded to 5.100, which leads people to believe the number is a lot closer to 5 than 6 :). Any suggestions?
q:
relative significance place
2
of
(
“5.999”
as
floating point)
A:
5.100
T:
0.048 ms
I:
singular floating point
q:
product
of
(
“5.999”
as
floating point
;
“1.00”
as
floating point)
A:
5.100
T:
0.089 ms
I:
singular floating point
(imported comment written by jeremylam)
That is very strange because I don’t get the same results when I run the same queries in my Fixlet Debugger.
What version of the Fixlet debugger (or actual client) are you using?
q: version of client
A: 8.2.1409.0
T: 0.385 ms
I: singular version
q: relative significance place 2 of (“5.999” as floating point)
A: 6.00
T: 0.031 ms
I: singular floating point
q: product of (“5.999” as floating point; “1.00” as floating point)
A: 6.00
T: 0.075 ms
I: singular floating point
(imported comment written by MattPeterson)
We’re running 9.1.1117, so maybe a bug with the new version. Clients are behaving the same way when reporting results from an analysis, so this isn’t limited to just the debugger.
(imported comment written by AlanM)
Definitely an issue which I’m working on fixing. Slight edge case in the handling here which was unfortunate. The 8.2 result (which is indeed correct) should be returned.