Rounding issues with relevance

Content Authoring
1

We provide VM services to customers. To calculate how much we charge for a VM with specific number of vCPU we make the following calculation:

Speed of CPU in GHz * Number of Cores

Since these VM can have their capacity changed by various people, we track capacity of existing VM with multiple custom properties in BigFix. Here’s the one we use to give us the “Capacity Units” for a VM CPU:

Q: significant digits 2 of (if (hyperthreading enabled = false) then ( speed of main processor/mhz * number of processors ) else ( speed of main processor/mhz * number of processors/2 )) / 1000

We’ve found that there’s an issue with rounding using this method. Here’s an example from a VM that should return 10 CU, but instead gives us 9:

Q: significant digits 2 of (if (hyperthreading enabled = false) then ( speed of main processor/mhz * number of processors ) else ( speed of main processor/mhz * number of processors/2 )) / 1000
A: 9
T: 0.119 ms

I broke it down step-by-step on this VM to show why it should return 10 instead of 9:

Q: speed of main processor/mhz
A: 2394
T: 0.053 ms

Q: speed of main processor/mhz * number of processors
A: 9576
T: 0.066 ms

Q: speed of main processor/mhz * number of processors /1000
A: 9
T: 0.102 ms

Even if I use significant digits, relative significance place, floating point, and integer, I can’t get it to round up correctly. Can anyone help me figure out what I’m doing wrong? Thanks.

2

I think I have this working using the following (cribbed together from this old forum thread: https://www.ibm.com/developerworks/community/forums/html/topic?id=aba035ba-3e9e-4ff1-b7e0-0ca52824e8b4)

relative significance place 1 of ((speed of main processor/mhz * number of processors) as floating point / 1000) as integer

Here’s how I got there:

Q: speed of main processor/mhz * number of processors / 1000
A: 11
T: 100

Q: (speed of main processor/mhz * number of processors) as integer / 1000
A: 11
T: 94

Q: (speed of main processor/mhz * number of processors) as floating point / 1000
A: 11.6000000000000
T: 150

Q: relative significance place 2 of ((speed of main processor/mhz * number of processors) as floating point / 1000)
A: 11.6
T: 134

Q: relative significance place 1 of ((speed of main processor/mhz * number of processors) as floating point / 1000)
A: 12.0
T: 97

Q: relative significance place 1 of ((speed of main processor/mhz * number of processors) as floating point / 1000) as integer
A: 12
T: 132
2 Likes
3

Thanks, I think that did it. I also came across that forum thread, but I just couldn’t get it to work for me. I was trying to test relative significance place against simple arithmetic. Either I got the syntax wrong, or you can’t use it against arithmetic like you can significant digits.