FJCC wrote:This is a limitation of doing math on a computer. The computer does math in base-2, not the base-10 notation that is used for display, and is limited to a certain number of digits in a number. The computer cannot express certain numbers exactly, though they may look "simple" in decimal notation. It is the same problem that decimal notation has expressing the value of 1/3. It cannot be done exactly with a finite number of digits. This can lead to tiny errors in common calculations and is a big concern when a great number of calculations are needed.
I understand that the math is done in base-2, but why are the results not consistant?
I guess because they lied to me better?
Perhaps LO uses the same conversion?
40.01 needs one bit more for its integer part than 30.01.
Decimal_sign___exponent_______________________mantissa__________________________
40.01 = __0___10000000100___0100000000010100011110101110000101000111101011100001
30.01 = __0___10000000011___1110000000101000111101011100001010001111010111000011
Lupp wrote:@jrkrideau: You surely wanted to point to the Sage mathematical software (See https://www.sagemath.org/). Users searching for "Sage" on the web may be led to the commercial software offered by the Sage Group.
(For the bit of math I still do, I personally use wxMaxima. I don't know Sagemath and can't compare the solutions therefor.)
keme wrote: I think it must have been Maxima or R, but I am not absolutely certain.
Users browsing this forum: No registered users and 22 guests