I am trying to add 3 cells with the following values:
Cell 1: =+A2*1E+017 (A2 cell value is 33, so it results correctly in 33 followed by 17 zeros)
Cell 2: =+C2*100 (C2 cell value is 88982563, so it displays correctly 8898256300)
Cell 3: a simple sum of the previous two values
Instead of getting 3300000008898256300, I get 3300000008898260000. All numbers are integers, there are no decimal places.
Why OO calc is rounding an integer number? Is it too long? The same problem occurred with Libre Office and Microsoft Office 2003. Thank you for any hint.
[Dropped] Error with long integers calculations
[Dropped] Error with long integers calculations
Last edited by MrProgrammer on Fri Dec 05, 2025 5:36 pm, edited 1 time in total.
Reason: Dropped: No reply from martago
Reason: Dropped: No reply from martago
open office 3.2 in MS Windows XP
Re: error with long integers calculations
Yes. Calc does calculations with values limited to about 15 digits. Yours are 20.martago wrote:... Why OO calc is rounding an integer number? Is it too long? ...
Can't you leave out the 10^17 factor?
AOO4/LO5 • Linux • Fedora 23
-
ken johnson
- Volunteer
- Posts: 918
- Joined: Sun May 31, 2009 1:35 am
- Location: Sydney, Australia
Re: error with long integers calculations
Yes, 15 digits is the limit.martago wrote:Is it too long?
Ken Johnson
AOO 4.1.3 on Ms Windows 10
Help improve this forum by clicking the EDIT button on your first post then adding "[Solved]" to its heading once a solution to your problem has been provided.
Help improve this forum by clicking the EDIT button on your first post then adding "[Solved]" to its heading once a solution to your problem has been provided.
Re: error with long integers calculations
The above calculation was the quick way to find out the two check digits of a 19-long bank account number. I can also calculate through an operation performed individually in each position of the 19-long integer. Anyway I should have the possibility to calculate longer integers, possibility through the installation of an OO extension. Maybe some day someone will built it. Thanks for the helpful information.
open office 3.2 in MS Windows XP
Re: error with long integers calculations
Maybe there is an arbitrary-precision math package for Calc; I have not heard of one.martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number. ...
There are some alternative suggestions here that may (or may not) be appropriate for your case:
http://en.wikipedia.org/wiki/Internatio ... on_on_IBAN
AOO4/LO5 • Linux • Fedora 23
- Hagar Delest
- Moderator
- Posts: 33625
- Joined: Sun Oct 07, 2007 9:07 pm
- Location: France
Re: Error with long integers calculations
Since it's almost an alphanumeric string, perhaps you can concatenate 2 sets of numbers as text.
LibreOffice 25.2 on Linux Mint Debian Edition (LMDE 7 Gigi) and 25.2 portable on Windows 11.
Re: error with long integers calculations
There must be more to it... Where in that calculation of yours do you find the check digits?martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number...
All you do with the calculation as shown is to insert a lot of zeroes, and without the rounding the position of the initial numbers in the result would be predictable. IOW, any digit extracted from the result of that calculation, could just as well be extracted from the given base numbers (unless the number in C2 has more than 15 digits).
Normally, check digits are calculated using a specific multiplier for each digit in the account number, and then applying some combination of digit sums and/or modulo calculations.
In many cases you can write a compact formula for it, and in most cases such a formula is easier to make if you handle the account number as a character string.
Where did you find the specs for your check digits?
Apache OO 4.1.16 and LibreOffice 25.8, mostly on Ms Windows 10 and 11.