[Dropped] Error with long integers calculations

Discuss the spreadsheet application
Locked
martago
Posts: 3
Joined: Thu Jul 15, 2010 2:06 pm

[Dropped] Error with long integers calculations

Post by martago »

I am trying to add 3 cells with the following values:

Cell 1: =+A2*1E+017 (A2 cell value is 33, so it results correctly in 33 followed by 17 zeros)

Cell 2: =+C2*100 (C2 cell value is 88982563, so it displays correctly 8898256300)

Cell 3: a simple sum of the previous two values

Instead of getting 3300000008898256300, I get 3300000008898260000. All numbers are integers, there are no decimal places.

Why OO calc is rounding an integer number? Is it too long? The same problem occurred with Libre Office and Microsoft Office 2003. Thank you for any hint.
Last edited by MrProgrammer on Fri Dec 05, 2025 5:36 pm, edited 1 time in total.
Reason: Dropped: No reply from martago
open office 3.2 in MS Windows XP
User avatar
acknak
Moderator
Posts: 22756
Joined: Mon Oct 08, 2007 1:25 am
Location: USA:NJ:E3

Re: error with long integers calculations

Post by acknak »

martago wrote:... Why OO calc is rounding an integer number? Is it too long? ...
Yes. Calc does calculations with values limited to about 15 digits. Yours are 20.

Can't you leave out the 10^17 factor?
AOO4/LO5 • Linux • Fedora 23
ken johnson
Volunteer
Posts: 918
Joined: Sun May 31, 2009 1:35 am
Location: Sydney, Australia

Re: error with long integers calculations

Post by ken johnson »

martago wrote:Is it too long?
Yes, 15 digits is the limit.

Ken Johnson
AOO 4.1.3 on Ms Windows 10
Help improve this forum by clicking the EDIT button on your first post then adding "[Solved]" to its heading once a solution to your problem has been provided.
martago
Posts: 3
Joined: Thu Jul 15, 2010 2:06 pm

Re: error with long integers calculations

Post by martago »

The above calculation was the quick way to find out the two check digits of a 19-long bank account number. I can also calculate through an operation performed individually in each position of the 19-long integer. Anyway I should have the possibility to calculate longer integers, possibility through the installation of an OO extension. Maybe some day someone will built it. Thanks for the helpful information.
open office 3.2 in MS Windows XP
User avatar
acknak
Moderator
Posts: 22756
Joined: Mon Oct 08, 2007 1:25 am
Location: USA:NJ:E3

Re: error with long integers calculations

Post by acknak »

martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number. ...
Maybe there is an arbitrary-precision math package for Calc; I have not heard of one.

There are some alternative suggestions here that may (or may not) be appropriate for your case:
http://en.wikipedia.org/wiki/Internatio ... on_on_IBAN
AOO4/LO5 • Linux • Fedora 23
User avatar
Hagar Delest
Moderator
Posts: 33625
Joined: Sun Oct 07, 2007 9:07 pm
Location: France

Re: Error with long integers calculations

Post by Hagar Delest »

Since it's almost an alphanumeric string, perhaps you can concatenate 2 sets of numbers as text.
LibreOffice 25.2 on Linux Mint Debian Edition (LMDE 7 Gigi) and 25.2 portable on Windows 11.
User avatar
keme
Volunteer
Posts: 3791
Joined: Wed Nov 28, 2007 10:27 am
Location: Egersund, Norway

Re: error with long integers calculations

Post by keme »

martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number...
There must be more to it... Where in that calculation of yours do you find the check digits?
All you do with the calculation as shown is to insert a lot of zeroes, and without the rounding the position of the initial numbers in the result would be predictable. IOW, any digit extracted from the result of that calculation, could just as well be extracted from the given base numbers (unless the number in C2 has more than 15 digits).

Normally, check digits are calculated using a specific multiplier for each digit in the account number, and then applying some combination of digit sums and/or modulo calculations.
In many cases you can write a compact formula for it, and in most cases such a formula is easier to make if you handle the account number as a character string.

Where did you find the specs for your check digits?
Apache OO 4.1.16 and LibreOffice 25.8, mostly on Ms Windows 10 and 11.
Locked