Page 1 of 1

[Dropped] Error with long integers calculations

Posted: Thu Aug 15, 2013 4:55 pm
by martago
I am trying to add 3 cells with the following values:

Cell 1: =+A2*1E+017 (A2 cell value is 33, so it results correctly in 33 followed by 17 zeros)

Cell 2: =+C2*100 (C2 cell value is 88982563, so it displays correctly 8898256300)

Cell 3: a simple sum of the previous two values

Instead of getting 3300000008898256300, I get 3300000008898260000. All numbers are integers, there are no decimal places.

Why OO calc is rounding an integer number? Is it too long? The same problem occurred with Libre Office and Microsoft Office 2003. Thank you for any hint.

Re: error with long integers calculations

Posted: Thu Aug 15, 2013 5:09 pm
by acknak
martago wrote:... Why OO calc is rounding an integer number? Is it too long? ...
Yes. Calc does calculations with values limited to about 15 digits. Yours are 20.

Can't you leave out the 10^17 factor?

Re: error with long integers calculations

Posted: Thu Aug 15, 2013 5:11 pm
by ken johnson
martago wrote:Is it too long?
Yes, 15 digits is the limit.

Ken Johnson

Re: error with long integers calculations

Posted: Thu Aug 15, 2013 5:23 pm
by martago
The above calculation was the quick way to find out the two check digits of a 19-long bank account number. I can also calculate through an operation performed individually in each position of the 19-long integer. Anyway I should have the possibility to calculate longer integers, possibility through the installation of an OO extension. Maybe some day someone will built it. Thanks for the helpful information.

Re: error with long integers calculations

Posted: Thu Aug 15, 2013 6:09 pm
by acknak
martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number. ...
Maybe there is an arbitrary-precision math package for Calc; I have not heard of one.

There are some alternative suggestions here that may (or may not) be appropriate for your case:
http://en.wikipedia.org/wiki/Internatio ... on_on_IBAN

Re: Error with long integers calculations

Posted: Thu Aug 15, 2013 6:29 pm
by Hagar Delest
Since it's almost an alphanumeric string, perhaps you can concatenate 2 sets of numbers as text.

Re: error with long integers calculations

Posted: Fri Aug 16, 2013 11:19 am
by keme
martago wrote:The above calculation was the quick way to find out the two check digits of a 19-long bank account number...
There must be more to it... Where in that calculation of yours do you find the check digits?
All you do with the calculation as shown is to insert a lot of zeroes, and without the rounding the position of the initial numbers in the result would be predictable. IOW, any digit extracted from the result of that calculation, could just as well be extracted from the given base numbers (unless the number in C2 has more than 15 digits).

Normally, check digits are calculated using a specific multiplier for each digit in the account number, and then applying some combination of digit sums and/or modulo calculations.
In many cases you can write a compact formula for it, and in most cases such a formula is easier to make if you handle the account number as a character string.

Where did you find the specs for your check digits?