[Solved] Calc mistakes in rounding numbers

Discuss the spreadsheet application
Post Reply
excelsior82
Posts: 2
Joined: Tue Feb 19, 2008 6:28 pm

[Solved] Calc mistakes in rounding numbers

Post by excelsior82 »

I already posted that issue on oooforums.org today.
I was working with invoices when I saw a strange error with numbers that weren't rounded correctly.

I had a value of 5302,27 € (without VAT)
When I applied the 5.5% VAT (value*1.055), the result given by OOo is 5593.89

If I look up a third figure after the decimal point, the real value is 5593.895
The number should be rounded up to 5593.90

I've tested with OOo 2.2.0 under windows (french locale) and OOo 2.3.0 under ubuntu (french locale). The error happens on the both configurations.

I have noticed that if I ask OOo to show 2 figures or 3 after the decimal point, the two values aren't equal for OOo.
More spectacular, if I create a formula that simply point on the VAT-applied result, the value given is rounded rightly !

See this file for a practical example

I'd be very glad if someone can explain me these strange things
Last edited by excelsior82 on Tue Feb 19, 2008 7:24 pm, edited 1 time in total.
User avatar
Villeroy
Volunteer
Posts: 31279
Joined: Mon Oct 08, 2007 1:35 am
Location: Germany

Re: Calc mistakes in rounding numbers

Post by Villeroy »

5302.27 * 1.055 = 5593.89485 (exactly)
Rounded to 3 digits: 5593.895 (the following 8 rounds up the 4)
Rounded to 2 digits: 5593.89 (the following 4 rounds down)
Please, edit this topic's initial post and add "[Solved]" to the subject line if your problem has been solved.
Ubuntu 18.04 with LibreOffice 6.0, latest OpenOffice and LibreOffice
excelsior82
Posts: 2
Joined: Tue Feb 19, 2008 6:28 pm

Re: Calc mistakes in rounding numbers

Post by excelsior82 »

GOSH, that was so obvious !!! *kicks himself in the nuts*

Thanks you for the information, now I understand the company invoices is rounded to the third digit.

SOLVED.

OpenOffice is better than me :(
Post Reply