Page 1 of 1

[Issue] 19/35 = 13/24 ?

PostPosted: Thu Mar 22, 2018 3:42 pm
by rweisensee
When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct. It's probably something with the way I am setting this up. What am I doing wrong?

Re: 19/35 = 13/24 ?

PostPosted: Thu Mar 22, 2018 4:51 pm
by MrProgrammer
rweisensee wrote:When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct
The fraction format has known bugs.
Arbitrary precision in fraction number formats
An equation =1/10 is displayed as 1/9 in Calc

You could download/install LibreOffice and see if the bugs have been corrected there.

If this solved your problem please go to your first post use the Edit button and add [Solved] to the start of the title. You can select the green checkmark icon at the same time.

[Tutorial] Ten concepts that every Calc user should know

Re: 19/35 = 13/24 ?

PostPosted: Thu Mar 22, 2018 8:23 pm
by keme
Annoying bug. One workaround is to increase the number of digits for numerator/denominator. This seems to improve the quality of the intermediate calculation performed to determine the fraction. Format code "# ##?/##?" will solve it for this case.

This "fraction extender" is not always desirable, as it gives unwanted precision for some fractions. MROUND() may be used to eliminate "monster fractions", but care must be taken to choose a suitable multiple for the function. This may yield a "monster formula" instead. :(

Re: 19/35 = 13/24 ?

PostPosted: Thu Mar 22, 2018 9:53 pm
by Lupp
The bugs concerning fraction formats (There were lots!) were common heritage of AOO and LibreOffice.
Addressing LibO I filed a related bug report on 2016-05-22: https://bugs.documentfoundation.org/sho ... i?id=99996 .
The issue was fixed by 2016-07-09 for LibO (since V5.3.0) thanks to Laurent BP and there were some additional improvements also concerning the (silly) setting 'Precision as shown' in this version and in some subsequent ones.
See: https://wiki.documentfoundation.org/Rel ... ber_Format.

(I still feel sure that independent buck tracking and fixing for two branches generally based on the same code is utterly inefficient and will probably not work at all. It's not only unsatisfying to still be harrassed by bugs for which resolving code is available for more than 20 months now, but also to have a persisting case where relying on the wrong behaviour can affect interoperability. That's like the famous "Right or wrong - MS!")

If interested to more depth you may look into the attached demo. It was last saved with LibO and being opened with any recent LibO it should show fractions applying the enhanced range of detailed fraction formats correctly in compliance with mathematics. The implemented algorithm uses the development of numbers into regular continued fractions.