[Issue] 19/35 = 13/24 ?

Discuss the spreadsheet application
Post Reply
rweisensee
Posts: 1
Joined: Thu Mar 22, 2018 3:37 pm

[Issue] 19/35 = 13/24 ?

Post by rweisensee »

When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct. It's probably something with the way I am setting this up. What am I doing wrong?
Last edited by Hagar Delest on Thu Mar 22, 2018 11:04 pm, edited 1 time in total.
Reason: tagged [Issue] (link to a bug report).
OpenOffice 4.1.5 on Windows 10
User avatar
MrProgrammer
Moderator
Posts: 4903
Joined: Fri Jun 04, 2010 7:57 pm
Location: Wisconsin, USA

Re: 19/35 = 13/24 ?

Post by MrProgrammer »

rweisensee wrote:When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct
The fraction format has known bugs.
Arbitrary precision in fraction number formats
An equation =1/10 is displayed as 1/9 in Calc

You could download/install LibreOffice and see if the bugs have been corrected there.

If this solved your problem please go to your first post use the Edit button and add [Solved] to the start of the title. You can select the green checkmark icon at the same time.

[Tutorial] Ten concepts that every Calc user should know
Mr. Programmer
AOO 4.1.7 Build 9800, MacOS 13.6.3, iMac Intel.   The locale for any menus or Calc formulas in my posts is English (USA).
User avatar
keme
Volunteer
Posts: 3702
Joined: Wed Nov 28, 2007 10:27 am
Location: Egersund, Norway

Re: 19/35 = 13/24 ?

Post by keme »

Annoying bug. One workaround is to increase the number of digits for numerator/denominator. This seems to improve the quality of the intermediate calculation performed to determine the fraction. Format code "# ##?/##?" will solve it for this case.

This "fraction extender" is not always desirable, as it gives unwanted precision for some fractions. MROUND() may be used to eliminate "monster fractions", but care must be taken to choose a suitable multiple for the function. This may yield a "monster formula" instead. :(
User avatar
Lupp
Volunteer
Posts: 3548
Joined: Sat May 31, 2014 7:05 pm
Location: München, Germany

Re: 19/35 = 13/24 ?

Post by Lupp »

The bugs concerning fraction formats (There were lots!) were common heritage of AOO and LibreOffice.
Addressing LibO I filed a related bug report on 2016-05-22: https://bugs.documentfoundation.org/sho ... i?id=99996 .
The issue was fixed by 2016-07-09 for LibO (since V5.3.0) thanks to Laurent BP and there were some additional improvements also concerning the (silly) setting 'Precision as shown' in this version and in some subsequent ones.
See: https://wiki.documentfoundation.org/Rel ... ber_Format.

(I still feel sure that independent buck tracking and fixing for two branches generally based on the same code is utterly inefficient and will probably not work at all. It's not only unsatisfying to still be harrassed by bugs for which resolving code is available for more than 20 months now, but also to have a persisting case where relying on the wrong behaviour can affect interoperability. That's like the famous "Right or wrong - MS!")

If interested to more depth you may look into the attached demo. It was last saved with LibO and being opened with any recent LibO it should show fractions applying the enhanced range of detailed fraction formats correctly in compliance with mathematics. The implemented algorithm uses the development of numbers into regular continued fractions.
Attachments
fractionsCompareAoo_2.ods
(14.58 KiB) Downloaded 107 times
On Windows 10: LibreOffice 24.2 (new numbering) and older versions, PortableOpenOffice 4.1.7 and older, StarOffice 5.2
---
Lupp from München
Post Reply