[Issue] 19/35 = 13/24 ?

Discuss the spreadsheet application

[Issue] 19/35 = 13/24 ?

Postby rweisensee » Thu Mar 22, 2018 3:42 pm

When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct. It's probably something with the way I am setting this up. What am I doing wrong?
Last edited by Hagar Delest on Thu Mar 22, 2018 11:04 pm, edited 1 time in total.
Reason: tagged [Issue] (link to a bug report).
OpenOffice 4.1.5 on Windows 10
rweisensee
 
Posts: 1
Joined: Thu Mar 22, 2018 3:37 pm

Re: 19/35 = 13/24 ?

Postby MrProgrammer » Thu Mar 22, 2018 4:51 pm

rweisensee wrote:When I format a cell to accept fractions of the form xx/yy and I put in 19/35, OpenOffice Calc reduces this to 13/24 which isn't mathematically correct
The fraction format has known bugs.
Arbitrary precision in fraction number formats
An equation =1/10 is displayed as 1/9 in Calc

You could download/install LibreOffice and see if the bugs have been corrected there.

If this solved your problem please go to your first post use the Edit button and add [Solved] to the start of the title. You can select the green checkmark icon at the same time.

[Tutorial] Ten concepts that every Calc user should know
Mr. Programmer
AOO 4.1.7 Build 9800 on MacOS 10.14.6.   The locale for any menus or Calc formulas in my posts is English (USA).
User avatar
MrProgrammer
Moderator
 
Posts: 3911
Joined: Fri Jun 04, 2010 7:57 pm
Location: Wisconsin, USA

Re: 19/35 = 13/24 ?

Postby keme » Thu Mar 22, 2018 8:23 pm

Annoying bug. One workaround is to increase the number of digits for numerator/denominator. This seems to improve the quality of the intermediate calculation performed to determine the fraction. Format code "# ##?/##?" will solve it for this case.

This "fraction extender" is not always desirable, as it gives unwanted precision for some fractions. MROUND() may be used to eliminate "monster fractions", but care must be taken to choose a suitable multiple for the function. This may yield a "monster formula" instead. :(
User avatar
keme
Volunteer
 
Posts: 3402
Joined: Wed Nov 28, 2007 10:27 am
Location: Egersund, Norway

Re: 19/35 = 13/24 ?

Postby Lupp » Thu Mar 22, 2018 9:53 pm

The bugs concerning fraction formats (There were lots!) were common heritage of AOO and LibreOffice.
Addressing LibO I filed a related bug report on 2016-05-22: https://bugs.documentfoundation.org/sho ... i?id=99996 .
The issue was fixed by 2016-07-09 for LibO (since V5.3.0) thanks to Laurent BP and there were some additional improvements also concerning the (silly) setting 'Precision as shown' in this version and in some subsequent ones.
See: https://wiki.documentfoundation.org/Rel ... ber_Format.

(I still feel sure that independent buck tracking and fixing for two branches generally based on the same code is utterly inefficient and will probably not work at all. It's not only unsatisfying to still be harrassed by bugs for which resolving code is available for more than 20 months now, but also to have a persisting case where relying on the wrong behaviour can affect interoperability. That's like the famous "Right or wrong - MS!")

If interested to more depth you may look into the attached demo. It was last saved with LibO and being opened with any recent LibO it should show fractions applying the enhanced range of detailed fraction formats correctly in compliance with mathematics. The implemented algorithm uses the development of numbers into regular continued fractions.
Attachments
fractionsCompareAoo_2.ods
(14.58 KiB) Downloaded 41 times
On Windows 10: LibreOffice 7.0 and older versions, PortableOpenOffice 4.1.7 and older, StarOffice 5.2
---
Lupp from München
User avatar
Lupp
Volunteer
 
Posts: 3003
Joined: Sat May 31, 2014 7:05 pm
Location: München, Germany


Return to Calc

Who is online

Users browsing this forum: No registered users and 19 guests