[Solved] Large file very slow to respond and CPU use is high

Discuss the word processor

[Solved] Large file very slow to respond and CPU use is high

Postby bedtime » Tue Jan 09, 2018 11:37 pm

I have a document that is 600+ pages (about 140,000+ words) and uses very fancy Adobe PRO font. It would slog along quite slowly, taking about 1+ seconds to process any key press. By chance, I moved the document onto another operating system without the Adobe font—the document had much less slow down (maybe .20 seconds per key press). Initially, I thought it to be the OS, but adding the Adobe font to this OS slowed it back down again. A few extra tests confirmed this. Adding the fonts to the system files and removing from the user .fonts (Linux) didn't make a difference.

Solution?

Replace the font 'temporarily':

Tools -> Options -> LibreOffice (I imagine this would work on OpenOffice. Please confirm) -> Fonts -> Replacement Table

Select the font you want to replace and which font you want to replace with (I chose 'Noto Sans' as I found it to be the most responsive on my computer—yes, I actually tested several to make sure), and apply it by ticking 'Always' and 'Screen only'. Before printing (PDF or paper), simply add it back by unticking those options. It has no effect on the contents of the saved document.

So, if I allow the Adobe font to display again, by unticking 'Always' and 'Screen only,' it immediately goes back into chug mode, even after it has fully repaginated (as it will have to do). Reticking those options solves it. Perhaps, I'm missing something, but this seems proof enough to me.

I wish I had known this long ago. Perhaps, it may help someone out there.

As for the reason why it works, I'm not fully sure. Maybe someone could fill me in? ;)
Last edited by bedtime on Thu Jan 11, 2018 1:10 pm, edited 2 times in total.
LibreOffice
Version: 5.4.2.2
Build ID: 1:5.4.2-0ubuntu0.17.10.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3;
bedtime
 
Posts: 11
Joined: Mon Jan 09, 2017 9:13 pm

Re: Tip: Replace fancy font in huge docs for better speed...

Postby RoryOF » Wed Jan 10, 2018 12:00 am

If the overall layout is important, it would be a good idea to choose a plain font that closely matched your fancy font's letter widths.
Apache OpenOffice 4.1.7 on Xubuntu 18.04.4 (mostly 64 bit version) and very infrequently on Win2K/XP
User avatar
RoryOF
Moderator
 
Posts: 31209
Joined: Sat Jan 31, 2009 9:30 pm
Location: Ireland

Re: Tip: Replace fancy font in huge docs for better speed...

Postby John_Ha » Wed Jan 10, 2018 2:33 am

I am surprised that changing a font makes any significant difference.

RoryOF wrote:If the overall layout is important, it would be a good idea to choose a plain font that closely matched your fancy font's letter widths.

It will never be a perfect match - you will almost certainly get different line and page spills.

I suggest that you create a new text document by File > New > Text document ..., and go Insert > File ... Now navigate to your file and insert it. Save As ..., and save as a .odt file. (See [Tutorial] Differences between Writer and MS Word files for why you should always work in and save files as .odt.)

Heavily edited documents (as yours no doubt is - File > Properties > General - how long has it been open for editing? and how many times?) seem to get "tangled". This removes redundant formatting information and speeds the response. See Re: [Solved] Compressing size of an odt containing large ima

Are you using direct formatting as opposed to Styles. If so, the file can become complex.

You may also care to think about using a Master Document, with sub-documents for each chapter where you edit only a chapter at any one time - the response is much quicker. See Ch 13 Using Master Documents, Creating a master document and Creating and Using Master Documents.

Try editing in Web View. See [Solved] AOO stops responding when opening this one document which says:

What is happening is that you (quite inadvertently!) are asking AOO to do a vast amount of work paginating the document because your many footnotes mean that any spill across a page end is likely to need a lot of work to sort it out and AOO may get into a loop doing so. The AOO developers thought of this and place a time limit on how long AOO stays in the loop so as to allow the document to work, albeit with layout errors like the blank gaps below the tables causing the pagination errors. I do not know if it is a bug that a table splits and leaves the white gaps because it could be working properly, and just that when AOO pulls out of the loop, AOO has not finished doing the table layout.


If your problem is solved please view your first post in this thread and click the Edit button (top right in the post) and add [Solved] in front of the subject.

NB I see you are running on Android. I have no direct experience of AndrOpen Office on Android but ...

... everything seems to be (much) slower on Android.
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK

Re: Tip: Replace fancy font in huge docs for better speed...

Postby bedtime » Wed Jan 10, 2018 3:15 pm

RoryOF wrote:If the overall layout is important, it would be a good idea to choose a plain font that closely matched your fancy font's letter widths.

Excellent idea. I did notice that the document was about 50+ pages with the new font.

John_Ha wrote:I am surprised that changing a font makes any significant difference.

Actually, with further testing, I am not so sure my conclusion was correct :? , and I am still wondering if it is making any difference at all. There are so many variables that could have affected the test. One being the time it takes to repaginate and if I waited long enough in the first place.

It will never be a perfect match - you will almost certainly get different line and page spills.

This has little effect on my document. Here is a link to a copy of the book (if anyone cares to see the type of layout it has): https://archive.org/stream/DeBelloGallico/Liber%2010#page/n11/mode/2up

There are 1500 individual 2 row columns (i.e., 1500 tables), which might be a factor. I cannot use columned pages as I want the formatting to be a specific way. The book uses 4 headers. 3000 same page footnotes. No sections. No chapters. Not sure if sectioning helps or not. No endnotes. Only 2 images. It also uses about 1000-1500? references (to vocabulary).

I suggest that you create a new text document by File > New > Text document ..., and go Insert > File ... Now navigate to your file and insert it. Save As ..., and save as a .odt file. (See [Tutorial] Differences between Writer and MS Word files for why you should always work in and save files as .odt.)

It is saved as .odt, but I will try this.

Heavily edited documents (as yours no doubt is - File > Properties > General - how long has it been open for editing? and how many times?) seem to get "tangled". This removes redundant formatting information and speeds the response. See Re: [Solved] Compressing size of an odt containing large ima

I will try this and report back with the results—thank you! I was meaning to ask about this very issue of speeding up and streamlining the document!

Are you using direct formatting as opposed to Styles. If so, the file can become complex.

For the most part it is using styles; the first 90% or so just uses custom styles, and the remaining ending content does use content that I have just dropped in italics, caps, and such, but I will eventually change this. All styles used are custom and the only style that is not custom is the footnote style.

You may also care to think about using a Master Document, with sub-documents for each chapter where you edit only a chapter at any one time - the response is much quicker. See Ch 13 Using Master Documents, Creating a master document and Creating and Using Master Documents.

I will have a look at this, but I don't like the idea of splitting it up.

Try editing in Web View. See [Solved] AOO stops responding when opening this one document which says:

What is happening is that you (quite inadvertently!) are asking AOO to do a vast amount of work paginating the document because your many footnotes mean that any spill across a page end is likely to need a lot of work to sort it out and AOO may get into a loop doing so. The AOO developers thought of this and place a time limit on how long AOO stays in the loop so as to allow the document to work, albeit with layout errors like the blank gaps below the tables causing the pagination errors. I do not know if it is a bug that a table splits and leaves the white gaps because it could be working properly, and just that when AOO pulls out of the loop, AOO has not finished doing the table layout.

I will try and see if it makes a difference. I remember trying this before, and it being much much slower.

If your problem is solved please view your first post in this thread and click the Edit button (top right in the post) and add [Solved] in front of the subject.

When and if I notice a difference, I will for sure mark as solved—I LOVE the ideas and the help, and in that way it is solved. ;)

NB I see you are running on Android. I have no direct experience of AndrOpen Office on Android but ...

... everything seems to be (much) slower on Android.

Not anymore. I initially started on Android, but now I am on Ubuntu using LibreOffice. Though, there was a time when I was using the document in a .doc or some such format. I know this because in LibreOffice it says that the document came from WPS editor, which is an Android editing suite. Perhaps there is some unefficient formatting within?

I am about to head out, but will try all this helpful ideas when I get back and can give them the attention they merit. I am quite excited about this!

If anyone is completely bored, I have added the original .odt file here, to peruse and critique (on style and use of Office tools alone, not content). I would very much appreciate some input on what I could do to improve things: *** Edit: Removed as served purpose ***

Thank you all so much! :D
Last edited by bedtime on Sat Jan 13, 2018 6:10 pm, edited 1 time in total.
LibreOffice
Version: 5.4.2.2
Build ID: 1:5.4.2-0ubuntu0.17.10.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3;
bedtime
 
Posts: 11
Joined: Mon Jan 09, 2017 9:13 pm

Re: Tip: Replace fancy font in huge docs for better speed...

Postby RoryOF » Wed Jan 10, 2018 3:27 pm

I have found that sometimes linux systems influence larger OpenOffice file. I'm thinking of one file I have, where the T(large) able of Contents takes two hours to rebuild, running on one existing Xubuntu 16.04.3; I have isolated this to the operating system, not to the installation of OO. On a fresh version of Xubuntu and OO, the ToC rebuilds in seconds. It might be worth changing your partition size so that you can install a fresh version of Ubuntu and LibreOffice alongside (as minimal customisation as possible), and seeing if that allows noticeably faster editing.
Apache OpenOffice 4.1.7 on Xubuntu 18.04.4 (mostly 64 bit version) and very infrequently on Win2K/XP
User avatar
RoryOF
Moderator
 
Posts: 31209
Joined: Sat Jan 31, 2009 9:30 pm
Location: Ireland

Re: Tip: Replace fancy font in huge docs for better speed...

Postby John_Ha » Wed Jan 10, 2018 5:14 pm

I downloaded Liber20-TestIndex.odt - it is 11.4 MB. It opened in about 17 seconds and I could immediately scroll although soffice.bin was still consuming about 30% of CPU, and was still consuming CPU many minutes after opening the file. I could immediately add characters without problems although it was sometimes slow. When I selected Web Layout AOO crashed. It has 715 pages and page 712 ends with "Protasis: a clause introduced by a conditional expression (if, when, whoever), leading to a conclusion called the Apodosis (§512)." When I closed Writer by clicking the X the title bar said AOO was not responding and it took many seconds for AOO to close, which it did successfully.

File > Properties shows the file has been open for editing for 684 hours and has been saved 1,408 times. This strongly suggests the file could be "tangled" so I created a new, empty text document and went Insert > File ..., and navigated to the file. I then saved it as Liber20-TestIndex after inserting.odt.

Liber20-TestIndex after inserting.odt was only 3.3 MB. Your file is so big because it has the fonts embedded in the .odt (is this a feature available in AndrOffice?) - mine does not. There is some reduction in the size of various components, suggesting some redundancy (tangles) in your file, but not much.

Clipboard03.gif

Liber20-TestIndex after inserting.odt also took about 17 seconds to open and soffice.bin continued to consume CPU. I could type as fast as my single finger :oops: would let me. It only has 562 pages where page 560 ends, as above, with "Protasis: a clause introduced by a conditional expression (if, when, whoever), leading to a conclusion called the Apodosis (§512)." suggesting all is complete. This is a link to Liber20-TestIndex after inserting.odt - please download the file and see if anything is missing.

I strongly suspect this is a "taking time to layout a complex file with many footnotes and many tables" problem as described in Re: [Solved] Compressing size of an odt containing large ima. If I add some text on, say, the first page with text, I cause the footnotes to spill ... which (somewhere) takes down text with a footnote ... which causes the footnotes to spill ... which releases space ... which allows the text to be pulled back ... which pulls back the footnote reference ... which pulls back the footnote ... which causes the text to spill ... and so on and on and on.

I suggest you try some of the ideas there to "help AOO to layout the file easily". In particular, insert some redundant page breaks and even redundant blank pages(every 30 pages? 50 pages?) to prevent a change on page 1 having to ripple all the way through to the end of the file - it stops rippling through when it arrives at a redundant page break or blank page which catches it. Delete right at the end.

Put all footnotes at the end instead of on the page they arise? That should prevent anything rippling through requiring the document to be laid out again.

Does removing the Table of Contents while editing help? If rippling adds a page, the entire document has to be relaid out to get the correct page numbers for all the ToC entries so they point to the new pages. Add it back when the document is finalised. The ToC is only updated when the user requests that it is updated.

Keep a careful watch to see if the number of pages changes due to "white space" being added, especially by "broken tables". Check for any tables which span a page break.

I think you have pasted photos into Writer which saves the photos as huge PNG files. Save them as JPG and do not paste them because they are then inserted as JPG files which are 10x smaller! See [Tutorial] Some useful hints on using images for a discussion on how best to handle images in Writer.

 Edit: It is a "layout / pagination / white space" problem where the total number of pages keeps changing.

When I open your file and scroll to the last page, I see white gaps and the page count keeps reducing as I scroll. It started at 715 pages on opening and has now fallen to 693 pages. A few minutes later it has fallen to 691 pages. The CPU use is still high.

Updating the ToC has not completed after many tens of seconds. When it finally completed after minutes the page count had fallen to 611. 
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK

Re: Tip: Replace fancy font in huge docs for better speed...

Postby John_Ha » Wed Jan 10, 2018 9:10 pm

I think the file is somehow also corrupted. Format > Sections ..., is greyed out. Correct - there are no sections. There are some hidden tables I cannot bring back. The file is very slow to save. The Navigator pane keeps flashing and when I cut the file into a FRONT and a back, both parts still flash, suggesting there is a problem in both halves. One table was inserted in a header - I deleted it.

It could just be that the file is large and complex - there are over 1,000 tables and many, many footnotes.
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK

Re: Tip: Replace fancy font in huge docs for better speed...

Postby bedtime » Wed Jan 10, 2018 10:15 pm

Sorry for the long post; I wanted to address all comments...


RoryOF wrote:I have found that sometimes linux systems influence larger OpenOffice file. I'm thinking of one file I have, where the T(large) able of Contents takes two hours to rebuild, running on one existing Xubuntu 16.04.3; I have isolated this to the operating system, not to the installation of OO. On a fresh version of Xubuntu and OO, the ToC rebuilds in seconds. It might be worth changing your partition size so that you can install a fresh version of Ubuntu and LibreOffice alongside (as minimal customisation as possible), and seeing if that allows noticeably faster editing.

I was using Ubuntu KDE edition and then I tried it in Gnome and found Gnome was faster for me. They are on two separate partitions.

File > Properties shows the file has been open for editing for 684 hours and has been saved 1,408 times. This strongly suggests the file could be "tangled" so I created a new, empty text document and went Insert > File ..., and navigated to the file. I then saved it as Liber20-TestIndex after inserting.odt.

I could not find 'File' under insert, nor even on the toolbar or in the toolbar settings. I did however find 'Document' and tried that. It actually increased the file size to 27+MB.

John_Ha wrote:I downloaded Liber20-TestIndex.odt - it is 11.4 MB. It opened in about 17 seconds and I could immediately scroll although soffice.bin was still consuming about 30% of CPU, and was still consuming CPU many minutes after opening the file. I could immediately add characters without problems although it was sometimes slow.

Then I tried something else in your post which mentioned just straight selecting 'all' and pasting into a new doc. Aside from one or two styles requiring minimal changes and a new first page style needing to be made and reinserting pics (a 10 min job, at most, altogether), it was perfect!

The file size went from 11.7 MB to ... wait for it...

... wait for it...

1.6 MB!!! :P

When I selected Web Layout AOO crashed. It has 715 pages and page 712 ends with "Protasis: a clause introduced by a conditional expression (if, when, whoever), leading to a conclusion called the Apodosis (§512)." When I closed Writer by clicking the X the title bar said AOO was not responding and it took many seconds for AOO to close, which it did successfully

It doesn't crash for me, but it hangs for a long time every time I scroll.

Liber20-TestIndex after inserting.odt was only 3.3 MB. Your file is so big because it has the fonts embedded in the .odt (is this a feature available in AndrOffice?) - mine does not. There is some reduction in the size of various components, suggesting some redundancy (tangles) in your file, but not much.

I'm not sure. But now I have only 3 fonts in the file. I remember trying several different ones, so there could have been dozens within it.

Clipboard03.gif

Liber20-TestIndex after inserting.odt also took about 17 seconds to open and soffice.bin continued to consume CPU. I could type as fast as my single finger :oops: would let me. It only has 562 pages where page 560 ends, as above, with "Protasis: a clause introduced by a conditional expression (if, when, whoever), leading to a conclusion called the Apodosis (§512)." suggesting all is complete. This is a link to Liber20-TestIndex after inserting.odt - please download the file and see if anything is missing.

I gave it a look and it appears to be all there. One thing I noticed is that your version included all my custom styles (even the ones I was not using). I believe my other method only included the styles that were applied and thus a smaller file. I stuck with my file as I no longer need those other styles.

I strongly suspect this is a "taking time to layout a complex file with many footnotes and many tables" problem as described in Re: [Solved] Compressing size of an odt containing large ima. If I add some text on, say, the first page with text, I cause the footnotes to spill ... which (somewhere) takes down text with a footnote ... which causes the footnotes to spill ... which releases space ... which allows the text to be pulled back ... which pulls back the footnote reference ... which pulls back the footnote ... which causes the text to spill ... and so on and on and on.
I suggest you try some of the ideas there to "help AOO to layout the file easily". In particular, insert some redundant page breaks and even redundant blank pages(every 30 pages? 50 pages?) to prevent a change on page 1 having to ripple all the way through to the end of the file - it stops rippling through when it arrives at a redundant page break or blank page which catches it. Delete right at the end.

Done. I inserted a page break before each new chapter (so 54 page breaks out of 650+ pages). The page break is a custom styled page, so I can tinker with it a little.

Put all footnotes at the end instead of on the page they arise? That should prevent anything rippling through requiring the document to be laid out again.

It looks like that one tip may have stopped my CPU usage from sticking at 100%. Now LibreOffice does not even register when idle. :D

Does removing the Table of Contents while editing help? If rippling adds a page, the entire document has to be relaid out to get the correct page numbers for all the ToC entries so they point to the new pages. Add it back when the document is finalised. The ToC is only updated when the user requests that it is updated.

At this point it is running so well that I need not remove it. I did add a custom page break right after it as well.

Keep a careful watch to see if the number of pages changes due to "white space" being added, especially by "broken tables". Check for any tables which span a page break.

I'm not sure how to do this, though I am interested if you have a link I could read. I wouldn't really know where to start.

I think you have pasted photos into Writer which saves the photos as huge PNG files. Save them as JPG and do not paste them because they are then inserted as JPG files which are 10x smaller! See [Tutorial] Some useful hints on using images for a discussion on how best to handle images in Writer.

Yes, there are two png files, but they are very small. Small enough that they are not shrunken down much at all. I used insert -> image to insert them.

I think the file is somehow also corrupted. Format > Sections ..., is greyed out. Correct - there are no sections. There are some hidden tables I cannot bring back. The file is very slow to save. The Navigator pane keeps flashing and when I cut the file into a FRONT and a back, both parts still flash, suggesting there is a problem in both halves. One table was inserted in a header - I deleted it.

Sadly, I had tried to use tables in both .doc and other formats, and in multiple editors, before saving as an .ODT, so that would make sense. :?

When I open your file and scroll to the last page, I see white gaps and the page count keeps reducing as I scroll. It started at 715 pages on opening and has now fallen to 693 pages. A few minutes later it has fallen to 691 pages. The CPU use is still high.

Updating the ToC has not completed after many tens of seconds. When it finally completed after minutes the page count had fallen to 611.[/edit]

With the aforementioned tips having been implemented, I can now open the file and scroll in 10 seconds (but the scrolling slogs a little), and after 30 seconds the cpu usage for the soffice task drops to nil (from 100%). Now takes 15 seconds to 'Update All'. And less than 10 seconds to save.

So, it's still not solved yet though. Not even close: we have lots of work to do!—it needs to be faster.




Just kidding! :lol:

Thank guys. I appreciated the time you all took and the info of the times of loading, links, and other such... It was very helpful. Right now, after 4 hours of intense LibreOffice formatting, I'm going to relax. 8-)

If anyone is curious to see the new file and comment with any new tips, here it is: *** Edit: Removed as served purpse ***
Last edited by bedtime on Sat Jan 13, 2018 6:10 pm, edited 1 time in total.
LibreOffice
Version: 5.4.2.2
Build ID: 1:5.4.2-0ubuntu0.17.10.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3;
bedtime
 
Posts: 11
Joined: Mon Jan 09, 2017 9:13 pm

Re: [Solved] Replace fancy font in huge docs for better spee

Postby John_Ha » Thu Jan 11, 2018 11:20 am

The document properties can be seen by going File > Properties ...

Monitor the page count at bottom left - you will see it vary.

Clipboard01.gif

Insert > File ..., and Copy everything > Paste ..., are similar. I don't know the difference but sometimes one works better than the other. AOO seems to have a better "XML checker" when it reads a file than "XML checker" when it writes a file.

Your images are physically small, but the files are huge - the red flag is 1.7 MB. There is an (unused??) 0.9 MB image of Roman ? soldiers in the file.

Clipboard02.gif
Image files as stored in the .odt file

I will look at your 1.6 MB file later. It sounds as though Copy > Paste has removed the "tangles".

What do I mean by tangled? I don't know. But Imagine a sentence with 5 words. If you edit it so the second word is italic, the third is bold, the fourth is highlighted, you then highlight half of words 2 and 3, and then you select each word individually and set it back to default. I would expect the XML to be more complex than if you highlighted the whole sentence and set it back to default.

A thought. Could you edit your Subject: to Large file very slow to respond and CPU use is high - we have discovered it was nothing to do with fonts which may mislead others searching!
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK

Re: [Solved] Replace fancy font in huge docs for better spee

Postby bedtime » Thu Jan 11, 2018 1:23 pm

John_Ha wrote:The document properties can be seen by going File > Properties ...

Monitor the page count at bottom left - you will see it vary.

I do have it set to 'insert blank pages'. Could that be the reason?

Your images are physically small, but the files are huge - the red flag is 1.7 MB. There is an (unused??) 0.9 MB image of Roman ? soldiers in the file.

How are you able to see that? I tried renaming the .odt file to .zip and opening, and I didn't see it. Though, indeed, I have used such a picture.

New file is here: *** Edit: Removed as served purpse ***


Amazing how much faster it is. Literally, instant, after it loads fully. Scrolling through contents is fast and smooth now.
Last edited by bedtime on Sat Jan 13, 2018 6:11 pm, edited 1 time in total.
LibreOffice
Version: 5.4.2.2
Build ID: 1:5.4.2-0ubuntu0.17.10.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3;
bedtime
 
Posts: 11
Joined: Mon Jan 09, 2017 9:13 pm

Re: [Solved] Large file very slow to respond and CPU use is

Postby John_Ha » Thu Jan 11, 2018 9:19 pm

Blank pages: If you are using Left and Right page styles and you insert a Left page after a Left page, AOO inserts a blank Right page if you say yes to blank pages.

Images: Look inside \Pictures folder in the unzipped .odt file. Or copy the image out of the document and paste it into an image processor. Then save it as a JPG (if a photo or similar) or as a PNG or GIF (if a graphic). Insert the saved image file back into the document.

Endnotes to end of document (and redundant page breaks): I think that was the key change - it prevents this repetitive layout problem. You have over 3,000 footnotes - it was causing chaos. Even so, look how long it takes for the page count to stabilise after opening the file - minutes rather than seconds.

File smaller after Copy and Paste: When I did it, content.xml isn't that much smaller than before. However, the images were not copied which is where the big change in file size occurred. styles.xml was also a lot smaller.
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK

Re: [Solved] Large file very slow to respond and CPU use is

Postby bedtime » Wed Jan 17, 2018 12:28 am

I've done some further tests and found the following:

Whenever I enter a character the program seems to do a little 'thinking.' This can be seen by looking at the CPU levels. A thread called soffice.bin will go to 100% on one core for a moment or so. I am still able to enter info, but things slow down a little. I'll call this time character entry, so I can easily refer to it later.

Inserting extra page breaks had no effect on speed (file opening, character entry, or update all). I imagine the breaks I had before sufficed, and I had many.

Having footnotes at the end of the document sped up document opening time from 90s to about 30s. Character entry sped up from 9s to 7s. Update all, 18s from 18s+.

The big change was adding sections. I divided the document into 8 sections. I then had only one section appear; the rest I chose to 'hide.' Now the program only sees 130 pages intead of the 600+. I added back same-page footnotesfor this test. The results were: file opening down to 12s. File save 9s. Character entry 2s. Update all, 2-3s.

And finally, to be fair, I had to try a test for the combination of both adding sections + end of document footnotes: File open 11s. File save 9s. Character entry 1s. Update all 2s.

All tasks were measured via CPU levels.

Hope someone finds this interesting. I know that I was pleased with this new info. :D
LibreOffice
Version: 5.4.2.2
Build ID: 1:5.4.2-0ubuntu0.17.10.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3;
bedtime
 
Posts: 11
Joined: Mon Jan 09, 2017 9:13 pm

Re: [Solved] Large file very slow to respond and CPU use is

Postby John_Ha » Sat Feb 01, 2020 1:57 am

I have been working on a large file with 2.5 million words, 700+ footnotes - it is 8,000 pages of double space text with no page breaks. It seems that AOO on Windows, AOO on Linux and LO all handle large files slightly differently.

Using LO 6.3.4.2 on W10 on desktop (Pentium dual core 2.6 GHz)

When opening the file, it took about 1 minute to open and once open I could edit it - it was quite responsive.

However, LO was working feverishly in the background for the next 10 or 15 minutes laying out all the pages (it needs to do so so that it can put page numbers into an Table of Contents). During this time the CPU use was high - about 50% - and the page count (total number of pages) was too high - it was 9,241 instead of 8,121. When I tried to scroll to the bottom of the file LO would often become unresponsive and I would get the spinner for many seconds.

After 10 or 15 minutes the CPU use fell to zero and the page count was correct. The file was now very responsive.

Using AOO 4.1.7 on W10 on laptop (i3-4010U dual core 1.7 GHz)

The file opened a little quicker and was immediately responsive. AOO was only using about 25% of CPU in the background. I could immediately scroll through the entire document without any stalling as in LO. However, AOO was still laying out the document in the background.

Using AOO 4.1.7 under Linux (i302100 quad core)

The behaviour was quite different. The file took 3 minutes 30 sec to open during which AOO was unresponsive.

It seems that AOO under Linux does the layout and pagination in the foreground and the user is locked out while this is happening.

Changing the page size from Letter to A4, which forces the document to be laid out again, caused the user to be locked out for 120 minutes and counting. (Thanks to RoryoF for this test)

So, if you are working with a large file be aware that AOO or LO is probably working hard in the background laying out all the pages but, once that has been done, the file will be very responsive. Until, that is, you make an edit which causes the pages to be laid out again ...
LO 6.4.4.2, Windows 10 Home 64 bit

See the Writer Guide, the Writer FAQ, the Writer Tutorials and Writer for students.

Remember: Always save your Writer files as .odt files. - see here for the many reasons why.
John_Ha
Volunteer
 
Posts: 7760
Joined: Fri Sep 18, 2009 5:51 pm
Location: UK


Return to Writer

Who is online

Users browsing this forum: Google [Bot] and 12 guests