AOO 4.1.4 chokes on opening 24MB .odt file

[Tatha]

Hello -

Rather odd problem -

1 - I have a 24MB plain vanilla text only .odt file that suddenly has decided NOT to open in AOO in Lubuntu 16.04 LTS.

Meanwhile,

2 - The same file opens just fine in other_profiles_on_same_OS as well as in Windows 7 so file corruption is not to blame.

3 - Other .odt files will open, however, but they are much smaller in size.

4 - I have uninstalled and then reinstalled AOO with zero effect. Same end result.

At my wit's end so any intelligent thoughts on the matter would be appreciated.

[FJCC]

24MB is huge for a text-only file. I have a 260 page file with some images that is only 4.7MB. Unless the file has a great many pages, there is something wrong with its structure. First, make a back up copy. Change the file name extension from odt to zip and open the zip archive. The largest file in the archive should be content.xml. Is that true and how big is it?

[Tatha]

24MB is huge for a text-only file. I have a 260 page file with some images that is only 4.7MB. Unless the file has a great many pages, there is something wrong with its structure. First, make a back up copy. Change the file name extension from odt to zip and open the zip archive. The largest file in the archive should be content.xml. Is that true and how big is it?

Hi - thx for replying.

Yes, the file in question is very large and comprised of 1300+ pages.

Screencap of the zip archive is attached. as you can see, content.xml is only 6.6MB.

Additionally, I have already tried resetting the user profile to no avail. Exact same outcome - AOO opens, but hangs/chokes on the file. The exact same file, however, has no problems opening in another OS profile on the same or different OS.

I'm wondering if my user profile on the OS itself is somehow to blame since the file will open in the same OS but from another user account.

[robleyd]

I note you have a directory Pictures in your image which contains nearly 24MB of data - if this is a text only document, one would not expect any content in that directory.

It might be worth resetting your user profile, and I would also investigate the Pictures directory.

[Tatha]

I note you have a directory Pictures in your image which contains nearly 24MB of data - if this is a text only document, one would not expect any content in that directory.

It might be worth resetting your user profile, and I would also investigate the Pictures directory.

1 - Resetting user profile in AOO was worthless. Same result.

2 - Pics directory deleted from zip.

3 - renamed .zip back to .odt

4 - AOO takes up to 5 minutes to open and allow navigation of the file.

5 - if images have been 100% deleted, then why does an exclusively ALL_TEXT .odt file take up to 5 minutes to open?

Suffice to say, I'm still seeking informed opinions on this topic.

I did attempt to report it as a bug however it was marked as a dupe and presence of images was also blamed by the bug board devs, but after extensive troubleshooting and having deleted the images file, presence of images in the file in question does not appear to be the problem.

6 - Reminder: SAME EXACT (INTACT with images) FILE OPENS IN DIFFERENT OS AND/OR WITH DIFFERENT USER ACCT ON SAME OS.

There is obviously some kind of corruption somewhere however, since the file in question opens without issue elsewhere in other OS and other user acct on same OS (with images present) solutions seem to have been exhausted.

OUTCOMES:

A - It seems to be that I cannot work on the file in question in my primary user acct on my primary OS.
B - AOO 4.1.4 is limited in the number of pages/size of .odt file that it will be able open successfully. So AOO users take note: AOO will choke on plain vanilla text .odt files at approximately 24MB and/or 1300 pages.


Since LibreOffice is also based on the same code as AOO, I'm wondering if this bug has carried over to LO Writer as well.

[FJCC]

I don't know for sure, thus failing your "informed opinion" criterion, but merely deleting the Pictures folder may result in a corrupted file. The presence of a manifest file makes me think that the contents of the archive are being tracked and manually deleting a folder will cause problems.
I made a file with 1667 pages of text. It is 84 KB on the disk. Opening the zip file, the content.xml is 53 KB compressed and 8036 KB uncompressed. Your content.xml is shown to be 6.6MB, which matches pretty well with the ratio of the page counts.
When I open my 1667 page file it takes only a few seconds, maybe 5s.

The most important fact, however, is that the file has a serious corruption. It may be opening on most systems at the moment but I strongly suggest you make a file with a reasonable size right away. If it is all text, a simple Edit -> Select All, Edit -> Copy and then Edit -> Paste into a blank document should do it. If that new document is also huge, we'll have to look for the source of the corruption.

[Tatha]

I don't know for sure, thus failing your "informed opinion" criterion, but merely deleting the Pictures folder may result in a corrupted file. The presence of a manifest file makes me think that the contents of the archive are being tracked and manually deleting a folder will cause problems.
I made a file with 1667 pages of text. It is 84 KB on the disk. Opening the zip file, the content.xml is 53 KB compressed and 8036 KB uncompressed. Your content.xml is shown to be 6.6MB, which matches pretty well with the ratio of the page counts.
When I open my 1667 page file it takes only a few seconds, maybe 5s.

The most important fact, however, is that the file has a serious corruption. It may be opening on most systems at the moment but I strongly suggest you make a file with a reasonable size right away. If it is all text, a simple Edit -> Select All, Edit -> Copy and then Edit -> Paste into a blank document should do it. If that new document is also huge, we'll have to look for the source of the corruption.

Eh, thx for your thoughts. I'm open to almost anything at this point on this particular matter.

I have resolved to split the file in question into smaller chunks using a command line/terminal 'Nux utility as that seems to be the safest option since a file corruption loss would be catastrophic.

I dislike using MS Word and AOO does not seem to be the industrial strength level word processor that I need. I'd prefer to continue using free open source software rather than being chained to proprietary crap like Word.

[RoryOF]

I agree with FJCC: copy/paste the content into a new .odt file. Allow it time to open correctly before the Copy/Paste. Use Task Manager to be certain that OO has finished allocating memory - this may take 60 minutes for such troublesome files. I regularly edit 4000 page files, with several million words, but response can be slow. Any trace of earlier MS Office editing/saving can cause difficulties.

[RoryOF]

My experience with large (book text formatted, no illustrations) OO files is that earlier versions of OO such as v2 and v3 could handle larger files; I note that I was able to build a 114918 page file, also a 61779 page file with one or other of these (sorry, I can't remember which); these files cannot be opened using OO 4. I stress that the files responded so slowly they were not effectively editable, but given time they did respond.

I suspect there has been some change in the memory handling of OO in the later version(s), which I haven't had time to enquire about.

Edit: A 57643 page A5/US Brochure text file of just under 18 million words has taken just 60 minutes to load on a reasonably specified 4 core computer with 8GB memory; it has consumed 7,3 GB of memory and started up the swap file. It is effectively uneditable as response is so slow (glacial).

[John_Ha]

The problem is the 24 MB Pictures folder. I expect you have pasted a photo(s) into the document. Fix by Tools > Options > OpenOffice Writer > View ..., and un-tick Display > Graphics and objects. You should now be able to edit the file.

Or open the Pictures folder. Remove any images and resample them to much smaller. Replace them without changing their names.

If you don't want any images open Navigator by pressing F11. How many images does it say you have? Delete them by r-click > delete.

Clipboard01.png (21.76 KiB) Viewed 2650 times

See [Tutorial] Some useful hints on using images for a discussion on how best to handle images in Writer.

If file is still a problem, create a new, empty text file by File > New > Text document. Go Insert > File ..., and pull in the old file. Save. Is the new file better? If not, upload it to a file sharing site so someone can look at it.

If your problem is solved please view your first post in this thread and click the Edit button (top right in the post) and add [Solved] in front of the subject.

[John_Ha]

... and, if all else fails, extract content.XML and edit it with Notepad++. Go Search > Replace ..., with search argument <[^>]+> and replace argument is blank. Be sure to tick Regular Expressions. Click Replace All.

This will delete all the XML tags and give you all your text but without all the formatting, tables etc. See Item 3 in [Tutorial] How to fix SAXParse errors in LibreOffice files for a discussion.

[RoryOF]

A quick thought, which I haven't explored: open a copy of the file, extract content.xml and run an XML optimiser on it. Then replace the optimised content.xml into the OO archive and try opening that. I've just examined a large file (1.8 million words) and its XML is peppered with redundant XML formatting tags. Removing such redundant tags may help speed the file opening.

[Tatha]

A quick thought, which I haven't explored: open a copy of the file, extract content.xml and run an XML optimiser on it. Then replace the optimised content.xml into the OO archive and try opening that. I've just examined a large file (1.8 million words) and its XML is peppered with redundant XML formatting tags. Removing such redundant tags may help speed the file opening.

Excellent suggestion. Thank you. Will report back to this thread with the outcome.

[Tatha]

The problem is the 24 MB Pictures folder. I expect you have pasted a photo(s) into the document. Fix by Tools > Options > OpenOffice Writer > View ..., and un-tick Display > Graphics and objects. You should now be able to edit the file.

Or open the Pictures folder. Remove any images and resample them to much smaller. Replace them without changing their names.

If you don't want any images open Navigator by pressing F11. How many images does it say you have? Delete them by r-click > delete.

Clipboard01.png

See [Tutorial] Some useful hints on using images for a discussion on how best to handle images in Writer.

If file is still a problem, create a new, empty text file by File > New > Text document. Go Insert > File ..., and pull in the old file. Save. Is the new file better? If not, upload it to a file sharing site so someone can look at it.

If your problem is solved please view your first post in this thread and click the Edit button (top right in the post) and add [Solved] in front of the subject.

Thanks for this. Another great suggestion.

[Tatha]

I don't consider the matter closed just yet so I won't mark this thread as resolved until I troubleshoot the file with the above referenced suggestions. Ideally, I would like to have an outcome and document it here for anyone who finds himself in the same position I am with a large file. When you have a 1300-1400 page file that you've worked on for hours upon hours, and, you suddenly find that your word processor no longer will open it, it's not a very nice feeling. I expect to work on this matter in the short term and post immediately thereafter with the results.

OUTCOME: In the interrim, I had to split the file in question manually into smaller chunks as no command line based 'Nux utility or otherwise would handle the job without corrupting the file.

[RoryOF]

Having broken the file into pieces, you could reassemble them into an OO Master Document. Working out how to do this will give you something to think about over the holidays!

[John_Ha]

When you have a 1300-1400 page file that you've worked on for hours upon hours, and, you suddenly find that your word processor no longer will open it, it's not a very nice feeling.

I hear you. Splitting it into bits is very sensible as it will load faster and reduce the risk of losing everything.

I trust you are backing it up safely. When I was writing large reports I would zip the folder containing the files and name it Report_Fred_ 25 Dec 2016.ZIP. I could therefore go back and create the document as it was on any day in the past.

RoryOF's suggestion of using a Master Document is an excellent one. Use one sub-document per chapter.

[Tatha]

Having broken the file into pieces, you could reassemble them into an OO Master Document. Working out how to do this will give you something to think about over the holidays!

Hmm, food for thought. I hadn't considered this before. May be well worth it for future endeavors.

[John_Ha]

If you have been editing the file a lot, as you no doubt have, it is well worth while to copy the contents to a new file because heavily edited files seem to get "tangled". Create a new, empty text file and go Insert > File > navigate to the file and insert it. Save the new file. See [Solved] Compressing size of an odt containing large images which says:

I'm amazed at the improvement of the second file after I saved the contents of the original into a new file ('removing the tangles'). Just doing that has made it tolerable to work with ...