Images Randomly Delete from File

[kepardue]

We have a number of large documents that we routinely send to the printer, most of them with images embedded throughout. Occasionally, when opening one of the files a random image will have a [Read Error] issue, which many times we won't notice since we're editing a different part of the document, save, and lose the image forever. It happens about once a month and is really coming down on me since I'm the one that recommended to my boss that we use an open standards solution. We never had problems while using the original, unedited Word variants of these files.

We were accessing our documents across a NAS. But I changed the storage to Dropbox hoping that it was just a network issue that the image file hadn't fully loaded before the text appeared. After seeing the issue a number of times, I can only assume that it has not. We do operate in a heterogeneous environment. Macs, PC's, and Linux computers.

I cannot recreate the issue, and looking at the source to the file it looks as though there was never an image there at all.

I've donated at least as much as two MS Office licenses to the then-Sun OpenOffice program over the last few years. It's incredibly frustrating to see that these open standards don't seem to be interoperable with OpenOffice by itself, much less with anything else.

[Hagar Delest]

Hi and welcome to the forum!

Do you confirm that you save in ODF?

Personally, the reason why I switched to OOo 4 years ago is that MS Word did the same for the pictures of one of my documents.

Have you tried to open the file with an archive manager and check if the pic was still in the /pictures subfolder?

[acknak]

There must be some problem, but until someone finds a way to reproduce it, the developers aren't likely to spend any time on it.

Here's another recent thread on this problem: viewtopic.php?f=10&t=31754

In that thread, the images were completely lost from the document and no longer in the document archive either.

You're welcome to discuss the problem here, just remember: we're merely users like you. There's no one on this forum that can do anything about fixing the problem.

[kepardue]

Yes, it is saved in ODF (1.2 Extended), and I also checked the archive. Not only does the reference appear to be gone from the content.xml file, but it also appears to be missing in the Pictures folder. I know the devs can't fix something if they can't break it, but this happens frequently enough to me that I can't imagine how someone hasn't figured out how to recreate it. At least it's happening to others. I'll follow the other thread and subscribe to it there, but this problem has been occuring frequently enough that we'll probably, reluctantly, convert our documents to OOXML and start using MS Office.

[RoryOF]

A Read Error would suggest that the file was not embedded, but linked, and that perhaps the link has failed. I vaguely remember reports concerning limitations on link lengths, which may vary from OS to OS. The figure of 48 or 64 characters sticks in my mind, because it was unrealistically short. I would suggest as a temporary expedient adopting this approach: set up a directory for each document, into which you move (before linking/embedding) every image and link or embed the images from the local file, not from the master. This may present difficulties if master images are updated, but hopefully that won't occur that often. Them you can send an image of that project directory to your printer. If this procedure works, you can be reasonably sure that the image losses you have been experiencing are due most likely to a linkage problem of some sort, and investigate that further.

[kepardue]

For reference, here's the markup for where the image *should* be contained:

<text:p text:style-name="Standard"/><text:p text:style-name="Standard"/><text:p text:style-name="Standard"><draw:frame draw:style-name="fr5" draw:name="Frame76" text:anchor-type="paragraph" svg:width="4.5783in" draw:z-index="20"><draw:text-box fo:min-height="3.9492in"><text:p text:style-name="Figure">Figure <text:sequence text:ref-name="refFigure8" text:name="Figure" text:formula="ooow:Figure+1" style:num-format="1">1-9</text:sequence>: Capacitor Bank</text:p><text:p text:style-name="Figure"/></draw:text-box></draw:frame></text:p>


Other images in the document, including the very next one, appear properly:

<text:p text:style-name="Standard"/><text:p text:style-name="Standard"/><text:p text:style-name="Standard"><draw:frame draw:style-name="fr5" draw:name="Frame77" text:anchor-type="paragraph" svg:width="5in" draw:z-index="21"><draw:text-box fo:min-height="0.1902in"><text:p text:style-name="Figure"><draw:frame draw:style-name="fr22" draw:name="graphics12" text:anchor-type="paragraph" svg:x="0in" svg:y="0.0165in" svg:width="4.5in" style:rel-width="100%" svg:height="3.3819in" style:rel-height="scale" draw:z-index="22"><draw:image xlink:href="Pictures/100000000000012C000000E16A181D27.png" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/></draw:frame>Figure <text:sequence text:ref-name="refFigure9" text:name="Figure" text:formula="ooow:Figure+1" style:num-format="1">1-10</text:sequence>: Shunt Reactor</text:p></draw:text-box></draw:frame></text:p>

[kepardue]

The images are embedded and not linked. All of the other images (including the one that went awry, as evidenced from a backup from about a year ago) appear in the Pictures folder. The markup from the backed up file is below:

<text:p text:style-name="Standard"/><text:p text:style-name="Standard"><draw:frame draw:style-name="fr5" draw:name="Frame76" text:anchor-type="paragraph" svg:width="3.5772in" draw:z-index="20"><draw:text-box fo:min-height="4.3102in"><text:p text:style-name="Figure"><draw:frame draw:style-name="fr18" draw:name="graphics11" text:anchor-type="paragraph" svg:x="0in" svg:y="0in" svg:width="3.4465in" style:rel-width="100%" svg:height="4.0409in" style:rel-height="scale" draw:z-index="21"><draw:image xlink:href="Pictures/10000000000001220000015453E734F0.png" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/></draw:frame>Figure <text:sequence text:ref-name="refFigure8" text:name="Figure" text:formula="ooow:Figure+1" style:num-format="1">1-9</text:sequence>: Capacitor Bank</text:p></draw:text-box></draw:frame></text:p>

The funny thing is that images appear fine for a time. It's only seemingly randomly that they disappear.

[RoryOF]

Does this happen on multiple machines, or only on the one? Have you adjusted the memory settings under Tools / Options / OpenOffice.org /Memory to be as large as possible? When the file is opening or saving, are there other programs running?

[kepardue]

It happens on multiple machines. I have my own settings at:

Number of Steps: 30
Use for OpenOffice.org: 128
Memory per Object: 20.0
Remove from Memory after 00:10
Number of Objects 20

Should these settings be bumped up higher? One of the machines that we're editing on is fairly low power. Using it as a least-common denominator, I'm not sure I could bump those settings up super high.

A 3d game I understand, but should I really be running into issues like this, serious enough to cause data loss, with... a word processor?

[kepardue]

Okay, I seem to have caught it in the act in one of my documents. It's saying "Read Error." If I were to save the file, the next time I open it the image would be gone from both the document display and the package. Is there any kind of troubleshooting that I can do while this document is open?

When I open the file in a different program (Symphony) before saving it, the image appears just fine. I'm sure if I do not save this file and re-opened it in OpenOffice, the image would appear just fine.

[acknak]

I'm sure if I do not save this file and re-opened it in OpenOffice, the image would appear just fine.

And therein is the key to why this isn't getting any attention.

I don't know of any thing you can do after it happens to get any further information. I think the developers would have to take steps ahead of time to find the actual cause of the read error.

Maybe, if you did exactly what you said: close the file and re-open it and take a screen shot showing that the image is ok, then you could submit the two shots as "proof" that there is a problem. I can't see any way that an image embedded in the document can get a valid "read error" when the document archive is intact and stored as a local file on a hard drive.

I suppose you could argue that it could be caused by something external to OOo interfering somehow, but that's really stretching it.

[RoryOF]

I've carried out a quick search. It seems that the maximum number of files any process can have open in OSX is 256. This number will be slightly reduced to allow for STDIN, STDOUT and some other channels. In XP, it seems to be 512, unless patches are applied. I don't know why there is a Read Error for an embedded file, unless the "embedding" means local storage of the image in an internal file of all images or as separate internal files of each image (i.e. "private" linking). Finding this out might tell me more than I need to know about the internals of OpenOffice

Edit: I would not wish to offer the above numbers and analysis as being certain sure. This was the result of a quick search, and I might be misinterpreting something fundamental.

[acknak]

Here is a list of the similar reports I found:
http://qa.openoffice.org/issues/buglist.cgi?resort=1&issue_id=18715,73281,107108,108035,108598

Although I understand their position, I find the developers comments rather disturbing: they completely refuse to acknowledge any problem unless it can be reproduced at will on their system(s). I think there is strong circumstantial evidence that there is some problem in OOo that's responsible for corrupting documents and causing lost work. That ought to be worth some effort to try and determine what's happening.

At the very least, a read error for an embedded image should be a fatal error. Of course, at that point, you have to choose whether the user loses the image or the other work done since the last save.

[kepardue]

I'm not sure if the number of objects matters in this case, because what I've just done is this:

1) Safe the file with the "Read Error" as a different file name.
2) Reopen the Original File in OpenOffice.

That would, presumably, double the number of open pictures since both files were open simultaneously. As expected, the image is back and present in the document (screenshot attached).

Also, this does happen on Windows as well as Linux, so I don't think it's platform specific.

[RoryOF]

Purely to satisfy me, would you please do the following:
Copy a master copy of a file that "went bad", at system level.
Now rename the copy to type .zip and open it with an archive manager. In the internal directory Pictures, how many pictures are there? in most archive managers typing Ctrl A in that directory will select all the contents and the status bar at the bottom wiill say "X Mb in y files" which saves having to count them.

[Hagar Delest]

Had the file been opened for a long time? Perhaps there is a cleaning application that could mess up with the temporary files. What you can try is to change the OOo temporary folder (set in Tools>Preferences>OOo>Paths) to an area in your documents folders instead of the system folder.

[kepardue]

In the document that was saved AFTER the error: 43.2MB, 53 items in the Pictures folder.

In the document BEFORE the error: 69.2MB, 75 items in the Pictures folder.

Looking through that same document, I have found a few other images that are missing, but I don't think it's as many as the 22 listed there.

[kepardue]

The file had been open maybe an hour. The unfortunate thing is that I don't know if the images were gone when thew file opened or just somehow disappeared after that.

I can try changign the Temporary Path, but would also have to change it on the Windows and Linux machines as well, since the problem doesn't seem common to one system, nor does it seem to happen all that often.

[Hagar Delest]

Then, it won't change anything I guess. Not really worth a try.

Do you caption all the pics? Personally, I never use that feature, I put the pics in tables. Perhaps frames are less reliable.

[kepardue]

Yes, all the pics are captioned. We have a figure index at the beginning of the book. I have noted that whenever the file is exported as a .doc, the resulting file has some frames, and is missing others. Really strange.

I agree with acknak, sure do wish this would signal at least some form of investigation on the part of the devs, even if there's no consistent way to recreate it.

[RoryOF]

In the document that was saved AFTER the error: 43.2MB, 53 items in the Pictures folder.

In the document BEFORE the error: 69.2MB, 75 items in the Pictures folder.

Looking through that same document, I have found a few other images that are missing, but I don't think it's as many as the 22 listed there.

Thanks. I was hoping it might be approaching 250+, which might be indicative of the files limit being to blame. Are other programs running concurrently? As Acknak says, there certainly seems to be a file save problem in OpenOffice; possibly one component module in the Save unit is not as robust as it should be, which would explain many other save failures (corrupted files).

[kepardue]

Are you sure it would be a save problem and not some kind of problem with opening the file? Remember that the "faulty" file was the same file as the "good" file. It's just that when I opened it, somehow some of the images disappeared. Saving it only seems to have sealed the deal, as it were.

Yes, other programs are running. A web browser, Mail.app, iTunes, all of them using less memory than OpenOffice. Other apps that are running (Photoshop, Symphony, Illustrator) were only opened up AFTER I saved the faulty file out. OpenOffice is curringly using 913MB in RAM (wow), 774MB Virtual Memory (wow x2), and 23 threads. The system I'm editing on has 4GB of RAM.

[acknak]

The same behavior has been reported on all platforms and--judging from the other recent discussion, linked above--it seems to affect (at least) Impress files as well, if the behavior is coming from the same cause.

When it happens, the images are completely lost from the document archive and the xml.

That's enough to convince me that this is worth investing some time to track down, whether it's a pure code bug or something else. I don't remember ever losing work using OOo, but having seen the reports here, and seen some of the corrupted data, I am convinced there are still problems lingering in the OOo code.

If I were working on a large document with many figures, I would keep all the images as links, rather than embedding them.

[RoryOF]

Are you sure it would be a save problem and not some kind of problem with opening the file? Remember that the "faulty" file was the same file as the "good" file. It's just that when I opened it, somehow some of the images disappeared. Saving it only seems to have sealed the deal, as it were.

Yes, other programs are running. A web browser, Mail.app, iTunes, all of them using less memory than OpenOffice. Other apps that are running (Photoshop, Symphony, Illustrator) were only opened up AFTER I saved the faulty file out. OpenOffice is curringly using 913MB in RAM (wow), 774MB Virtual Memory (wow x2), and 23 threads. The system I'm editing on has 4GB of RAM.

Yes, it certainly looks like an Opening problem - I was instinctively thinking of a Closing problem because of the many reports of file corruption on closing. I have no file approaching your usage figures - my usual OpenOffice files are 160 or so pages, plain text with "book" formatting, frequently opened, edited and closed, or some Impress files mostly of full page captioned images (70/80), perhaps 36MB reducint to 4/5MB after optimisation. I (touch wood!) have never lost any file under OpenOffice, be it on Win2K or Ubuntu machines. Being old school, I run as few programs concurrently as possible. Email (Eudora) in the background, perhaps a browser (Firefox or Opera) and will open an image editor if one is needed, closing it after use. I don't use online music or videos. If I had to look for a culprit, I'd start with programs that demanded constant attention, such as audio or video programs.

[Hagar Delest]

Are you sure it would be a save problem and not some kind of problem with opening the file? Remember that the "faulty" file was the same file as the "good" file. It's just that when I opened it, somehow some of the images disappeared. Saving it only seems to have sealed the deal, as it were.

Good point!

I use many pics embedded in my documents and never had lost anything for the moment (but I never caption them in frames). Note that you can have the same behavior with tables instead of frames, just use the Number Range field to make the index afterward.

NB: I often have many applications running in parallel (Firefox, MS Excel, Lotus Notes & Powerdesk, quite all the time).

[kepardue]

Since it looks like my issue with OpenOffice is at a bit of a dead end since it cannot be recreated, I need to find an alternative. I suppose as a less drastic option from moving our documents to Word I can run Symphony for a while and see if the same issue occurs there. I suspect it will since it's OpenOffice-derived. I tried to convert one of our documents to MS Word today and... wow... just wow. I thought OpenOffice was frustratingly tedious. Word is much worse, at least Word for Mac.

I hate to move away from an open standard, especially one that I went to great lengths to advocate, but it's unacceptable to my boss that we check each of these documents to make sure the images are showing up before sending them to the printer, which we do often.

[RoryOF]

What would happen if you marked each file Read Only at system level, supposing it was intact?

[kepardue]

Well, that would keep the images from permanently deleting, but would come at the cost of sacrificing convenience for my boss when he occasionally goes in to edit the files. We'd also still have to scroll through each of the documents on open to check to see if all the pictures were in tact.

[kepardue]

It looks like it probably is a memory issue. I opened the OpenDocument file in Office 2007 (trying to figure out what is less lossy, going from ODT -> OOXML using Office 2007, or going from OpenOffice -> .doc -> OOXML using Office). Two of the images said they couldn't be displayed because hte computer didn't have enough memory. It was consistently the same two images. When I opened the same file in OpenOffice, both of the images displayed fine.

One thing I'm curious about is, why don't these programs generate relatively low quality images just for on-screen display, which would speed up the load time and performance of the document, and just save the higher res original for printing? Or, is that what it already does? I mean, certainly if I load up 75 images properly scaled in a web browser I don't see performance degredation.

[RoryOF]

OpenOffice Impress has available an extension, Sun Presentation Minimizer, which processes a presentation file, embeds all the images, and downsamples them to fit the target display device. This most useful of applications has reduced (in my case) an 80MB presentation to 4 MB, with no apparent loss of quality and an improvement in load time. The source code for this is available; it is my hope that some inspired person might take the code and adapt it to optimise a Writer document (but I'm not holding my breath...!)

Previous postings to this forum on earlier threads have suggested that there is little or no quality improvement in inserting continuous tone images at resolutions higher than about 150-200 dpi, and that these should be carefully cropped to size before insertion. Line illustrations benefit from higher resolutions; some practical experimentation using the eventual output device might be advantageous in reducing unnecessary resolution, yielding a dramatic decrease in size.