Another Hash-Filled Document

[TheBoyFromNorfolk]

I just lost a script I was working on. I've had a quick look through the results in the search and just spent a few hours trawling through .tmp files. No luck, but I did find some that seemed promising... only for them to also be Hash tagged to death.

Obviously no one wants to lose a document, but as far as I can see, this one has a larger file size than an empty file... is that just the dreaded hastags?

I've tried Recuva and recovery for Writers Demos, and I would really like to save, if possible, this work, instead of having to rewrite a dozen scripts.
Thanks

Adam

[robleyd]

Sorry but that file is full of null bytes - there is no, repeat no, data in it at all.

Have you tried "Previous Versions" in Windows, as mentioned in the tutorial?

[RoryOF]

What happened? Did OpenOffice crash, or the computer crash? Or did it just show as ### when you came to it again? This information is important, as the problem of the lost file in OO is being worked on.

[John_Ha]

First, there is no user data in your file. See Why is my file full of #####? for a discussion of files full of ######.

You can try the methods in [Tutorial] How to find and un-delete AOO temporary files to try and recover previous versions; still existing temporary or AutoSave files; and now deleted temporary or AutoSave files.

Can you please tell us in excruciating detail exactly what happened. Development is currently looking at how files are saved but no-one can see any place in which nulls get written to the disk. Questions I would ask you to think about include:

Did you do a normal save and you thought everything was normal but, when you next opened the file, it was full of #####?

Did you save the file and then quickly slam the laptop lid shut?

Did AOO hang? By hang I mean AOO stops responding but the window stays there.

Did AOO crash? By crash I mean the AOO window disappears completely.

Did the PC hang and need rebooting?

Did you hibernate the PC? or put it to sleep? or power it off?

[TheBoyFromNorfolk]

Did the PC hang and need rebooting?

Exactly that, I had too many windows open and the computer was unresponsive.

It's an older desktop and a getting a little iffy, so I thought I might lose a few paragraphs, a days writing at most. Instead I lost everything. Fortunately I have the core stroy saved elsewhere, but all my commentary is gone.

I have shut it down in this way many times before, but never had an issue. This time my son pressed the power button after I started the shut down, I think cutting the power before the shutdown completed.

Thansks for everyones help guys, with all this help it's been a comfort in trying to fix it, but it seems like it's gone, I can't find any relevent temporary files. Though I have found .tmp files that looked promising, but were also Hashtagged and null, even though their save dates were days, or weeks in the past.

[John_Ha]

I have done a quick forum survey of "files full of ####" where users have reported how it happened and one common thread is that the PC crashed while the document was open or was being saved.

Many years ago a bug report was raised suggesting crash or power loss was the cause - see Issue 107847 - File content changed to hashes at crash or power loss

I collected these where users described how it occurred:

viewtopic.php?f=7&t=92880

My computer crashed due to no electricity (without battery) but I thought saving the file was completed before the crash. I saved the edited odt more than once during the writing process but I could not find any previous versions. As I restarted the computer and tried to open the odt it was corrupted.

viewtopic.php?f=7&t=104676

I opened it and had been working on it for few hours, saving it every 10 minutes or so, when my computer froze and showed a grey screen. As this hadn't shifted despite my best efforts I had to do a forced shut down after about half an hour. When I restarted the computer it was all fine apart from the document I had open on the screen where the text had been replaced by ######
If it helps then the file name was just Writing

viewtopic.php?f=7&t=100754

while I was playing a video my lap just outright wnent nuts, even though the battery was fully charged, and my screen froze and I was unable to coninue playing the video. However, at the moment I was just saving this document of around 40 pages to my OneDrive cloud and I had to reboot my entire computer.
As I turned my laptop on again I openede immediatly Open Office, and weirdly enough it asked me to register my program online and choose a language for my program, and as I searched thourg recently opened files I found nothing, and now I have even trouble to efficiently run Word or Open Office.
And my document as I open it, it just reads "########", there are no words on it.

viewtopic.php?f=7&t=92880

My computer crashed due to no electricity (without battery) but I thought saving the file was completed before the crash. I saved the edited odt more than once during the writing process but I could not find any previous versions. As I restarted the computer and tried to open the odt it was corrupted.

viewtopic.php?f=7&t=90475

... three OO Writer documents ... created and saved on C: disk. ... I saved the updates (waited till savings are finished) but unfortunately have not closed all documents. ... I put my notebook in Standby status [overnight] to continue early morning finishing the documents. I just selected Standby and closed Laptop. Today early morning I opened laptop (screen) but could not start the notebook.

[ie PC shut down overnight]

So I had to shut down and newly start. ... as I tried to open the first document (I worked for 2 days on) it again opened with window message "ASCII Filter Options" and as I clicked further the documents was full of hashtags. Other 2 documents from yesterday I could open and they are Ok. Only the one document which is most important is not ok.

[Hagar Delest]

For the record, LibreOffice seems not to be subject to this issue. It may come from the code sanitizing they performed when they forked from AOO.

[John_Ha]

Hagar

I suspect so.

fyi development is now working to fix the AOO bug where AOO does not prevent a PC shutdown if the shutdown is issued while the .odt file is being written. This bug may be picked up in that work.

[Hagar Delest]

Yes, I've seen that.
So many years for that being tackled...

[John_Ha]

See also Damaged File

I had an odt file open (had saved since the last edit) when I lost power. This is Open Office 3.0, on Ubuntu 9.x. Open Office attempts to recover the file, but is unable to do so. Opening the file directly results in 3 pages of hash marks (#)

I thought acknak's comment was very interesting:

All modern OS's buffer the data written to a file and, even after the file is closed, the hardware may buffer it. The time is short--it should all be safely on the disk in a few seconds, but laptops can take a while for the hard disk to spin up, and if the power fails at just the wrong time, you can lose data even in a manually saved file.

It puts the direct cause on the OS, rather than on AOO file saving code, a possibility I wondered about earlier today. If so, AOO would presumably need to be changed so it works on a copy and not on the actual file on disk as suggested in Issue 111290.

Also see Issue 111290 - Better protection for file if crash occurs

If a crash occurs, it often happens that OOO can't recover the file. The file is damaged: zip is corrupted and/or content.xml size is 0.

LO may have made that change when the OOo code was cleaned up so that is why it doesn't happen on LO.

I am hopeful that the work to investigate AOO not preventing a shutdown while writing a file may fix the "my file is full of #####" too.

[John_Ha]

I have been digging and, while the internet is the largest source of information, one unfortunately has to make one's own judgement if the information is correct. I found Are file edits in Linux directly saved into disk? which says:

3 In practice, on a lightly loaded system, the file will hit the disk within a moment ... if your editor uses fsync() after writing the file. The Linux default for /proc/sys/vm/dirty_writeback_centisecs is 500 (5 seconds), and PowerTop recommends setting it to 1500 (15 seconds). (kernel.org/doc/Documentation/sysctl/vm.txt). On a lightly loaded system, the kernel will just let it sit dirty in the page-cache that long after write() before flushing to disk, to optimize for the case where it's deleted or modified again soon. – Peter Cordes Aug 24 '18 at 14:06

So, AOO writes the file into memory knowing that Linux will flush it to disk. But, by default, Linux runs the "flush to disk" command only every five seconds giving a huge window of opportunity for the file not to be flushed to disk following an abrupt OS crash or a loss of power. No doubt other OS are similar.

So, if AOO edits MyDocuments\fred.odt and if AOO overwrites MyDocuments\fred.odt, there is an opportunity for that overwrite not to be properly or fully flushed to disk. AOO thinks it has done everything properly but, because the AOO design is to overwrite the existing file instead of "writing a new file and deleting the original when the new one has been written", there is an opportunity for the OS to trash the file on disk.

So the root cause is because AOO works in this risky way and the method should be changed. The direct cause is the OS not properly or fully flushing the data following an OS crash or a power outage.