[py] WrappedTargetException accessing modules in Basic libra

[probe1]

I posted this on AOO devel mailing list a couple of days ago, but no response by now, so I give it a try here:

I'm writing a Python script, trying to access all Basic modules from files 
in a directory. 
In most cases I get a WrappedTargetException accessing the module(s) from a 
Basic library [line: txt = one_lib.getByName(m)] ... approx. 30% of modules 
are retrieved and printed.

"xdoc" is reference to a Writer/Calc document, opened hidden within a 
function and returned from there, if this matters.

What's wrong?

Code: Select all

    if xdoc:
       log.debug('have document')
       libs = xdoc.BasicLibraries
    else:
       log.debug('no libs found')
       libs = None
     if libs and libs.hasElements():
        insert_text(text, cursor, 'Heading 2', name)

        for lib in libs.ElementNames:
            one_lib = libs.getByName(lib)

            if one_lib.hasElements():
               for m in one_lib.ElementNames:
                   try:
                        insert_text(text, cursor, stylename='Heading 3', s=m)
                        txt = one_lib.getByName(m)
                        insert_text(text, cursor, s=txt)
                   except WrappedTargetException:
                        insert_text(text, cursor, s='FEHLER BEIM LESEN')

[hanya]

What kind of exception was warpped by the exception? Check TargetException element of the instance of WrappedTargetException.

[probe1]

Thanks for looking into this.

Check TargetException element of the instance of WrappedTargetException.

I don't understand how to do that

This is the traceback recorded by the log.txt

Code: Select all

Mon Jul 13 23:35:29 2015 [DEBUG] opening >file:///home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py<
Mon Jul 13 23:35:29 2015 [DEBUG] checking for existence of /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/pythonpath.zip
Mon Jul 13 23:35:29 2015 [DEBUG] checking for existence of /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/pythonpath
Mon Jul 13 23:35:29 2015 [DEBUG] mapped file:///home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py to <module 'ooo_script_framework' (built-in)>
Mon Jul 13 23:35:29 2015 [DEBUG]  got mod <module 'ooo_script_framework' (built-in)>
Mon Jul 13 23:35:29 2015 [DEBUG] got func <function main at 0x2bcb5f0>
Mon Jul 13 23:35:29 2015 [DEBUG] PythonScript.invoke (0,)
Mon Jul 13 23:35:30 2015 [DEBUG] Error during invoking function main in module file:///home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py (<class 'uno.com.sun.star.lang.WrappedTargetException'>: 
  /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py:183 in function process_all() [txt = one_lib.getByName(m)]
  /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py:137 in function fill_doc() [si, hidden]
  /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py:114 in function main_task() [si, hidden]
  /home/USERNAME/.openoffice/4/user/Scripts/python/make_Examples/make_Code_Docu.py:438 in function main() [main_task(log)]
  /opt/openoffice4/program/pythonscript.py:869 in function invoke() [ret = self.func( *args )]

[/size]

The error pop-up window says it's a com.sun.star.uno.Exception
I'm confused by the fact that some retrieval works

[B Marcelly]

Hi,
In a document, only the Standard library is loaded by default.
You must load each library before reading it. (There is no harm to load (Standard")

Code: Select all

        for lib in libs.ElementNames:
            libs.loadLibrary(lib)
            one_lib = libs.getByName(lib)

I suppose that your routine insert_text() adds a paragraph break for each string written. If you don't, you will have a problem because in OpenOffice a paragraph contains at most 65535 characters.

Another problem is that you write the entire content of a Basic module in one instruction, probably with method insertString(). It results in a single string with a line break (not end of paragraph) for each line of Basic code.
No problem if the module contains less than 65536 characters.

But you may have Basic modules of greater length (e.g. from VBA macros of Microsoft Office documents). Then the paragraph will exceed 65535 characters. I tested a too big module with a similar Python code : the Basic code is simply not written by insertString().
In OpenOffice Basic you should not have modules greater than about 60 000 characters. The editor accepts it, but Basic will behave strangely.

[probe1]

Thanks for answering, Bernard.

In a document, only the Standard library is loaded by default.
You must load each library before reading it.

This is true for executing any code in a library other than Standard, but not for accessing the module text... if have written an Extension in Basic (OOoBTL2**) that have the same functionality I now want to reproduce in Python, which doesn't load any library, and it's working fine (with non-Standard libraries, too).

I suppose that your routine insert_text() adds a paragraph break for each string written

Yes, it does.

Then the paragraph will exceed 65535 characters. I tested a too big module with a similar Python code : the Basic code is simply not written by insertString().

I was aware of the paragraph limit, but never encountered (and never tested with) modules of a bigger size.
I will consider adding a scenario for that case. Thanks.



** the version available on the Extension website still uses the depreciated DocumentInfo structure and will not work with actual versions of LibreOffice 4 and Apache OpenOffice 4.
I have an update ready for months (years?) but was forgetful to upload it. Great reminder of doing it the next days.

[B Marcelly]

getByName
Throws
com::sun::lang::WrappedTargetException If the implementation has internal reasons for exceptions, then wrap these in a ::com::sun::lang::WrappedTargetException exception.

You don't say if the exception arises for some specific documents, or randomly ?
- specific : corrupted library container or corrupted module in this document
- random : it could be a real-time problem. Did you try to wait a little after closing each document ?

[probe1]

It happens first on the first document (sorted dir by name), which is totally fine if opened manually.


I tried a time.sleep(2) on .loadComponentFromURL() and .close() already - no difference.

As I already wrote, about 30% of the module's text are printed, the others throw the exception

[B Marcelly]

It happens first on the first document (sorted dir by name), which is totally fine if opened manually.
about 30% of the module's text are printed, the others throw the exception

You did not answer clearly : does the exception arise for some specific documents, or randomly ?
I understand it can happen or not on any document.

If I were you I would do various tests and ask me questions:

• I would review, once again, my whole code.
• What kind of documents throw exception ? OpenOffice docs, or Microsoft docs, or docs converted from Microsoft Office to OpenOffice ?
• Are the documents in a local directory, or on a network server ?
  If they are on a server, what happens if I put them on a local disk ?
• I would move the first document out of the directory and run again on the directory : does it now happen on the first of the remaining docs ?
• I would run the script on a directory containing only one document that was previously correctly analyzed.
• If I write the equivalent algorithm in Basic, does it run flawlessly ?

[probe1]

Bernard,

• files are on local drive,
• only native AOO writer and calc files, in ODF format (no MS files, no converted)
• mentioned OOoBTL2 extension can print all module text with no problem

[RoryOF]

Is the problem consistent to particular files, or does it occur randomly (not always on the same files)?

[karolus]

Most of this stuff with python but no API:

Code: Select all

from glob import glob
from fnmatch import filter as fnfilter
from zipfile import ZipFile

from xml.etree.ElementTree import tostring, ElementTree as ET



def basic_from_odf(odfpath):
    """
    return list of (Basic-module-names, modules-content)
    from  given odf-file-path
    
    """
    outlist = []
    with ZipFile(odfpath) as odfzip:
        filelist = set(fnfilter(odfzip.namelist(), 'Basic/*/*.xml'))
        excludes = fnfilter(filelist, 'Basic/*/[dialog|script]*.xml')
        
        for fpath in filelist.difference(excludes):
            with odfzip.open(fpath) as basfile:
                bas = ET(file=basfile)
                out = next(bas.iter())
                sbas = tostring(out,encoding='utf8',method='text')
                outlist.append((fpath, sbas.decode('utf8')))
    return outlist

def main():
    odflist = glob('/home/*/Documents/*.od[st]')
    for filename in odflist:
        for name, content in basic_from_odf(filename):
            print('%s\n\t%s' %(filename, name))
            print(content) 

[probe1]

After a few test runs: it's always the same set of files.
So, this has to be some issue with the files, rather than with the code (I guess)

Printed module's files are as old as from 2007 or as new as from last week.
Same goes for the failing ones.

I can print the text from all (with Python failing) modules from Basic, or access the modules text from MRI.

[mrodent33]

I have a theory about WrappedTargetException and OpenOffice/LibreOffice.

In my experience it is not usually possible to get the target exception.

I suspect (no more than this!) that this may be caused by trying to do something in "the wrong kind of thread". In particular, when you double-click on an LO app, any listeners which may be responding immediately to this opening seems to generate this error if you try to open another component. In my case I'm trying to automate Base, so if I try to open a form on opening the document (.odb file), I get this error.

And sometimes, very occasionally, I don't get it (i.e. WrappedTargetException does NOT occur). This sort of intermittent error is typical of GUI frameworks (e.g. PyQt) where you try to do something with a GUI component in a non-GUI thread.

I'm using Python, and have no idea how threads might work in a UNO Basic context...

I have yet to get much information about how OO/LO uses threads.