[Solved] Import scipy.stats in a OpenOffice Python Macro

[morgan_sickness]

Hello there,

I've been learning Python Macros on OpenOffice and some trouble came to me.
First of all, Im trying to do this:

Code: Select all

import scipy.stats as st
#from scipy import stats as st

def RunMacro():
    pass

This code is on C:\Program Files (x86)\OpenOffice.org 3\Basis\share\Scripts\python.

So I can see the Macro when I open OpenOffice Calc.
The problem comes when I run it: it freezes the Calc.
I've already tried the solution given here: http://forum.openoffice.org/en/forum/vi ... 20&t=55675 , by karolus.
But I have no sucess (you can see I've wrote and commented this solution on the code above).

I dont know if it's wrong when I just copy and paste the Numpy and Scipy lib folders on C:\Program Files (x86)\OpenOffice.org 3\Basis\program\python-core-2.6.1\lib\site-packages.
These Numpy and Scipy folders are the ones in my Python 2.6 installed in C:\Python26.
When I do this I guess Im installing it with sucess, since I can import those modules in the shell: C:\Program Files (x86)\OpenOffice.org 3\program\python.exe
Ex.:

Code: Select all

>>>import scipy.stats

Furthermore, when I run a Macro with the script:

Code: Select all

import scipy
import numpy

def RunMacro():
    pass

it works.


Does anyone knows where should I go? What am I doing wrong? Is that possible to import scipy.stats to my OpenOffice Python Macro?

[Jan_J]

Generally, if internal package information contains no absolute paths, the copying could be safely applied. As far as native Python source and bytecode.
But I am not sure whether the binary code included in “official” and “openoffice” Python releases are compatible. They might have been produced by different C compilers or linked against incompatible libraries.
Some time ago, in years of the Sun Corp., there was documentation available for this topic. I am not sure about the current state, but I am afraid the problem still exists.
Btw. one of the tasks being proceeded by Linux distribution packagers is to integrate LibreOffice with the system. This includes also unification with “system” Python release, and requires building binaries over common library base.

[hanya]

Frozen in:

Code: Select all

numpy.finfo(float)

[morgan_sickness]

Thinking about what Jan_J said, Im wondering that my copy and paste isn't right.
Maybe, my solution, should be to build and install numpy and scipy directly on the OpenOffice's Python. You know, install all pre-requisites, BLAS, LAPACK and ATLAS stuff...
This going to be challenging!

Anyone knows some good tutorial for doing this?

[hanya]

On Linux 32bit environment, I got the same dead lock. The problem happened during the calculation of iexp in numpy.core.machar.MachAr class. In tenth roop of the calcuation, temp = z*t never returned when z: ndarray: [ 5.56268465e-309] and t: ndarray: [ 1.]. This blocks Python's main thread.
I wrote the following code to avoid the problem:

Code: Select all

import sys
# On Linux environment, shipped Python is build with ucs2 on AOO 3.4. 
# So this can not work in general. I used custom ucs4 build.
sys.path.append("/usr/lib/python2.7/dist-packages")

import threading

scipy = None


class Foo(object):
    """ Scipy wrapper. 
        
        This class imports scipy module in the different thread to 
        avoid blocking of Python's main thread.
    """
    
    SCIPY_IMPORTED = False
    _scipy = None
    _scipy_stats = None
    
    def __init__(self):
        pass
    
    def is_imported(self):
        return self.__class__.SCIPY_IMPORTED
    
    def import_scipy(self):
        """ Import scipy in another thread. """
        if self.is_imported(): return True
            
        t = threading.Thread(None, self._import_scipy)
        t.start()
        if t.is_alive():
            t.join(5) # importing scipy needs long time
        #if not self.is_imported():
        #    raise Exception("Failed to import scipy.")
        return self.is_imported()
    
    def _import_scipy(self):
        try:
            import scipy as _scipy
            global scipy
            scipy = _scipy
            
            import scipy.stats as _stats
            
            self.__class__.SCIPY_IMPORTED = True
        except Exception as e:
            print(e)
    
    def scipy_stats_describe(self, a):
        if self.is_imported():
            return scipy.stats.describe(a)
        raise Exception("Not imported.")


def run():
    foo = Foo()
    if foo.import_scipy():
        s = foo.scipy_stats_describe((1, 2, 3, 4))
    else:
        s = "Failed to import"
    XSCRIPTCONTEXT.getDocument().getText().setString(str(s))

This works on Xubuntu 12.04 32bit and AOO 4-dev with ucs4 build of Python as shipped, default Scipy and Numpy package provided by the Ubuntu package system. I do not know this works on Windows environment. I will try.

[morgan_sickness]

hanya, it works on Windows! Yay!

But... i dont know exactly what we did... let me see if I understood:
- we imported scipy.stats in a different thread, giving 5 seconds to the thread do it

I wrote what you did like this:

Code: Select all

def _import_scipy():
    global sp
    import scipy.stats as sp

import threading
t = threading.Thread(None, _import_scipy)
t.start()
t.join(5)

def run():
    XSCRIPTCONTEXT.getDocument().Sheets.Sheet1.getCellByPosition(5, 0).setValue(sp.norm.pdf(0))

and then, everything worked! Thanks a lot!!!

Just one more thing:
- this "importing from another thread" can bring some future in my implementations?


Thanks a lot again!!

[hanya]

Thank you for testing it on Windows environment.

giving 5 seconds to the thread do it

5 seconds is little bit long but depends on the environment. So checking the t.is_alive() in each hundreds second while a few or more seconds passed would be ok. But needs experiments.

- this "importing from another thread" can bring some future in my implementations?

May be other problems would be popping up. But the result of finfo(type) is cached and thay do not be the problem once they are calculated. What do you want to do with it? It depends on your implementation. I have tried to use scipy to implement statistical calculation tool but I met the same problem at that time and I did not check it deeply.

[morgan_sickness]

Ok, hanya, I need your help again.

Let's explain, I have 3 things:
A. MyMacro.py - with a RunMacro() function
B. MyModule.py - with functions which needs scipy.stats and which Im going to use on MyMacro.py
C. scipy.stats - our old friend which one helps us with the statiscal stuff

In MyMacro.py we have:

Code: Select all

from MyModule import *

def MacroRun():
    XSCRIPTCONTEXT.getDocument().Sheets.Sheet1.getCellByPosition(5, 0).setValue(Function1(0))

In MyModule.py we have:

Code: Select all

def _import_scipy():
    global sp
    import scipy.optimize as sp

import threading
t = threading.Thread(None, _import_scipy)
t.start()
t.join()

def Funcion1(x):
    return sp.norm.pdf(x)

Am I doing something wrong or the threading thing doesnot like this?

Im implemetating like this because I want MyModule.py independent of MyMacro.py, so I can test and debug my rotine outside OpenOffice environment.

[hanya]

I couldn't solve to import in module level. I do not have enough knowledge what is happen in the import.
When I set time out of join method to 3 second, it did not freeze but sp could not find in MyModule.
But in the second execution, it worked.
I tried some ways but the following one seems the choice:

Code: Select all

# in MyModule.py
__all__ = ["Function1", "init"]

def _import_scipy():
    global sp
    import scipy.stats as sp # norm is in scipy.optimize?

def init():
    #if not "sp" in globals(): # to avoid new thread creation
    import threading
    t = threading.Thread(None, _import_scipy)
    t.start()
    t.join() # even no timeout, it returns

def Function1(x):
    return sp.norm.pdf(x)

Code: Select all

# in MyMacro.py
from MyModule import *
init() # import scipy in MyModule, ok in module level

def MacroRun():
    XSCRIPTCONTEXT.getDocument().Sheets.Sheet1.getCellByPosition(5, 0).setValue(Function1(0))

To request to import in init function of MyModule seems OK.

[morgan_sickness]

Thank you hanya!!

This was what I needed. It worked!

I would like just to say that I had another Module been called inside MyModule, and both of them needed to import scipy.stats. So I made what you suggested for both modules: i created new functions called initModule1 and initModule2 for each particulas modules and called them in MyMacro.

Everything working! Thank you again!

The optimize there is wrong, my mistake, ops.