Yes, use the version from PortableApps. Unless the standard version of a program is designed to be portable, it has to have some adjustments made before it can run on a portable drive. I use the PortableApps version daily, and can vouch for its stability and quality. (I know that at least one or two other places offer a portable version of OOo, but this one is the best.)
John T. Haller is the guy who runs the PortableApps site and is responsible for converting new versions to portable format. He's usually responsive, normally able to release a new portable version within a couple of weeks of the program's general release.
There is a support forum for the portable version too:http://portableapps.com/forums/support/ ... e_portable
John would like members to limit questions there to issues specific to running the portable software. For questions about using the software itself, this OOo Community forum is the right place to ask.