[Solved] Unicode - Different char displayed

[DaveStone]

I am starting with a sqlite database (data from a test instrument), reading it using Python and ultimately creating a CSV file which combines data from various tables within the database.

Reading the data using 'Notepad' shows that the data is correct in the CSV file, eg.

1,200,69,2017-04-28,09:57:10,2,110v Transformer,Main shed,1,0.04 Ω
2,538,547,2017-04-28,10:29:43,4,13A to IEC Test ,PAT 420,1,0.02 Ω

However, when I open the file in OpenOffice Calc, the Greek char appears as Ω , see below

1 200 69 28/04/17 09:57:10 2 110v Transformer Main shed 1 0.04 Ω
2 538 547 28/04/17 10:29:43 4 13A to IEC Test PAT 420 1 0.02 Ω

I know that OpenOffice can display the special char Ω, but I am lost as to where/how the corruption is occurring, and more importantly how to fix it ?

Dave

[FJCC]

If you can post a couple of lines of the actual file, it will be easier to see what is going on. To post a file, click Post Reply and then look for the Upload attachment tab below the box where you type a response.

[MrProgrammer]

Hi, and welcome to the forum.

reading [SQLITE] using Python and ultimately creating a CSV file …. However, when I open the file in OpenOffice Calc, Ω appears as Ω

What character encoding did Python use when it created the CSV file? You must specify that same character encoding in Calc's Text Import dialog. If you don't know what character encoding your Python system uses, it is time to discover that. You are the programmer.

I will guess that Python created the file with Unicode UTF-8 encoding where Ω is represented by bytes X'CE' and X'A9'. In other encodings, X'CE' and X'A9' represent different characters. For example in ISO-8859-1 (Western European) they represent two individual characters, Î and ©. In Unicode UTF-8 they represent Ω.

If this solved your problem please go to your first post use the Edit button and add [Solved] to the start of the title. You can select the green checkmark icon at the same time.

[DaveStone]

I specified Unicode UTF-8 in Python

I had tried Greek, but I get the correct result if I use Unicode UTF-8.

Thank you

[Villeroy]

Permanent database connection without csv import/export:
Get a 32bit Java Runtime for Windows from https://java.com/en/download/manual.jsp
Download the latest jdbc driver from https://bitbucket.org/xerial/sqlite-jdbc/downloads/
Call office-menu:Tools>Options>Java...
Point to the 32-bit Java and add the downloaded jar file to the class path.
Reload the office suite.

File>New>Database...
[X] Connect to existing db
Type: JDBC
Connection URL: jdbc:sqlite:path_to_your_database.db
Driver class: org.sqlite.JDBC
[X] Register database
Save the database document under some name. This document represents a connection to your SQLite database. The document itself does not contain any data.
menu:Edit>Database>Advanced Settings: "Respect the result set type from the database driver" = ON
Save again.

Open a spreadsheet.
Hit F4 for the data source window
Browse your database and drag a table icon from the left pane into the sheet.
Save the spreadsheet in ODF format (*.ods).
###### END OF SETUP ############
Update the database link via Calc-menu:Data>Refresh while the cell cursor is anywhere in the import range.

You can do the same linking with queries. Save a meaningful query in the Base document and drag it from the data source window into a document.