Comparing large text files in Base

dBase, Calc, CSV, MS ACCESS, MySQL, PostgrSQL, OTHER
Post Reply
paulthomas
Posts: 1
Joined: Tue Jul 23, 2013 3:05 am

Comparing large text files in Base

Post by paulthomas »

I've looked around but haven't found anyone asking my question so I am posting here - hopefully someone can help me out!

I've used Access and Paradox 15+ years ago but haven't touched a database since then. I'm familiar with the basics (tables, queries, reports) but its all really out of date knowledge.

I am trying to find duplicate phone numbers in two text files. They are large - one is 45 million records - so I am unable to open in Calc or Excel they way I usually would.

I need to do two things:
1. Standardize the content. One is just digits e.g. 5555551212 and the other has dashes, e.g. 555-555-1212. I need to run some kind of query or report to remove the dashes in the 2nd one.
2. Run a query or report to find the numbers in one file that are also in the other file and export that list.

I've created a new database and added the text files as sources but I can't seem to do much more. What am I missing?

Most options are greyed out. "Create table in design view", everything under Forms and Reports. All I can do it create a query, but the option to "Add table" so I can compare the two is also greyed out.

I assume this is because I'm working with the text files. When I create a table from scratch all the options are there but I don't see any way to import the data from the text files.

Any idea what I'm doing wrong?

Many thanks in advance for the help!
OpenOffice Base 3.4.1 on Windows 7 64-bit
User avatar
Villeroy
Volunteer
Posts: 31279
Joined: Mon Oct 08, 2007 1:35 am
Location: Germany

Re: Comparing large text files in Base

Post by Villeroy »

When you connect Base to a directory of text files, the text files remain plain text files and Base displays them as if they were database tables. Same with certain mail address books and spreadsheets. You can do many useful things with such a pseudo database but the overall functionalitiy is limited to most simple queries on one table using filtering, sorting and a tiny set of field functions as described in: http://www.openoffice.org/dba/specifica ... tions.html including REPLACE(str,from_str,to_str)

Otherwise there might be thousands of trivial software tools to remove dashes from a certain field of a plain text file.
A powerful csv editor for Windows: http://csved.sjfrancke.nl/
Please, edit this topic's initial post and add "[Solved]" to the subject line if your problem has been solved.
Ubuntu 18.04 with LibreOffice 6.0, latest OpenOffice and LibreOffice
Post Reply