Page 1 of 1

Similarity Search not handling 'special' characters?

Posted: Sat Jul 27, 2019 6:20 pm
by ooGuillaume
Hello,

Maybe I'm not understanding Similarity Search, but:

In test file attached, using LibreOffice 6.2.5.2, there's only one cell filled with i@mlost.

I do a Search & Replace of the string iamlost with Similarity Search ticked, Exchange, Add and Remove characters settings all set to 1 and Combine is unticked.

Cell i@mlost is not found. If I replace its content with iOmlost, then the same search finds the cell.

What gives?

Re: Similarity Search not handling 'special' characters?

Posted: Sat Jul 27, 2019 7:04 pm
by RusselB
Like you, with the similarity settings at 1, I got Not Found,
However, experimenting with the settings, I did get a match by putting the Remove characters setting to 2, leaving the rest at 1 and Combine unchecked.

Re: Similarity Search not handling 'special' characters?

Posted: Sat Jul 27, 2019 7:21 pm
by Bill
When the cell content is "iOmlost", only 1 character from the search term "iamlost" has been exchanged, so that cell is found.

When the cell content is "i@mlost", the special character "@" appears to act as a word separator, so Search sees two words "i" and "mlost" in the cell. Since both of those words have more than 1 character removed from the search term "iamlost", the cell is not found. Change the "Remove characters" setting to "2" and the cell will be found.

Re: Similarity Search not handling 'special' characters?

Posted: Sun Jul 28, 2019 6:57 pm
by ooGuillaume
Thanks for the explanation. Indeed, the dot "." as a separator gives the same behaviour.

My real use case is looking for mail duplicates inside a list of 1800+ addresses.
For instance searching for contact@gmail.com should find the mistyped contact@gmial.com, searching for first.lastname@gmail.com should find firstlastname@gmail.com.

The only idea I have right now is copying all these mail addresses to another column and removing any special character from them before doing the search.
Would you have any other better ideas?

Re: Similarity Search not handling 'special' characters?

Posted: Sun Jul 28, 2019 7:56 pm
by RusselB
Test some of your other entries with the Remove setting at 2 and see if you get any unexpected results?
Since the characters specified so far are acting as word barriers, and making that setting works for those, then it should work for any other word barriers.
I don't know if single character matches would also be detected by the 2 setting, as I don't know if the 2 setting is a fixed number to remove, or if it's an upper limit.

Re: Similarity Search not handling 'special' characters?

Posted: Sun Jul 28, 2019 9:40 pm
by ooGuillaume
It's an upper limit: without these specific characters, strings get found with 0, 1 or 2 characters removed.

But on a simple case below I can't find a way to make things work just by tweaking the settings.

I tried setting Exchange, Remove and Add settings at 3 characters, and adding the possibility to combine these changes.

1/ Cell is filled with contact@gmail.com.
Searching the strings contact@gmail.com, contactgmail.com, contact@gmailcom or contactgmailcom won't find the cell.

2/ On the contrary, modifying the cell to contactgmailcom will have it be found by the four strings above.

3/ With the same similarity settings, a cell filled with contactgmail.com will be found by the search contactgmailcom, but not by the search contactgmail.com.