smd_fuzzy_find: making search results less precise

Bloke · 2007-12-30 23:12:22

After a millennia in beta it’s time for this plugin to hit the mainstream. Used in conjunction with the existing <txp:if_search /> tag it can offer a lifeline to people who type badly or mis-spell search terms. It offers alternative (hopefully close) matches of words it thinks were really meant and also lists article excerpts to show the matches in context.

The docs explain how it all works. It’s not perfect and is a little rough round the edges as the old beta thread shows; if anyone has any ideas on how to fix the current crop of known issues or any improvements / weird bug reports, please post them here. Bear in mind I don’t have a scooby how the fuzzy find algorithm that I ~~stole~~ borrowed off the Internet works. Answers on a postcard please.

Give the plugin a spin and let me know how you get on.

Revision history
————————

All available versions and changes are listed here. Each entry indexes the relevant post(s) in this thread (or the beta thread) to learn about the features.

26 Mar 07 | 0.01 | Initial public beta
31 Mar 07 | 0.02 | Fixed case sensitivity. Sped up and improved wordlist generation. Added refine attribute to switch off soundex/metaphone if required
06 Aug 07 | 0.03 | Fixed: multi-word searching (smd_getWord bug) and search_result_excerpt. Added category and status support
30 Dec 07 | 0.1 | Official first release. Added labeltag and custom field support. Fixed bad counting in limit attribute (thanks Els)
23 Jan 08 | 0.11 | Fixed MLP reliance bug (thanks Els)
01 Apr 08 | 0.12 | compressed | Moved some stuff about ; no code changes but required if you have smd_lib_v0.32 installed
02 Dec 08 | 0.2 | compressed | Requires smd_lib v0.33 ; tentative unicode support ; will read search locations set by wet_haystack (thanks Els/wet) ; enhanced subcats and renamed it sublevel ; added delim ; enhanced debug output ; plugin can now be used as a container ; tightened code, enhanced ? and ! support ; fixed smd_getWord (again!) and field list bug
19 Dec 08 | 0.21 | compressed | Fixed MLP string bug

Last edited by Bloke (2009-04-11 21:09:24)

Pat64 · 2007-12-31 09:19:48

Hi Stef.

I use smd_fuzzy_find since august and [as I said in an another thread] it’s a fantastic plugin. I’ll go to upgrade with this official version.

Thanks lot Stef.

Just one question. Do you still work on a watermark plugin? :)

Best regards,

Bloke · 2007-12-31 09:33:54

Pat64 wrote:

I use smd_fuzzy_find since august and… it’s a fantastic plugin.

Thanks, glad it’s of use to you. Please report any odd behaviour here — I know it has its problems sometimes and I’d really like to track down and fix some of the more obscure “features”, shall we say :-)

Just one question. Do you still work on a watermark plugin? :)

Oops, thanks for the reminder, I’d forgotten about it. I’ll have a look at it in the New Year when I get time. It’s something I’m going to need for a site I’ll be working on this year so I’ll try and get it working and release it for you.

Happy New Year!

Last edited by Bloke (2007-12-31 09:35:22)

Pat64 · 2008-01-02 20:14:49

Oops, thanks for the reminder, I’d forgotten about it. I’ll have a look at it in the New Year when I get time. It’s something I’m going to need for a site I’ll be working on this year so I’ll try and get it working and release it for you.
Happy New Year!

Fine Stef.

Happy new year for you too mate :)

Best regards,

els · 2008-01-23 00:19:58

Hi Stef,

I don’t know how I missed this thread ;) I just updated and, just like I (think I) did in the previous version, I modified the code to display the message in Dutch. I can change it like this:

	$smd_fuzz_str = array(
		'too_short' => 'De tekst waarnaar je zoekt is waarschijnlijk te kort. Probeer een langer woord. ',
		'no_match' => 'Sorry, geen resultaten gevonden die exact overeenkomen met "{search_term}". ',
		'suggest' => 'Hier zijn wat {thingies} die in de buurt komen en misschien helpen te vinden wat je zoekt: ',
		'suggest_join' => 'and',
		'articles' => 'articles',
		'words' => 'words',
	);

and it still works – though of course ‘words and articles’ is still in English, but when I try to change those, the output on my search page ends after the ‘suggest’ string. So it displays:

Sorry, geen resultaten gevonden die exact overeenkomen met “blahblah”. Hier zijn wat woorden en artikelen die in de buurt komen en misschien helpen te vinden wat je zoekt:

but no suggestions and no search results. Maybe I just don’t remember how I did it before? Am I overlooking something?

EDIT: I found that I can change everything but ‘articles’.

Another edit: debug="1" shows this:

(status IN ('4','5')) AND (section IN ('artikelen'))

but also when I don’t change ‘articles’ in the code, so probably not important.

Edit again (sorry, I should really do some more testing before posting!):

it works as it should if I only change ‘and’
if I change ‘and’ and ‘words’, it displays the search results but not the suggestions
if I change ‘and’ and ‘articles’ it displays the suggestions but not the search results
if I change ‘words’ and ‘articles’, or all three, it doesn’t display suggestions nor search results

I’m beginning to see a kind of logic here. And I understand that I am probably not supposed to change anything in the plugin code, but I don’t want to install the MLP pack only for this plugin, and it worked before… and I just enjoy testing ;)

Last edited by els (2008-01-23 01:05:01)

Bloke · 2008-01-23 01:12:15

Els wrote:

‘words and articles’ is still in English, but when I try to change those, the output on my search page ends after the ‘suggest’ string.

Hahaha! What a chump I am, sorry. I’d inadvertently converted the MLP strings to their “localized” content before I’d finished using them in the plugin. Hence, as you found, it looked in the code for variables named “artikelen” and “woorden” which of course are just double-dutch to the plugin :-P

Try v0.11 instead and let me know how you get on.

Sorry for the screw up, and thanks for finding the bug!

els · 2008-01-23 01:18:10

Yay, it works! I was almost convinced that I was being punished for messing with your code ;) Thank you Stef, you are the fastest replying plugin author I know!!

Bloke · 2008-01-23 01:42:55

Els wrote:

Yay, it works! I was almost convinced that I was being punished for messing with your code ;)

lol, when my code starts to exhibit true AI is when you should worry about that ;-)

Thank you Stef, you are the fastest replying plugin author I know!!

No probs, happy to help. I’m probably also in the top 10 plugin authors with the least sleep… which might explain why I let bugs like that out in the first place :-o

Thanks again for spotting it.

*stef sidles away, head low, muttering "must test more, must test more..." *

Bloke · 2008-04-01 13:31:17

Minor update that is only of significance if you update smd_lib to v0.32 (the new smd_slimbox requires it).

With the latest version of smd_lib you will need v0.12 of smd_fuzzy_find [ compressed version ]

Some stuff has just moved from smd_lib to the plugin; no other changes to the code have been made.

whocarez · 2008-07-29 10:53:08

Hello,

on my site often occurs a problem with the plugin.
If I set “tolerance” greater than “1”, I get frequently a “Fatal error: Maximum execution time of 30 seconds exceeded”. I think the solution is clear: I have to set a greater value as execution time. Something like that

ini_set("max_execution_time","false");

or

set_time_limit(xy);

But where in the source code, I can put this line in to be recognized at execution time?

thanks

ruud · 2008-07-29 10:59:36

You can put that in the config.php file in the /textpattern directory.

whocarez · 2008-07-29 13:07:55

Thanks, it helped, but now, after setting the time limit to “90”, I get only a “Internal Server Error”, when the time has elapsed.
hm, there should be another solution.

Is it possible to integrate this solution in the plugin??

<?php

set_time_limit(900);

// Start output buffering
ob_start();

$message = "First test message";
$oldmessage = "bla";

// Keep on repeating this to prevent PHP from stopping the script
while (true)
{
    $timeoutcounter = 0;
    while ($message == $oldmessage)
    {
        // If 10 seconds elapsed, send a dot (or any other character)
        if ($timeoutcounter == 10)
        {
            echo ".";
            flush();
            ob_flush();
            $timeoutcounter = 0;
        }
        // Timeout executing
        sleep(1);
        // Check for a new message
        $message = file_get_contents("chatdata.txt");
        $timeoutcounter++;
    }

    // Keep the old message in mind
    $oldmessage = $message;

    // And send the message to the user
    echo "<script>window.alert(\"" . $message . "\");</script>";

    // Now, clear the output buffer
    flush();
    ob_flush();
}

?>




Thanks

Textpattern CMS

Textpattern CMS support forum

#1 2007-12-30 23:12:22

smd_fuzzy_find: making search results less precise

#2 2007-12-31 09:19:48

Re: smd_fuzzy_find: making search results less precise

#3 2007-12-31 09:33:54

Re: smd_fuzzy_find: making search results less precise

#4 2008-01-02 20:14:49

Re: smd_fuzzy_find: making search results less precise

#5 2008-01-23 00:19:58

Re: smd_fuzzy_find: making search results less precise

#6 2008-01-23 01:12:15

Re: smd_fuzzy_find: making search results less precise

#7 2008-01-23 01:18:10

Re: smd_fuzzy_find: making search results less precise

#8 2008-01-23 01:42:55

Re: smd_fuzzy_find: making search results less precise

#9 2008-04-01 13:31:17

Re: smd_fuzzy_find: making search results less precise

#10 2008-07-29 10:53:08

Re: smd_fuzzy_find: making search results less precise

#11 2008-07-29 10:59:36

Re: smd_fuzzy_find: making search results less precise

#12 2008-07-29 13:07:55

Re: smd_fuzzy_find: making search results less precise

Board footer