etc_search: when the default search is not enough

etc · 2023-10-17 11:20:34

I have quickly updated etc_search (download) to fix a bug with complex queries. This custom-type query should work now, please test:

SELECT 'article' AS type, Excerpt AS description, Posted AS posted FROM textpattern WHERE {Excerpt}
UNION ALL
SELECT 'file' AS type, description, created AS posted FROM txp_file WHERE {description}
ORDER BY posted DESC

You could create article_form and file_form and use something like <txp:output_form form="{type}_form" /> as search output content.

Bloke · 2023-10-17 12:29:24

When the hell is this plugin going to become a core component? It’s insanely good and would throw the barn doors open wide for custom searches out of the box alongside unlimited custom fields. Just sayin’.

etc · 2023-10-17 21:07:02

Bloke wrote #335793:

When the hell is this plugin going to become a core component? It’s insanely good and would throw the barn doors open wide for custom searches out of the box alongside unlimited custom fields. Just sayin’.

T’would be nice, sure, but etc_search configuration is not exactly intuitive. When we find something better than {these::weird::patterns}, perhaps…

Bloke · 2023-10-17 22:09:32

etc wrote #335796:

etc_search configuration is not exactly intuitive. When we find something better than {these::weird::patterns}, perhaps…

If inspiration strikes, please go for it. I don’t know of any CMS that would be able to touch us in terms of flexibility if we had that ability alongside unlimited CFs.

jakob · 2023-10-24 17:13:04

etc wrote #335792:

This custom-type query should work now, please test:

SELECT 'article' AS type, Excerpt AS description, Posted AS posted FROM textpattern WHERE {Excerpt}...

You could create article_form and file_form and use something like <txp:output_form form="{type}_form" /> as search output content.

Thank you Oleg. After some experimentation – I realized I needed Body instead of Excerpt – I managed to get some way there, but I can’t use txp:tags to output the actual content in the search result forms. This is what I have set in the plugin settings:

Context: custom

Static search form: search_results_{type}

Query:

SELECT 'article' AS type, Body AS description, Posted AS posted FROM textpattern WHERE {Body}
UNION ALL
SELECT 'file' AS type, description, created AS posted FROM txp_file WHERE {description}
ORDER BY posted DESC

search_input (excerpt):

<txp:etc_search id="2" action='<txp:site_url />suche/' …

search_display (excerpt):

<txp:etc_search_results wraptag="div" break="" class="search-results">
    <txp:output_form form="search_results_{type}" />
<txp:else />
    <txp:text item="no_search_matches" wraptag="h3" />
</txp:etc_search_results>

Then I have two forms search_results_file and search_results_article, each with some specific tags like txp:title, txp:article_url_title, etc. analog for files. I do get a larger total number of matches under items_count and I also see that the different forms have been referenced and output and are interleaved by descending date order as per the query.

However, the article and file tags in those forms produce no output, likewise etc_search_result_excerpt, so I have only the markup-scaffolding but no content in the results list.

If I add {description} or {posted} directly into the <txp:etc_search_results>…</txp:etc_search_results> container, they do act as content placeholders. If I try the same in the forms, I just get that as text. It seems the respective article/file details are not being passed through the forms. Any ideas?

PS: I couldn’t choose custom as a context. It required changing ''=>'custom' to 'custom'=>'custom' in the radioSet in the plugin, and also adding that option to the ENUM datatype in the DB: 'article','image','file','link','category','section','custom'.

etc · 2023-10-25 21:29:24

Yep, that’s annoying. Populating article/file/etc data for custom searches is tricky, since the current syntax does not allow mapping to standard txp fields. Parsing etc_search placeholders in forms looks doable, but a bit dirty.

What I can think of, is passing article/file/etc ids to forms via yield and retrieving data individually for each entry:

Query:

SELECT 'article' AS type, ID AS id, Posted AS posted FROM textpattern WHERE {Body}
UNION ALL
SELECT 'file' AS type, id, created AS posted FROM txp_file WHERE {description}
ORDER BY posted DESC

Display:

<txp:etc_search_results wraptag="div" break="" class="search-results">
    <txp:output_form form="search_results_{type}" id="{id}" />
<txp:else />
    <txp:text item="no_search_matches" wraptag="h3" />
</txp:etc_search_results>

Forms:

<txp:article_custom yield="id" />
<!-- same idea for files -->

That’s not very effective, but should be fast enough if search results are paginated in a reasonable way, since textpattern and txp_file tables have indexes on id.

Thanks for the remark on custom context. Could it be browser-related?

jakob · 2024-02-27 08:53:10

Apologies if this has already been mentioned in this long thread. On recent PHP8+ I get a

passing null to parameter #1 ($string) of type string is deprecated on line 576

with etc_search. It results when parse_url($action, PHP_URL_QUERY) returns NULL and then parse_str tries to process that into variables.

Presumably this only ever returns something when a user actively specifies an action attribute that contains other own url queries.

I have worked around it by changing line 576 from:

else parse_str(parse_url($action, PHP_URL_QUERY), $qs);

into

else {
        $parts = parse_url($action, PHP_URL_QUERY);
        if($parts) parse_str($parts);
    }

though I suspect you may have a more elegant solution.

jakob · 2024-02-27 09:12:34

A second minor query:

The search is pleasantly unfussy in that if you search for burgerzentrum, you get matches for “Bürgerzentrum”, which is good.

However etc_search_result_excerpt is more exacting. You get no search result excerpts with burgerzentrum but you do with bürgerzentrum (i.e. with the umlaut, as is contained in the actual article).

etc · 2024-02-28 11:22:15

Thank you, Julian, will take a look as soon as time permits.

etc · 2024-02-29 13:29:17

jakob wrote #336763:

A second minor query:

The search is pleasantly unfussy in that if you search for burgerzentrum, you get matches for “Bürgerzentrum”, which is good.

However etc_search_result_excerpt is more exacting. You get no search result excerpts with burgerzentrum but you do with bürgerzentrum (i.e. with the umlaut, as is contained in the actual article).

This one does not look minor, unfortunately. The search uses sql LIKE query, which (for your server collation) considers u and ü as equivalent. But this is not necessary the case for other possible collations. On the other hand, (etc_)search_result_excerpt calls php preg_match_all() function to extract matching chunks from article bodies returned by sql query. But php regex does not seem to support character equivalents, so /u/ pattern does not match ü. We could construct these equivalents ‘by hand’, but they depend on sql collation, and I can not see how to efficiently extract equivalent characters groups atm.

jakob · 2024-03-01 10:21:21

etc wrote #336810:

This one does not look minor, unfortunately. The search uses sql LIKE query, which (for your server collation) considers u and ü as equivalent. But this is not necessary the case for other possible collations. On the other hand, (etc_)search_result_excerpt calls php preg_match_all() function to extract matching chunks from article bodies returned by sql query. But php regex does not seem to support character equivalents, so /u/ pattern does not match ü. We could construct these equivalents ‘by hand’, but they depend on sql collation, and I can not see how to efficiently extract equivalent characters groups atm.

Thanks for investigating. Could the \pL escape sequence help here? I’m not sure I understand it fully, so I may have it exactly the wrong way around, but if I’ve understood, for example, this discussion then it’s designed to allow matches of equivalent characters in regexes.

Otherwise, is your alternative suggestion that – for search purposes – the article bodies (and the search term?) would be transliterated to ‘plain ascii’ characters, i.e. diacritics removed, to check for matches, but then the actual term and matching chunk be used for displaying the excerpt? Might the Transliterator::transliterate help there?

EDIT: extensive discussion here of transliteration alternatives.

etc · 2024-03-01 14:23:36

jakob wrote #336811:

Could the \pL escape sequence help here?

As I get it, \pL matches any ‘letter’ character. But what we need here is a pattern that matches, say, u, ü and ù, but not ë or à. We could construct some [uüù] equivalent letters groups, but ‘equivalent’ depends on the current db charset/collation. In German, for example, ss and ß could be considered equivalent, but not in other languages. And I have no idea which ‘letters’ are ‘equivalent’ in non-European languages.

Textpattern CMS

Textpattern CMS support forum

#109 2023-10-17 11:20:34

Re: etc_search: when the default search is not enough

#110 2023-10-17 12:29:24

Re: etc_search: when the default search is not enough

#111 2023-10-17 21:07:02

Re: etc_search: when the default search is not enough

Bloke wrote #335793:

#112 2023-10-17 22:09:32

Re: etc_search: when the default search is not enough

etc wrote #335796:

#113 2023-10-24 17:13:04

Re: etc_search: when the default search is not enough

etc wrote #335792:

#114 2023-10-25 21:29:24

Re: etc_search: when the default search is not enough

#115 2024-02-27 08:53:10

Re: etc_search: when the default search is not enough

#116 2024-02-27 09:12:34

Re: etc_search: when the default search is not enough

#117 2024-02-28 11:22:15

Re: etc_search: when the default search is not enough

#118 2024-02-29 13:29:17

Re: etc_search: when the default search is not enough

jakob wrote #336763:

#119 2024-03-01 10:21:21

Re: etc_search: when the default search is not enough

etc wrote #336810:

#120 2024-03-01 14:23:36

Re: etc_search: when the default search is not enough

jakob wrote #336811:

Board footer