etc_query: all things Textpattern

johnstephens · 2017-06-22 20:01:45

Hi, @etc!

This appeared to work at first, but I ran into another snag.

Calling etc_query with markup="xml" breaks the search-and-replace thing I’m doing.

As explained here, I need to parse my content for span elements with the class ‘fnt’, replace those elements in the content with footnote markers, and add their content to an endnotes variable placed later in the document.

When I call that tag without a markup attribute, etc_query reads it as HTML (I guess). It finds all my //span[@class=‘fnt’] elements, replaces them with footnote markers, populates the endnotes variable, and outputs all the content with the target pattern replaced with the appropriate replacement—but it removes the / from self-closing XHTML elements.

<txp:etc_query
  data='<div><txp:yield /></div>'
  globals="variable"
  query="//span[@class='fnt']"
  replace='&=<sup><a href="#endnote_{$+({?endnote_no}|{#row})}x<txp:article_id/>" id="marker_{$+({?endnote_no}|{#row})}x<txp:article_id/>">{$+({?endnote_no}|{#row})}</a></sup>'
  specials="replace,content"
  >

  <txp:variable name="endnotes">

    <txp:variable name="endnotes"/>

    <li id='endnote_{$+({?endnote_no}|{#row})}x<txp:article_id/>'>{text()} <a href='#marker_{$+({?endnote_no}|{#row})}x<txp:article_id/>'>^</a></li>

  </txp:variable>

  {$=({#row}|{#rows}).?({//body/node()}<txp:variable name="endnote_no" value="{$+({?endnote_no}|{#row})}" />)}

</txp:etc_query>

But when I add markup="xml" to the tag, the output is drastically different. It still finds all my //span[@class=‘fnt’] elements, and it still populates my endnotes variable. But instead of returning my content with the target pattern replaced with replacements, it displays nothing.

<txp:etc_query
  data='<div><txp:yield /></div>'
  globals="variable"
  markup="xml"
  query="//span[@class='fnt']"
  replace='&=<sup><a href="#endnote_{$+({?endnote_no}|{#row})}x<txp:article_id/>" id="marker_{$+({?endnote_no}|{#row})}x<txp:article_id/>">{$+({?endnote_no}|{#row})}</a></sup>'
  specials="replace,content"
  >

  <txp:variable name="endnotes">

    <txp:variable name="endnotes"/>

    <li id='endnote_{$+({?endnote_no}|{#row})}x<txp:article_id/>'>{text()} <a href='#marker_{$+({?endnote_no}|{#row})}x<txp:article_id/>'>^</a></li>

  </txp:variable>

  {$=({#row}|{#rows}).?({//body/node()}<txp:variable name="endnote_no" value="{$+({?endnote_no}|{#row})}" />)}

</txp:etc_query>

Is there a way to get it to do the search-and-replace I want, and display the content with replacements, without turning my self-closing XHTML tags into unclosed HTML tags?

Thank you in advance for any guidance you can offer!

Last edited by johnstephens (2017-06-22 20:03:12)

etc · 2017-06-22 21:03:14

Hi John,

I think that’s because there is no <body /> in XML-imported document (in HTML it is created automatically). Try to replace {//body/node()} with {div/node()} or simply {div} or {/}, some of them should work.

Hope it helps.

Edit: btw, if you call etc_query with debug="h" attribute, it will display the data as it sees it.

johnstephens · 2017-06-22 21:23:13

I can’t thank you enough, @etc! At first brush, {div/node()} appears to work. I have to hit it with a battery of tests before declaring victory, though. I’ll keep the debug attribute in mind!

johnstephens · 2017-07-14 21:22:48

Dear @etc,

It seems I’m determined to find out everything that can possibly go wrong when parsing XHTML with etc_query.

Here is the tag:

<txp:etc_query
    data='<div><txp:yield /></div>'
    globals="variable"
    markup="xml"
    query="//span[@class='fnt']"
    replace='&=<sup><a href="#endnote_{$+({?endnote_no}|{#row})}x<txp:article_id/>" id="marker_{$+({?endnote_no}|{#row})}x<txp:article_id/>">{$+({?endnote_no}|{#row})}</a></sup>'
    specials="replace,content"
    >
    {$=({#row}|{#rows}).?({//div/node()}<txp:variable name="endnote_no" value="{$+({?endnote_no}|{#row})}" />)}
</txp:etc_query>

Here is the content before it is consumed by etc_query:

<p>Lorem ipsum</p>

<div class="figure right col_25">
<img id="figure1" src="http://example.com/images/559.jpg" alt="" />
<p class="figcaption2"><strong>Figure 1</strong>. An image, of some kind</p>
</div>


<p>Dolor sit amet</p>

And this is what etc_query returns:

<p>Lorem ipsum</p><div class="figure right col_25"><img id="figure1" src="http://example.com/images/559.jpg" alt=""/><p class="figcaption2"><strong>Figure 1</strong>. An image, of some kind</p></div><img id="figure1" src="http://example.com/images/559.jpg" alt=""/><p class="figcaption2"><strong>Figure 1</strong>. An image, of some kind</p><p>Dolor sit amet</p>

For some reason, etc_query appears to be duplicating the image caption paragraph, and appending the duplicate markup directly after the containing div. The affected markup does not contain either the query pattern or the replacement pattern, and I can’t tell what’s going on.

Do you have any suggestions how I might identify the root of the problem?

Thank you!

Last edited by johnstephens (2017-07-14 21:28:03)

etc · 2017-07-14 21:34:00

Hi John, no worries. ~~Do you happen to call //div/node()~~ yes, I think that was it. This matches all div tags, you should replace it with /div/node() (or simply div/node()?) to match only the root div. Tell me if it helps?

johnstephens · 2017-07-14 21:52:35

Since I accidentally solved it, I wanted to post the solution here for posterity—or at least for myself if I ever face a similar problem:

It seems like //div/node() was duplicating every single div in the data, not just the wrapping div:

{$=({#row}|{#rows}).?({//div/node()}<txp:variable name="endnote_no" value="{$+({?endnote_no}|{#row})}" />)}

Replacing that with /div/node() seems to resolve the issue:

{$=({#row}|{#rows}).?({/div/node()}<txp:variable name="endnote_no" value="{$+({?endnote_no}|{#row})}" />)}

johnstephens · 2017-07-14 21:53:15

etc wrote #306277:

Hi John, no worries. ~~Do you happen to call //div/node()~~ yes, I think that was it. This matches all div tags, you should replace it with /div/node() (or simply div/node()?) to match only the root div. Tell me if it helps?

Ha, yes! It looks like I stumbled over this while you were writing. Thank you!

johnstephens · 2017-07-14 21:54:13

P.S. I did try div/node(), and that returned nothing—neither the root div nor any nesting divs. But /div/node() worked.

etc · 2017-09-11 12:02:32

For txp-dev testers, etc_query v1.3.4 is released. The main new feature (for txp 4.7-dev only) is two global attributes: etc-query and etc-replace. Examples:

<txp:body etc-replace="//img@@class=someclass" />

will output articles body with class="someclass" applied to all <img /> tags, and

<txp:body etc-query="//img" etc-replace="@@height=200" />

will extract all <img /> tags and set their height attribute to 200.

maniqui · 2017-10-10 03:09:59

An XPath cheatsheet that could come in handy next time anyone here needs some help with XPath queries (which aren’t always that easy).

lazlo · 2018-12-10 00:09:02

Hi Oleg

I recently came across xpath and have been trying to get my head around it I am hoping for guidance with using it with etc_query.

I have a custom field called “ONIX_record” that contains a single XML record in a block code below.
I have been playing around with xpath at the XPath Expression Testbed which seems understandable but I am having trouble translating it to etc_query.

I want to display the text inside: <PersonNameInverted>Homel, David</PersonNameInverted>

This code highlights the node in the test bed but how do I translate this to etc_query?

//Product/Contributor[ContributorRole='B06']/PersonNameInverted

And the biggie (for me) is how do I search for across my database for multiple matches?

An example of what I want to do is display an alpha list of all my Contributors, similar to what I had done before previously with your help.
see here:

lazlo wrote #298483:

Hey Oleg

Hoping you can help me out again as the <!— alpha list A-Z —> works wonderfully for the “Title” field. In my case I use it for both an Book Title List and an Author List. These articles have alternative titles that exist in a custom field, in one case it is <txp:custom_field name=“Contributor_Full_Name_Reversed” />

the xml code

<xml><Product>
    <RecordReference>9780889227729</RecordReference>
    <NotificationType>03</NotificationType>
    <ProductIdentifier>
      <ProductIDType>03</ProductIDType>
      <IDValue>9780889227729</IDValue>
    </ProductIdentifier>
    <ProductIdentifier>
      <ProductIDType>15</ProductIDType>
      <IDValue>9780889227729</IDValue>
    </ProductIdentifier>
    <ProductIdentifier>
      <ProductIDType>02</ProductIDType>
      <IDValue>0889227721</IDValue>
    </ProductIdentifier>
    <ProductIdentifier>
      <ProductIDType>01</ProductIDType>
      <IDTypeName>ONIX_NOSEQ</IDTypeName>
      <IDValue>b99c6a66-b284-4882-842b-2bbf85c57a9b</IDValue>
    </ProductIdentifier>
    <Barcode>00</Barcode>
    <Barcode>00</Barcode>
    <ProductForm>DG</ProductForm>
    <ProductFormDetail>B102</ProductFormDetail>
    <EpubType>099</EpubType>
    <NoSeries></NoSeries>
    <Title>
      <TitleType>01</TitleType>
      <TitleText>A Covenant of Salt</TitleText>
      <TitlePrefix>A</TitlePrefix>
      <TitleWithoutPrefix>Covenant of Salt</TitleWithoutPrefix>
    </Title>
    <WorkIdentifier>
      <WorkIDType>15</WorkIDType>
      <IDValue>9780889225664</IDValue>
    </WorkIdentifier>
    <Contributor>
      <SequenceNumber>1</SequenceNumber>
      <ContributorRole>A01</ContributorRole>
      <PersonName>Martine Desjardins</PersonName>
      <PersonNameInverted>Desjardins, Martine</PersonNameInverted>
      <NamesBeforeKey>Martine</NamesBeforeKey>
      <KeyNames>Desjardins</KeyNames>
    </Contributor>
    <Contributor>
      <SequenceNumber>2</SequenceNumber>
      <ContributorRole>B06</ContributorRole>
      <PersonName>David Homel</PersonName>
      <PersonNameInverted>Homel, David</PersonNameInverted>
      <NamesBeforeKey>David</NamesBeforeKey>
      <KeyNames>Homel</KeyNames>
    </Contributor>
    <Contributor>
      <SequenceNumber>3</SequenceNumber>
      <ContributorRole>B06</ContributorRole>
      <PersonName>Fred A. Reed</PersonName>
      <PersonNameInverted>Reed, Fred A.</PersonNameInverted>
      <NamesBeforeKey>Fred A.</NamesBeforeKey>
      <KeyNames>Reed</KeyNames>
    </Contributor>
    <NoEdition></NoEdition>
    <Language>
      <LanguageRole>01</LanguageRole>
      <LanguageCode>eng</LanguageCode>
    </Language>
    <NumberOfPages>160</NumberOfPages>
    <MainSubject>
      <MainSubjectSchemeIdentifier>10</MainSubjectSchemeIdentifier>
      <SubjectCode>FIC019000</SubjectCode>
      <SubjectHeadingText>FICTION / Literary</SubjectHeadingText>
    </MainSubject>
    <Audience>
      <AudienceCodeType>01</AudienceCodeType>
      <AudienceCodeValue>01</AudienceCodeValue>
    </Audience>
      <OtherText>
      <TextTypeCode>02</TextTypeCode>
      <TextFormat>05</TextFormat>
      <Text>&lt;p&gt;This novel explores the snares of individual and collective memory as they are used to justify and preserve ancestral grudges.&lt;/p&gt;</Text>
      <TextPublicationDate>20130403</TextPublicationDate>
    </OtherText>
    <OtherText>
      <TextTypeCode>05</TextTypeCode>
      <TextFormat>05</TextFormat>
      <Text>“Mining from the past, Desjardins extracts treasures without \\\'getting caught,\\\' and surfaces like a breath of fresh air. &lt;em&gt;A Covenant of Salt&lt;/em&gt; marries literary traditions in a sleek gothic ceremony, silvery salt sprinkled like confetti and the Saint Lawrence coursing through.&lt;br /&gt;— &lt;em&gt;Montreal Review of Books&lt;/em&gt;</Text>
      <TextPublicationDate>20130403</TextPublicationDate>
    </OtherText>
    <MediaFile>
      <MediaFileTypeCode>04</MediaFileTypeCode>
      <MediaFileFormatCode>03</MediaFileFormatCode>
      <MediaFileLinkTypeCode>01</MediaFileLinkTypeCode>
      <MediaFileLink>http://talonbooks.com/images/9780889225664.jpg</MediaFileLink>
      <MediaFileDate>20160229</MediaFileDate>
    </MediaFile>
    <Imprint>
      <NameCodeType>06</NameCodeType>
      <NameCodeValue>9780889227729</NameCodeValue>
      <ImprintName>Talonbooks</ImprintName>
    </Imprint>
    <Publisher>
      <PublishingRole>01</PublishingRole>
      <PublisherName>Talonbooks</PublisherName>
    </Publisher>
    <CityOfPublication>Vancouver</CityOfPublication>
    <CountryOfPublication>CA</CountryOfPublication>
    <PublishingStatus>04</PublishingStatus>
    <PublicationDate>20130901</PublicationDate>
    <YearFirstPublished>2013</YearFirstPublished>
    <SalesRights>
      <SalesRightsType>01</SalesRightsType>
      <RightsTerritory>WORLD</RightsTerritory>
    </SalesRights>
    <RelatedProduct>
      <RelationCode>13</RelationCode>
      <ProductIdentifier>
        <ProductIDType>15</ProductIDType>
        <IDValue>9780889225664</IDValue>
      </ProductIdentifier>
      <ProductForm>BC</ProductForm>
      <ProductFormDetail>B106</ProductFormDetail>
    </RelatedProduct>
    <SupplyDetail>
      <SupplierName>Constellation</SupplierName>
      <SupplierRole>01</SupplierRole>
      <ReturnsCodeType>02</ReturnsCodeType>
      <ReturnsCode>N</ReturnsCode>
      <ProductAvailability>20</ProductAvailability>
      <OnSaleDate>20130901</OnSaleDate>
      <Price>
        <PriceTypeCode>01</PriceTypeCode>
        <ClassOfTrade>Electronic Book Text</ClassOfTrade>
        <PriceAmount>17.95</PriceAmount>
        <CurrencyCode>CAD</CurrencyCode>
      </Price>
    </SupplyDetail>
  </Product></xml>

Last edited by lazlo (2018-12-10 00:18:10)

etc · 2018-12-10 10:10:44

Hi Les,

your xpath looks ok, you just need to pass markup="xml" to etc_query:

<txp:etc_query data='<txp:custom_field name="ONIX_record" escape="" />'
    query="//Product/Contributor[ContributorRole='B06']/PersonNameInverted"
    markup="xml" wraptag="ul" break="li"
>
    Contributor: {?}
</txp:etc_query>

Constructing alpha lists is more complicated, since your custom fields can contain multiple contributors, so sorting at db level is problematic. I think you can extract all records and sort them via XSL, like here, but haven’t tried yet.

Textpattern CMS

Textpattern CMS support forum

#301 2017-06-22 20:01:45

Re: etc_query: all things Textpattern

#302 2017-06-22 21:03:14

Re: etc_query: all things Textpattern

#303 2017-06-22 21:23:13

Re: etc_query: all things Textpattern

#304 2017-07-14 21:22:48

Re: etc_query: all things Textpattern

#305 2017-07-14 21:34:00

Re: etc_query: all things Textpattern

#306 2017-07-14 21:52:35

Re: etc_query: all things Textpattern

#307 2017-07-14 21:53:15

Re: etc_query: all things Textpattern

etc wrote #306277:

#308 2017-07-14 21:54:13

Re: etc_query: all things Textpattern

#309 2017-09-11 12:02:32

Re: etc_query: all things Textpattern

#310 2017-10-10 03:09:59

Re: etc_query: all things Textpattern

#311 2018-12-10 00:09:02

Re: etc_query: all things Textpattern

lazlo wrote #298483:

#312 2018-12-10 10:10:44

Re: etc_query: all things Textpattern

Board footer