etc_query: all things Textpattern

etc · 2012-10-27 10:06:09

nardo wrote:

Another question! I am using url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDN11060.xml" but it appears the plugin is not parsing that link. That protocol not supported by etc_query?

It is, and this works just fine for me:

<txp:etc_query url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDN11060.xml"
  markup="xml"
  query="//area"
  wraptag="ul" break="li">
{@description?}: {forecast-period[1]/text[@type="precis"]?}
</txp:etc_query>

nardo · 2012-10-27 10:54:02

Thanks Oleg. I’ve pasted in your example above, and removed everything else from the page – but nothing is returned.

Could it be my PHP installation doesn’t have an extension that your’s does?

etc · 2012-10-27 11:40:56

That would be strange, etc_query uses nothing more than php file_get_contents to import data. My php is 5.4, and everything seems to work fine. There is a lag due to slow network connection, but after a second or two the data is here.

Try to pass debug="d" to etc_query to see if the feed is imported.

nardo · 2012-10-27 11:49:28

Preloaded in 3.0031430721283 seconds.

DATA

Loaded in 1.8119812011719E-5 seconds.

Processed in 4.1007995605469E-5 seconds.

So it is loading, I guess.

Within the etc_query container I put some plain text to test:

TEST OUTPUT: {@description?}: {forecast-period[1]/text[@type="precis"]?}@

That “TEST OUTPUT” text is not being displayed on page load.

etc · 2012-10-27 12:02:10

Would it be mb_convert_encoding..? Try to pass some weird encoding="abc" to etc_query. And your etc_query version is probably older than 1.2.2, please redownload if so.

nardo · 2012-10-27 12:24:16

Upgraded to 1.2.2. New debug trace added below. Adding encoding="abc" didn’t appear to do anything.

Preloaded in 3.0038778781891 seconds.

DATA xml

Loaded in 1.9073486328125E-5 seconds. 

Processed in 3.7908554077148E-5 seconds.

Should mention that I have other successful etc_query hooks on this site (although I removed them from this page when debugging).

etc · 2012-10-27 13:21:04

This Preloaded in 3.0038778781891 seconds. makes me think that your server drops the connection after 3s, but my network knowledge is rather limited. Could you try to replace the line $data = file_get_contents($url); by

$ctx = stream_context_create(array(
    'ftp' => array(
        'timeout' => 15
        )
    )
); 
$data = file_get_contents($url, false, $ctx);

in the plugin code, and report back?

Edit: and please enable txp debug mode.

Edit: no, that does not seem to work. But if you always get Preloaded in 3.00... seconds. trace, I am pretty sure it’s a timeout problem. It takes me some 7s to download from this url.

Last edited by etc (2012-10-27 13:50:13)

nardo · 2012-10-27 20:37:32

Made that change but no real difference – same output as before (no output) and similar debug trace:

Preloaded in 3.0047500133514 seconds.

DATA xml

Loaded in 2.0027160644531E-5 seconds. 

Processed in 3.9815902709961E-5 seconds.

Thanks for your help Oleg

nardo · 2012-10-27 20:50:39

I tried a smaller file – ftp://ftp2.bom.gov.au/anon/gen/fwo/IDT16090.xml – 11kb (as opposed to the other file, 250kb+)

<txp:etc_query url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDT16090.xml"
  markup="xml" debug="d" 
  query="//area"
  wraptag="ul" break="li">
{.?}
</txp:etc_query>

Similar debug trace:

Preloaded in 3.0033280849457 seconds.

DATA xml

Loaded in 2.0027160644531E-5 seconds. 

Processed in 4.0054321289062E-5 seconds.

Just for comparison…

etc · 2012-10-27 21:05:04

It takes more than 3s to load for me (~4.7s), so your timeout must be still reached. Have you put your site in debug mode, maybe file_get_contents complains about something?

nardo · 2012-10-27 21:09:49

Txp debug trace:

<txp:etc_query url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDT16090.xml"
  markup="xml" debug="d" 
  query="//area"
  wraptag="ul" break="li">
	[<txp:etc_query url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDT16090.xml"
  markup="xml" debug="d" 
  query="//area"
  wraptag="ul" break="li">: false]
	[<txp:etc_query url="ftp://ftp2.bom.gov.au/anon/gen/fwo/IDT16090.xml"
  markup="xml" debug="d" 
  query="//area"
  wraptag="ul" break="li">: true]
</txp:etc_query>
[ ~~~ secondpass ~~~ ]
 -->

etc · 2012-10-27 21:17:47

Could you try to import it with smd_xml, chances are it will do the job fine?

nardo · 2012-10-27 21:24:40

Good thinking. As smd_xml outputs nothing on the FTP XML, but outputs fine with a known HTTP XML feed, I guess it’s my environment – not the plugins?

Have checked obvious PHP config (execution time limit, etc) but I don’t know much about servers.

etc · 2012-10-27 21:35:02

It works on my server, so that must be some config issue, but we need an advice of txp:gods here. I have tried every google-found thing to change file_get_contents timeout, with no visible success. Stef uses curl, so I hoped it will work and give us a clue, but it does not import ftp(?). Well, have to test curl, but since I can not reproduce this issue on my server, it might take some time, sorry.

nardo · 2012-10-27 21:58:31

Should have tried this earlier. I saved the XML file from the FTP server to my own site and accessed it as HTTP. Worked fine, and fast!

Loaded in 0.015221118927002 seconds. 

Processed in 0.0038979053497314 seconds.

Just for reference, the etc_query was:

<txp:etc_query url="http://mywebsite.net/dump/IDN11060.xml"
  markup="xml" debug="d" 
  query="//area"
  wraptag="ul" break="li">
{@description?}: {forecast-period[1]/text[@type="precis"]?}
</txp:etc_query>

So – the possibility of timeout causing the error is still there but at least I know my environment can open & process that file.

Last edited by nardo (2012-10-27 21:59:40)

Textpattern CMS

Textpattern CMS support forum

#136 2012-10-27 10:06:09

Re: etc_query: all things Textpattern

#137 2012-10-27 10:54:02

Re: etc_query: all things Textpattern

#138 2012-10-27 11:40:56

Re: etc_query: all things Textpattern

#139 2012-10-27 11:49:28

Re: etc_query: all things Textpattern

#140 2012-10-27 12:02:10

Re: etc_query: all things Textpattern

#141 2012-10-27 12:24:16

Re: etc_query: all things Textpattern

#142 2012-10-27 13:21:04

Re: etc_query: all things Textpattern

#143 2012-10-27 20:37:32

Re: etc_query: all things Textpattern

#144 2012-10-27 20:50:39

Re: etc_query: all things Textpattern

#145 2012-10-27 21:05:04

Re: etc_query: all things Textpattern

#146 2012-10-27 21:09:49

Re: etc_query: all things Textpattern

#147 2012-10-27 21:17:47

Re: etc_query: all things Textpattern

#148 2012-10-27 21:24:40

Re: etc_query: all things Textpattern

#149 2012-10-27 21:35:02

Re: etc_query: all things Textpattern

#150 2012-10-27 21:58:31

Re: etc_query: all things Textpattern

Board footer