smd_xml : extract data from XML feeds

aslsw66 · 2011-11-06 22:43:41

I deleted my version of smd_xml and grabbed the latest beta.

No change to the first issue (duplicate links).

On the second (ie. date issue), I’m running on PHP5.2.4. A bit more troubleshooting shows that just one space either side of the slash removes the error. Putting /%Y on the end correctly produces the year, but the date and month are still muddled. Any other characters anywhere else in the format string has no impact, and it is only he slash between day and month that causes this. Weird!

TNT · 2011-11-23 21:14:39

Simple question – at least I hope the solution is simple…

I publish my instagr.am pics to a Tumblr-page. I’m trying to use the Tumblr XML-feed combined with smd_xml to put the pics on my website.

The Tumblr-feed looks like this (simplified):

<post id="12884292408" url="http://arjan.tumblr.com/post/12884292408" url-with-slug="http://arjan.tumblr.com/post/12884292408/kou-en-zon-taken-with-instagram" type="photo" date-gmt="2011-11-16 16:27:25 GMT" date="Wed, 16 Nov 2011 17:27:25" unix-timestamp="1321460845" format="html" reblog-key="B8ftEayl" slug="kou-en-zon-taken-with-instagram" width="612" height="612">
   <photo-caption><p>Kou en zon. (Taken with <a href="http://instagr.am">instagram</a>)</p></photo-caption>
   <photo-link-url>http://instagr.am/p/UafDf/</photo-link-url>
   <photo-url max-width="1280">http://www.tumblr.com/photo/1280/12884292408/1/tumblr_lurhpq7ag61qz5i2k</photo-url>
   <photo-url max-width="500">http://29.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_500.jpg</photo-url>
   <photo-url max-width="400">http://25.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_400.jpg</photo-url>
   <photo-url max-width="250">http://29.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_250.jpg</photo-url>
   <photo-url max-width="100">http://30.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_100.jpg</photo-url>
   <photo-url max-width="75">http://24.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_75sq.jpg</photo-url>
</post>

The code I’m using is the following:

<txp:smd_xml data="http://arjan.tumblr.com/api/read" record="post" fields="photo-caption, photo-url" limit="3" cache_time="43200">
<article>
	<figure>
		<img src="{photo-url}" />
		<figcaption>{photo-caption}</figcaption>
	</figure>
</article>
</txp:smd_xml>

As you can see, I want to use <photo-url>. The problem is <photo-url> has multiple values. I want to use the one with max-width="500" but how do I have to feed this to the smd_xml-monster?

Any help is gladly appreciated!

Bloke · 2011-11-23 21:46:22

TNT wrote:

<photo-url> has multiple values. I want to use the one with max-width="500"

Since the tags inside your record are repeated and thus concatenated by the time the record ends, the best way I can figure is to use ontagend to check each one as it passes by. My attempt is thus:

<txp:smd_xml
     data="http://arjan.tumblr.com/api/read"
     record="post" fields="photo-caption, photo-url"
     limit="3" cache_time="43200" concat="0"
     ontagend="tnt|photo-url" />

Notice it’s now a self-closing tag and I’m using concat="0" to prevent concatenation of individual values. Without that you’d get this sequence every time you took a look at the {photo-url|max-width} entry inside a record:

1280
1280 500
1280 500 400
1280 500 400 250
…

Once you have that tag in place you can create a form (tnt in this case) which uses <txp:variable> to set the current value of the tag-att and then <txp:if_variable> to immediately test to see if it’s 500:

<txp:variable name="url_mw">{photo-url|max-width}</txp:variable>
<txp:if_variable name="url_mw" value="500">
<article>
	<figure>
		<img src="{photo-url}" />
		<figcaption>{photo-caption}</figcaption>
	</figure>
</article>
</txp:if_variable>

Et voila. You could use smd_if to achieve the same thing and test the value directly without having to do the assignment first, but in the interests of keeping the plugin count down, this works just as well. Hope that helps.

EDIT: I’m assuming you’re using the beta 0.40 here.

Last edited by Bloke (2011-11-23 21:48:50)

TNT · 2011-11-24 08:13:47

Bloke, this is brilliant. I would never have come up with this solution, so thank you very much!

newnoise · 2011-12-24 13:28:09

Hi,

I got a problem with smd_xml not being able to parse my xml, always giving memory-exhausted error:

Allowed memory size of xxx bytes exhausted (tried to allocate yyy bytes) in /path/to/textpattern/textpattern/lib/txplib_misc.php(653) : eval()‘d code on line 141

the xml is in this form:

<latest_events>
<event>
<location_name>Location</location_name>
<title>Title</title>
<url>www.url.com</url>
</event>
<event>
<location_name>Location</location_name>
<title>Title</title>
<url>www.url.com</url>
</event>
</latest_events>

and I use smd_xml like this:

<txp:smd_xml data=“http://url.com/external_latest” record=“event” fields=“title,location_name,url” wraptag=“ul”>
<li>
{location_name} 
<a href=”{url}” >{title}</a>
</li>
</txp:smd_xml>

anyone can help?
thanks

Bloke · 2011-12-24 14:03:31

newnoise

If it’s saying out of memory then I’m afraid without access to the actual URL I can’t diagnose the problem. A few questions that might help:

Have you tried the transport="curl" attribute to see if it works any better than the default fsock?
Are you using the latest beta plugin?
If the above fails, are you able to send the XML feed URL to me, perhaps by PM, so I can try it?

ruud · 2011-12-24 14:17:04

newnoise wrote:

I got a problem with smd_xml not being able to parse my xml, always giving memory-exhausted error

Though not related to this plugin, using this XML workaround is inefficient (it’s nice if there’s no alternative though), since you do have direct access to the database server that contains the data you want. See the other topic you opened. If you don’t explicitly use mysql_close and set the 4th parameter in mysql_connect to TRUE, you get what you want.

aslsw66 · 2012-01-19 23:56:12

I’ve been getting weird, intermittent error messages from my development site for a while, but just lately they have become permanent. It’s the very informative ’500 Internal Server Error’. When it was intermittent, I put it down to some hosting issues (what do I expect for $5 a month?) but now that it is permanent I’ve started to dig deeper.

Now, I’m assuming my hosting service is having some problem with pulling in data from feeds and that this is not a problem with smd_xml.

What is happening is that I have two instances of smd_xml on my homepage for bringing in the current weather and also the official fire danger ratings. I also have one instance of kml_twitter to bring in the latest tweets from a feed.

If I remove the two instances of smd_xml, my homepage displays although kml_twitter outputs a nice error message to say that it couldn’t retrieve data from Twitter. So, it looks like when smd_xml fails in this way, it fails spectacularly! I’ve had a poke around, I see some error handling in there but all I can see is it doesn’t handle this particular error.

Yes, I know, I need to move to a better host (maybe this weekend) but I just thought that I would raise the results of my troubleshooting.

[EDIT]

All of the feeds are working. I have tested them separately, and each of them returns data.

Last edited by aslsw66 (2012-01-19 23:57:47)

nardo · 2012-02-08 06:39:35

A feed transformed on a web page that was working … is no longer working.

I’ve checked that the feed is available; upgraded to beta version of the plugin; no cache attribute added … is there a way to debug output? and is there a way to ‘flush’ it – or without the cache, it’s reading every time, right … ?

Bloke · 2012-02-08 09:43:28

nardo wrote:

is there a way to debug output?

For the time being (until I get round to doing it properly) you can add debug="3" to the tag. You’ll see the feed contents dumped to the screen if it’s working right, then a load of gubbins after that detailing each matched record it finds and what the replacements are.

without the cache, it’s reading every time, right … ?

Yep.

nardo · 2012-02-08 10:03:52

I get no output with debug=“3”

looking at the feed again, could the quotes be causing an issue?

<title>"The theatre is a source of heartaches and immeasurable joys."</title><description>“The theatre is a source of heartaches and immeasurable joys.” - Konstantin Stanislavsky</description>

Bloke · 2012-02-08 10:12:40

nardo wrote:

I get no output with debug=“3”

Odd. It sounds like the plugin’s not being called. Any errors if you put the site in testing/development mode? No dangling tags or anything in the markup anywhere?

could the quotes be causing an issue?

Maybe. Do you have a feed link you can post here (or send me), along with your smd_xml tag/form/container? I can try it out myself then and see if I can spot the issue.

aslsw66 · 2012-02-09 02:55:36

I didn’t notice the debugging option before. Here’s what happens when I use it:

1. Production status is set to “Debugging”.

2. My coding is as follows:

<txp:smd_xml data="http://rss.weather.com.au/act/canberra" record="item" fields="description,pubDate,link" limit="1" cache_time="900" format="pubDate|date|%d/%m/%Y %H:%I" debug="3">
{description}
{pubDate} <a href="{link}">weather.com.au</a>
</txp:smd_xml>

3. The browser displays the following:

++ READING CACHE smd_xml_data_bd2986 ++

++ FILTERED SOURCE DATA ++

4. My tag trace is:

<!-- txp tag trace:
[SQL (0.00014305114746094): select name, data from txp_lang where lang='en-gb' AND ( event='public' OR event='common')]
[SQL (0.0026988983154297): select name, code, version from txp_plugin where status = 1 AND type IN (0,1) order by load_order]
[SQL (0.00054311752319336): select name,code,version from txp_plugin where status = 1 AND name='mem_form']
[SQL (0.00010895729064941): select name,code,version from txp_plugin where status = 1 AND name='glz_custom_fields']
[SQL (0.00018095970153809): select name from txp_section where `name` like 'weather' limit 1]
[SQL (5.793571472168E-5): select page, css from txp_section where name = 'weather' limit 1]
[SQL (0.00041007995605469): select host from txp_log where ip='203.217.150.69' limit 1]
[SQL (0.00029516220092773): insert into txp_log set `time`=now(),page='/weather',ip='203.217.150.69',host='203.217.150.69',refer='',status='200',method='GET']
[SQL (5.8174133300781E-5): select user_html from txp_page where name='weather']
[Page: weather]
<txp:feed_link flavor="atom" format="link" label="Atom" />
<txp:feed_link flavor="rss" format="link" label="RSS" />
<txp:css format="link" />
<txp:rsd />
<txp:smd_xml data="http://rss.weather.com.au/act/canberra" record="item" fields="description,pubDate,link" limit="1" cache_time="900" format="pubDate|date|%d/%m/%Y %H:%I" debug="3">
</txp:smd_xml>
[ ~~~ secondpass ~~~ ]

Finally, if I change the production status to “Live”, then I get a ’500 Internal Server Error’; the server log shows a timeout error. This makes me wonder whether it is a problem with my host, or with my Textpattern installation, or with the plugin.

Bloke · 2012-02-09 11:40:51

aslsw66 wrote:

whether it is a problem with my host, or with my Textpattern installation, or with the plugin.

For the record, I tried your exact code on my server and it worked a treat in both Live and Debugging production status.

There is a subtle bug in the latest beta if you’re using the ‘escape’ format on servers below PHP 5.4.0 but that doesn’t apply in your case since you’re not using it. I’ve fixed it on my dev server but haven’t pushed it live yet. Will do that later.

Out of curiosity, what PHP/MySQL are you using?

Possibly realted: the problem with nardo’s code is down to the content of the feed itself because I tried it with a different feed from the same server and it worked fine. I have not as yet tracked down the issue because each record on its own is fine, and I can serve a clone of the feed from my own server or even directly in the data attribute and it’s all fine. But try and get that exact same feed from the real server and it just does nothing. Weird beyond belief.

Last edited by Bloke (2012-02-10 09:27:15)

aslsw66 · 2012-02-10 05:00:45

Thanks for checking this out – I knew it wouldn’t be a problem with the plugin, but this confirms that something weird is happening with my hosting. I’m about ready to move.

By the way, this is using PHP5.2.4 and MySQL5.0.51a.

Textpattern CMS

Textpattern CMS support forum

#136 2011-11-06 22:43:41

Re: smd_xml : extract data from XML feeds

#137 2011-11-23 21:14:39

Re: smd_xml : extract data from XML feeds

#138 2011-11-23 21:46:22

Re: smd_xml : extract data from XML feeds

#139 2011-11-24 08:13:47

Re: smd_xml : extract data from XML feeds

#140 2011-12-24 13:28:09

Re: smd_xml : extract data from XML feeds

#141 2011-12-24 14:03:31

Re: smd_xml : extract data from XML feeds

#142 2011-12-24 14:17:04

Re: smd_xml : extract data from XML feeds

#143 2012-01-19 23:56:12

Re: smd_xml : extract data from XML feeds

#144 2012-02-08 06:39:35

Re: smd_xml : extract data from XML feeds

#145 2012-02-08 09:43:28

Re: smd_xml : extract data from XML feeds

#146 2012-02-08 10:03:52

Re: smd_xml : extract data from XML feeds

#147 2012-02-08 10:12:40

Re: smd_xml : extract data from XML feeds

#148 2012-02-09 02:55:36

Re: smd_xml : extract data from XML feeds

#149 2012-02-09 11:40:51

Re: smd_xml : extract data from XML feeds

#150 2012-02-10 05:00:45

Re: smd_xml : extract data from XML feeds

Board footer