smd_xml : extract data from XML feeds

tye · 2011-10-31 05:10:21

Stef – your “previous explanation“http://forum.textpattern.com/viewtopic.php?pid=253658#p253658 works great :)

<txp:smd_stef name=“god” />

:)

aslsw66 · 2011-11-06 16:10:35

Stef, I thought I would use smd_xml to standardise all of my feeds, rather than using another plugin for a Twitter feed.

Here’s my code:

<txp:smd_xml data="http://twitter.com/statuses/user_timeline/actrfs.xml" record="status" fields="id; text; created_at" skip="user" wraptag="ul" delim=";" limit="4" format="text|link;created_at|date|%a, %e/%m">
<li>
<a href="http://twitter.com/textpattern/statuses/{id}">
{created_at}
</a>
<br />{text}
</li>
</txp:smd_xml>

The result is here on a test page.

As you can see, there are two problems:

the date formatting is messed up. I have tested this a bit, and discovered that if I used a dash in the format attribute it works correctly eg. %e-%m. If I separate the backslash with spaces, it also works correctly eg. %e / %m. It only chokes when I remove the spaces.
the links are messed up. Looking in the source code, there are three hrefs being created for each line. I had a look in the source code, but that regular expression stuff is worse than Greek to me (I can actually read a few street signs in Greek!).

[EDIT]
I’m definitely using the latest version of the beta. I grabbed a fresh copy just now to make sure.

Last edited by aslsw66 (2011-11-06 16:12:10)

Bloke · 2011-11-06 19:24:49

aslsw66 wrote:

the links are messed up

Bizarre. For some reason the list of fields was being parsed twice, hence it was parsing the link into an anchor and then doing it again, destroying all the good work it had just done. Thanks for the report: fixed in the “lastest beta-beta-beta-…“http://stefdawson.com/sw/beta-plugin-downloads (this is getting silly!)

the date formatting is messed up

My initial thought after looking at the feed was that it was because the format used in the stream was not one of the approved compound formats that PHP understands. But when I tried it on my server I got Fri 4/11 for the first item, when yours reports Fri 4/05.

If the latest beta doesn’t fix it (i.e. it might have been a side-effect of erroneously doing the conversion twice) then my best guess is it’s a PHP bug. If you need me to investigate further then please let me know which version of PHP you’re running.

EDIT: please also note that I’ve flip-flopped back to forcing inter-tag whitespace removal by default. Thus it is no longer a transform, but a dedicated option again called kill_spaces. It is highly recommended to leave it on as I found it improves the performance of the internal XML parser, but if you are debugging it can make the feed unnecessarily difficult to read so you may elect to turn it off while testing attributes.

EDIT 2: incidentally, if anyone would like some particular facets of the plugin depicted in the Examples segment of the help then please let me know. If you have a feed in mind then that helps: I’ll extract the relevant parts and concoct an example from it. Thanks.

Last edited by Bloke (2011-11-06 19:55:26)

aslsw66 · 2011-11-06 22:43:41

I deleted my version of smd_xml and grabbed the latest beta.

No change to the first issue (duplicate links).

On the second (ie. date issue), I’m running on PHP5.2.4. A bit more troubleshooting shows that just one space either side of the slash removes the error. Putting /%Y on the end correctly produces the year, but the date and month are still muddled. Any other characters anywhere else in the format string has no impact, and it is only he slash between day and month that causes this. Weird!

TNT · 2011-11-23 21:14:39

Simple question – at least I hope the solution is simple…

I publish my instagr.am pics to a Tumblr-page. I’m trying to use the Tumblr XML-feed combined with smd_xml to put the pics on my website.

The Tumblr-feed looks like this (simplified):

<post id="12884292408" url="http://arjan.tumblr.com/post/12884292408" url-with-slug="http://arjan.tumblr.com/post/12884292408/kou-en-zon-taken-with-instagram" type="photo" date-gmt="2011-11-16 16:27:25 GMT" date="Wed, 16 Nov 2011 17:27:25" unix-timestamp="1321460845" format="html" reblog-key="B8ftEayl" slug="kou-en-zon-taken-with-instagram" width="612" height="612">
   <photo-caption><p>Kou en zon. (Taken with <a href="http://instagr.am">instagram</a>)</p></photo-caption>
   <photo-link-url>http://instagr.am/p/UafDf/</photo-link-url>
   <photo-url max-width="1280">http://www.tumblr.com/photo/1280/12884292408/1/tumblr_lurhpq7ag61qz5i2k</photo-url>
   <photo-url max-width="500">http://29.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_500.jpg</photo-url>
   <photo-url max-width="400">http://25.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_400.jpg</photo-url>
   <photo-url max-width="250">http://29.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_250.jpg</photo-url>
   <photo-url max-width="100">http://30.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_100.jpg</photo-url>
   <photo-url max-width="75">http://24.media.tumblr.com/tumblr_lurhpq7ag61qz5i2ko1_75sq.jpg</photo-url>
</post>

The code I’m using is the following:

<txp:smd_xml data="http://arjan.tumblr.com/api/read" record="post" fields="photo-caption, photo-url" limit="3" cache_time="43200">
<article>
	<figure>
		<img src="{photo-url}" />
		<figcaption>{photo-caption}</figcaption>
	</figure>
</article>
</txp:smd_xml>

As you can see, I want to use <photo-url>. The problem is <photo-url> has multiple values. I want to use the one with max-width="500" but how do I have to feed this to the smd_xml-monster?

Any help is gladly appreciated!

Bloke · 2011-11-23 21:46:22

TNT wrote:

<photo-url> has multiple values. I want to use the one with max-width="500"

Since the tags inside your record are repeated and thus concatenated by the time the record ends, the best way I can figure is to use ontagend to check each one as it passes by. My attempt is thus:

<txp:smd_xml
     data="http://arjan.tumblr.com/api/read"
     record="post" fields="photo-caption, photo-url"
     limit="3" cache_time="43200" concat="0"
     ontagend="tnt|photo-url" />

Notice it’s now a self-closing tag and I’m using concat="0" to prevent concatenation of individual values. Without that you’d get this sequence every time you took a look at the {photo-url|max-width} entry inside a record:

1280
1280 500
1280 500 400
1280 500 400 250
…

Once you have that tag in place you can create a form (tnt in this case) which uses <txp:variable> to set the current value of the tag-att and then <txp:if_variable> to immediately test to see if it’s 500:

<txp:variable name="url_mw">{photo-url|max-width}</txp:variable>
<txp:if_variable name="url_mw" value="500">
<article>
	<figure>
		<img src="{photo-url}" />
		<figcaption>{photo-caption}</figcaption>
	</figure>
</article>
</txp:if_variable>

Et voila. You could use smd_if to achieve the same thing and test the value directly without having to do the assignment first, but in the interests of keeping the plugin count down, this works just as well. Hope that helps.

EDIT: I’m assuming you’re using the beta 0.40 here.

Last edited by Bloke (2011-11-23 21:48:50)

TNT · 2011-11-24 08:13:47

Bloke, this is brilliant. I would never have come up with this solution, so thank you very much!

newnoise · 2011-12-24 13:28:09

Hi,

I got a problem with smd_xml not being able to parse my xml, always giving memory-exhausted error:

Allowed memory size of xxx bytes exhausted (tried to allocate yyy bytes) in /path/to/textpattern/textpattern/lib/txplib_misc.php(653) : eval()‘d code on line 141

the xml is in this form:

<latest_events>
<event>
<location_name>Location</location_name>
<title>Title</title>
<url>www.url.com</url>
</event>
<event>
<location_name>Location</location_name>
<title>Title</title>
<url>www.url.com</url>
</event>
</latest_events>

and I use smd_xml like this:

<txp:smd_xml data=“http://url.com/external_latest” record=“event” fields=“title,location_name,url” wraptag=“ul”>
<li>
<span class=“published”>{location_name}</span><br />
<a href=”{url}” >{title}</a>
</li>
</txp:smd_xml>

anyone can help?
thanks

Bloke · 2011-12-24 14:03:31

newnoise

If it’s saying out of memory then I’m afraid without access to the actual URL I can’t diagnose the problem. A few questions that might help:

Have you tried the transport="curl" attribute to see if it works any better than the default fsock?
Are you using the latest beta plugin?
If the above fails, are you able to send the XML feed URL to me, perhaps by PM, so I can try it?

ruud · 2011-12-24 14:17:04

newnoise wrote:

I got a problem with smd_xml not being able to parse my xml, always giving memory-exhausted error

Though not related to this plugin, using this XML workaround is inefficient (it’s nice if there’s no alternative though), since you do have direct access to the database server that contains the data you want. See the other topic you opened. If you don’t explicitly use mysql_close and set the 4th parameter in mysql_connect to TRUE, you get what you want.

aslsw66 · 2012-01-19 23:56:12

I’ve been getting weird, intermittent error messages from my development site for a while, but just lately they have become permanent. It’s the very informative ’500 Internal Server Error’. When it was intermittent, I put it down to some hosting issues (what do I expect for $5 a month?) but now that it is permanent I’ve started to dig deeper.

Now, I’m assuming my hosting service is having some problem with pulling in data from feeds and that this is not a problem with smd_xml.

What is happening is that I have two instances of smd_xml on my homepage for bringing in the current weather and also the official fire danger ratings. I also have one instance of kml_twitter to bring in the latest tweets from a feed.

If I remove the two instances of smd_xml, my homepage displays although kml_twitter outputs a nice error message to say that it couldn’t retrieve data from Twitter. So, it looks like when smd_xml fails in this way, it fails spectacularly! I’ve had a poke around, I see some error handling in there but all I can see is it doesn’t handle this particular error.

Yes, I know, I need to move to a better host (maybe this weekend) but I just thought that I would raise the results of my troubleshooting.

[EDIT]

All of the feeds are working. I have tested them separately, and each of them returns data.

Last edited by aslsw66 (2012-01-19 23:57:47)

nardo · 2012-02-08 06:39:35

A feed transformed on a web page that was working … is no longer working.

I’ve checked that the feed is available; upgraded to beta version of the plugin; no cache attribute added … is there a way to debug output? and is there a way to ‘flush’ it – or without the cache, it’s reading every time, right … ?

Textpattern CMS

Textpattern CMS support forum

#133 2011-10-31 05:10:21

Re: smd_xml : extract data from XML feeds

#134 2011-11-06 16:10:35

Re: smd_xml : extract data from XML feeds

#135 2011-11-06 19:24:49

Re: smd_xml : extract data from XML feeds

#136 2011-11-06 22:43:41

Re: smd_xml : extract data from XML feeds

#137 2011-11-23 21:14:39

Re: smd_xml : extract data from XML feeds

#138 2011-11-23 21:46:22

Re: smd_xml : extract data from XML feeds

#139 2011-11-24 08:13:47

Re: smd_xml : extract data from XML feeds

#140 2011-12-24 13:28:09

Re: smd_xml : extract data from XML feeds

#141 2011-12-24 14:03:31

Re: smd_xml : extract data from XML feeds

#142 2011-12-24 14:17:04

Re: smd_xml : extract data from XML feeds

#143 2012-01-19 23:56:12

Re: smd_xml : extract data from XML feeds

#144 2012-02-08 06:39:35

Re: smd_xml : extract data from XML feeds

Board footer