smd_xml : extract data from XML feeds

wornout · 2010-12-03 13:55:18

I’m sorry! I changed the url: XML Feed
This is the smd_xml code:

<txp:smd_xml data="http://www.ristorantelapoiana.it/?rah_external_output=eventi" record="evento" fields="title" form="event-feed" />

And the very simple form:

<p>{title}</p>

Thanks

Bloke · 2010-12-03 14:52:16

wornout

Odd, this works for me on my test site using smd_xml v0.3:

<txp:smd_xml data="http://www.ristorantelapoiana.it/?rah_external_output=eventi" record="evento" fields="title">
<p>{title}</p>
</txp:smd_xml>

I get a list of titles shown. Which version of TXP/PHP/the plugin are you running? I could try and send you the alpha of the next version and see if that helps, but I’d like to find out why it’s not working for you with the current version when it seems fine on my site. Unless the form attribute isn’t working for some reason… try it as I have above — as a container — and see if it changes anything.

wornout · 2010-12-03 15:39:25

I’m sorry again!
I was using v0.2 :(
Now everything works fine. The form works too.

However this is my config:
PHP: 5.2.14
Textpattern: 4.3.0 (r3451)

Many thanks!

Last edited by wornout (2010-12-03 15:41:34)

Maurice · 2010-12-04 04:58:38

Stef,

Much kudos to your plugin, really appreciate the alpha/almost beta version. Works like a dream.

Many thanks, Maurice

photonomad · 2010-12-16 19:41:32

Hello! I want to use smd_xml’s “convert” attribute to remove part of a field in a blogger xml feed. The problem I have is that the text contains a comma. The plugin removes the text, but it always leaves a comma.

Example full contents of the field:
tag:blogger.com,1999:blog-5067636433974081748.post-4686107486591754652

The convert attribute in my smd_xml tag:
convert=“tag:blogger.com,1999:blog-5067636433974081748.post-|”

What smd_xml returns:
,4686107486591754652

I’ve tried using \ to escape the comma, but that doesn’t work. Is it possible to get that pesky comma to go away?

aslsw66 · 2011-01-05 06:38:37

Stef

Back in post #54, you implied there was a new version coming along that would handle subfields and compound tags. Any news on progress?

Here is what I need to do…

I want to get the weather from our Bureau of Meteorology site; the feed is here.

The first issue was the data comes via FTP, so I modified your plugin to allow FTP as one of the options (and it works!).

The real problem is the format of the data. It has multiple fields with the same name but different attributes, so all I have been able to do so far is extract all of the fields.

I know there are lots of other weather sites out there with neater XML formats, but I really need to use this feed as it has the official fire danger warning which I want to display.

Bloke · 2011-01-05 09:02:34

aslsw66 wrote:

Back in post #54, you implied there was a new version coming along

Yeah, I’m such a slacker. It’s part done and people are using the alpha version successfully.

I modified your plugin to allow FTP as one of the options (and it works!)

Brilliant, would you mind sending me the code and I’ll build that in? Thanks in advance.

multiple fields with the same name but different attributes

The new plugin has the ability to filter by attribute (erm, I think I got that far — been a long time since I looked at it) so that should help. If it’s not there right now I’ll put some time in to getting it ready for you this week.

aslsw66 · 2011-01-06 05:08:16

Here are my modifications:

Somewhere around line 91, I added the following to allow ‘ftp’ to be one of the acceptable URLS:

if ((strpos($data, 'http:') = 0) || (strpos($data, 'https:') = 0) || (strpos($data, 'ftp:') === 0)) {

Around line 120, I add the following switch case for fsock allow for the FTP port to be used:

case 'ftp':
$url['scheme'] = '';
$url['port'] = 21;

In the end, this didn’t matter as I needed to use curl instead because getting the feed was killing my page. Not that I have the faintest idea of what curl or fsock are (apart from what Wikipedia tells me).

tye · 2011-02-14 03:55:55

Just having a bit of a problem with the limit tag.

Last week it was working well, only showing 1 image, now it is showing all 8 included in the feed.

I checked the feed and the field is still correct, and the format of the feed hasn’t changed… but I wonder if it is because it is quite a complicated record structure:

<txp:smd_xml limit="1" data="http://gdata.youtube.com/feeds/api/users/roundmountaingirls/uploads?v=2" record="entry" fields="yt:videoid" cache_time="345600">
<img src="http://i.ytimg.com/vi/{yt:videoid}/hqdefault.jpg" style="width:276px" />
</txp:smd_xml>

I tried using ‘media’, ‘entry|media’ as the record, but got no results – so do I do nested records.

Not sure where I am going wrong here :(

Bloke · 2011-02-14 09:18:40

tye wrote:

Just having a bit of a problem with the limit tag.

Gah! A stupid bug, thanks for finding it. Fixed in the next version. For now go and find the following line in the code (line 227 in the official version, around 279ish in the beta) :

$rowinfo['numrecs'] = substr_count($src, '<'.$record.'>');

and swap that last line out for this:

$rowinfo['numrecs'] = substr_count($src, '<'.$record);

The reason is that each record in that particular feed’s structure begins like this:

<entry gd:etag='W/"D0IEQX47eCp7ImA9Wx5aEkU"'>

but I’d (foolishly) assumed that the record headings wouldn’t contain attributes, i.e. you would always have just <entry>. Hope that fixes it for you. I shall nip off for a dose of self-flagellation as penance for my stupiditude.

tye · 2011-02-14 22:18:11

Thats it – that fixed it!!

Nice one Stef – hold back on the dose of self-flagellation – we need you :)

I thought it was me being stupid – I am 100% sure it was working, then it suddenly stopped.

Thanks for looking at it so quickly – its definitely working now :)

jan · 2011-02-26 16:02:29

Dear Stef,

Still enjoying this one very much!
Today I had a small issue though, and I hope that maybe someone could give advice on this.

I use the plugin to read a Twitter user timeline, which is an RSS feed.
This afternoon there was apparently some overload on Twitter’s servers, because the request to http://twitter.com/statuses/user_timeline/{my_user_id}.rss?count=1 went into an infinite loop; the stream never arrived and PHP threw the “maximum execution time exceeded”-error, causing my whole website to fail, haha :)

So I dived into the code, and apparently it stuck between lines 137 and 144, a.ka. the while loop in which the stream is pulled from the external source.

I’m definitiely not that big of an expert on PHP’s curl and fsock functions, but I could see that some kind of a timeout appears to be built in to prevent the script from hanging; the function stream_set_timeout() is used for this.

Is it perpaps possible that the timeout does’t work correctly, or has anyone experienced fatal errors causes by unretrievable feeds?

Any help would be appreciated!

Cheers, Jan

Bloke · 2011-02-26 17:10:54

jan wrote:

Is it perpaps possible that the timeout does’t work correctly, or has anyone experienced fatal errors causes by unretrievable feeds?

Oops, sorry it brought your site down. I haven’t actually checked this under a non-existent feed situation, which is pretty lax of me. I’ve also learned a bit more since writing this so perhaps I can revisit the code and do it properly. You’re right that the set_timeout() should catch it but perhaps it’s not working as intended.

Can’t think of anything offhand to sort it out now, but I’ll try and whip up a fix next week.

Last edited by Bloke (2011-02-26 17:11:14)

nardo · 2011-03-16 04:57:30

I use this plugin so often — thanks Bloke

Question: could it deal with a CSV data source?

Bloke · 2011-03-16 09:02:46

nardo wrote:

could it deal with a CSV data source?

If you export your CSV file as XML, yes (Excel permits this :-) Otherwise no.

I can build other protocols in (like aslsw66 proved) but at the end of the pipe must sit a valid XML file for the plugin to work.

Textpattern CMS

Textpattern CMS support forum

#76 2010-12-03 13:55:18

Re: smd_xml : extract data from XML feeds

#77 2010-12-03 14:52:16

Re: smd_xml : extract data from XML feeds

#78 2010-12-03 15:39:25

Re: smd_xml : extract data from XML feeds

#79 2010-12-04 04:58:38

Re: smd_xml : extract data from XML feeds

#80 2010-12-16 19:41:32

Re: smd_xml : extract data from XML feeds

#81 2011-01-05 06:38:37

Re: smd_xml : extract data from XML feeds

#82 2011-01-05 09:02:34

Re: smd_xml : extract data from XML feeds

#83 2011-01-06 05:08:16

Re: smd_xml : extract data from XML feeds

#84 2011-02-14 03:55:55

Re: smd_xml : extract data from XML feeds

#85 2011-02-14 09:18:40

Re: smd_xml : extract data from XML feeds

#86 2011-02-14 22:18:11

Re: smd_xml : extract data from XML feeds

#87 2011-02-26 16:02:29

Re: smd_xml : extract data from XML feeds

#88 2011-02-26 17:10:54

Re: smd_xml : extract data from XML feeds

#89 2011-03-16 04:57:30

Re: smd_xml : extract data from XML feeds

#90 2011-03-16 09:02:46

Re: smd_xml : extract data from XML feeds

Board footer