[plugin] jk_parse_xml: get data from XML feeds

Jeff_K · 2007-09-27 19:45:01

Glad that was sorted out. Is this something I should roll into a future version?

sthmtc · 2007-09-27 19:46:36

i think so, yes… i’m not an expert in that field though :)

sthmtc · 2007-09-28 17:03:10

another problem that came up: sometimes the xml files i’m grabbing contain invalid characters, forcing the parser to stop working and instead return an error (or just nothing, respectively, depending on the production status of the site). is there a workaround for this?

variaas · 2008-01-02 05:09:40

I just found this plugin and it looks great for some of my needs. If I understand correctly, you don’t cache the file yourself when pulling them, but recommend using a cron job to cache the file. I was wondering if you were planning to roll in some caching mechanism into the plugin itself. That way it’ll attempt to pull a local copy first, if not available or older than x hours it will go out and grab the latest from the external site.

Gocom · 2008-01-02 05:23:34

Caching is made to be easy. It’s all about choises. For helping the overloading society ;)

Cheers!

Last edited by Gocom (2008-01-02 05:25:08)

variaas · 2008-01-02 18:23:48

Thanks Gocom, this looks interesting, but I’m not sure how to implement. I use shared hosting and if I’m reading correctly I don’t have access to a memcache server.

I’m wondering if there is anything wrong with the cache solution I mentioned. So the first request for the xml will just pull the xml file (via CURL or get_file_content) and then write the file to a local file. Then you just reference that file every time until a set amount of time has passed and you just refresh the file.

Jeff_K · 2008-01-03 23:56:33

Hey, yeah. good suggestions! I will investigate when I get a second…

variaas · 2008-01-04 08:34:05

Jeff – I wrote up the cache feature myself for you to review – jk_parse_xml_v0.3a.php. It caches both the xml and images (jpg, bmp, gif, png) to your textpattern/cache/ folder. Let me know your thoughts.

sthmtc · 2008-01-04 08:50:13

variaas wrote:

Jeff – I wrote up the cache feature myself for you to review – jk_parse_xml_v0.3a.php. It caches both the xml and images (jpg, bmp, gif, png) to your textpattern/cache/ folder. Let me know your thoughts.

this is somewhat off-topic but wouldn’t it be nice to combine your asv_tumblelog plugin with the additional xml parsing power of jk_parse_xml? just a thought…

variaas · 2008-01-08 02:32:21

stmtc – that’s a good idea but out of curiosity does the new xml features of php5 make SimplePie somewhat obsolete?

Vitruvius · 2008-03-09 09:41:32

I’ve finally got around to testing this plugin (the project was put on hold until now)… but I am getting the same error as sthmtc

tag_error <txp:jk_parse_xml xml="http://www.eventbrite.com/rest/organizer_list_events/15857229" form="xml-test" local_file="0" parents="events" elements="title" limit="5" /> ->  Warning: DOMDocument::loadXML() [function.DOMDocument-loadXML]: Start tag expected, '<' not found in Entity, line: 1  on line 47

I’ve tried to set the useragent (using my some borrowed code) as sthmtc which is perhaps the problem? I’ve updated the cURL part of your plugin to read:

  // else if remote, and remote fopen off, try curl
  elseif (ini_get('allow_url_fopen') == 0) {
    $ch = curl_init();
    $timeout = 5; // set to zero for no timeout
    // set user agent
    // spoofing random agent
    $useragent="Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/521.25 (KHTML, like Gecko) Safari/521.24";
    curl_setopt($ch, CURLOPT_USERAGENT, $useragent);  
    curl_setopt ($ch, CURLOPT_URL, $xml);
    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
    $xml = curl_exec($ch);
    curl_close($ch);
  }

I’d really appreciate your assistance.

I’d be happy to financially contribute to you or your fantastic site if you are still around to work on this and maybe get 0.3a out as a 0.4 release??

SH

Last edited by Vitruvius (2008-03-09 09:42:23)

Jeff_K · 2008-03-10 06:04:29

Hi Vitruvius.

I can’t fully test out the caching plugin because Curl is throwing a server error on Dreamhost right now (?), but I tried loading (a trimmed down version of) your XML from a local file.. I got it to work, although I needed to switch “events” to “event”, based on looking at the file xml you are linking too.

I will see if I can get Curl up and running again on my end, and try adding the caching feature as mentioned above (Thanks!) and see if I can replicate your problem.. In the meantime, can you try loading the plugin fresh, and change parent from “events” to “event” and see if that helps?

Also, try loading the file locally and see if that works.

The error looks like an XML error…. That is a huge huge xml file to parse with DOMDocument::loadXML. I am not sure, but that might be causing some trouble. Its 156k and 2000+ lines from the looks of it.

Thanks!

Textpattern CMS

Textpattern CMS support forum

#13 2007-09-27 19:45:01

Re: [plugin] jk_parse_xml: get data from XML feeds

#14 2007-09-27 19:46:36

Re: [plugin] jk_parse_xml: get data from XML feeds

#15 2007-09-28 17:03:10

Re: [plugin] jk_parse_xml: get data from XML feeds

#16 2008-01-02 05:09:40

Re: [plugin] jk_parse_xml: get data from XML feeds

#17 2008-01-02 05:23:34

Re: [plugin] jk_parse_xml: get data from XML feeds

#18 2008-01-02 18:23:48

Re: [plugin] jk_parse_xml: get data from XML feeds

#19 2008-01-03 23:56:33

Re: [plugin] jk_parse_xml: get data from XML feeds

#20 2008-01-04 08:34:05

Re: [plugin] jk_parse_xml: get data from XML feeds

#21 2008-01-04 08:50:13

Re: [plugin] jk_parse_xml: get data from XML feeds

#22 2008-01-08 02:32:21

Re: [plugin] jk_parse_xml: get data from XML feeds

#23 2008-03-09 09:41:32

Re: [plugin] jk_parse_xml: get data from XML feeds

#24 2008-03-10 06:04:29

Re: [plugin] jk_parse_xml: get data from XML feeds

Board footer