Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
asy_sitemap: Google-Sitemap (as-is/for developers)
Today Google announced that it wil try a new feature called Sitemaps, which will let webmasters have a say (in the form of a “suggestion”) in the crawling process of Google. This plugin is a first step in enabling textpattern-powered sites to use this feature.
I wrote this plugin because I needed it just this way for a site, I won’t have the time (and frankly: I do not have the interest) in maintaining and extending this plugin. There are some caveats to using it the way it currently is (read below). If anybody wants to “officially” take this over, let me know so this thread can be closed and the discussion be continued in the thread to your followup-plugin. Of course everybody is also free to just take the code and “fork” it or just adapt it to your needs – actually this is even encouraged.
<hr>
Version: 0.7 updated 1. Sept. 2005
Download: http://www.sencer.de/code/txp/asy_sitemap.txt
<hr>
Help-Section as provided in the plugin:
DISCLAIMER
THIS SOFTWARE IS PROVIDED AS IS AND FOR FREE. THE AUTHOR DISCLAIMS ANY AND ALL FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DAMAGES WHATSOEVER ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE.
Important: Read Known Issues before using this plugin.
What is a Google Sitemap?
You can find the relevant Information on Google’s website:
- - Basic Information: https://www.google.com/webmasters/sitemaps/docs/en/about.html
- - Protocol: https://www.google.com/webmasters/sitemaps/docs/en/protocol.html
How do I use this Plugin?
Out of the box this plugin will let you manually create a gzip-compressed sitemap through the extension-tab. However you can edit the code of this plugin and uncomment the two lines that start with register_callback if you want the sitemap to be automatically re-created everytime you post or edit an article. (You can also add more register_callbacks for other events). For automatic re-creation, you should also change the default variables in the code in a way that suits your needs. Before you rely on automatic re-creation of the sitemap, manually test the creation and make sure that it works.
Once a sitemap is generated you have to submit it to Google, see the above links.
What does the sitemap look like and what does it contain?
The Sitemap as generated by this plugin will inlude:
- 1. A Link to the root of your txp-install with change frequency set to always
- 2. A Link to each one of your sections (that means every section), change frequency is configurable
- 3. A Link to every article that is Live or Stickied and has a Post-date that is not in the future (that should cover every publicly available article). A Last Modified date will be included. Change frequency can be configured. You can optionally also set a different change frequency for articles that have not been edited in the last X days. (Useful if you don’t want Google to keep re-fetching old articles that don’t change.)
Known Issues
- - Comments are totally ignored: The last-modified date is only the date for the article. This is useful for sites that rely mostly on static content as written by the authors. If your site has very active commenters, you may not want to use this plugin or change the code according to your desires.
- - Password Protected Articles will be in the sitemap: If your article titles are included in the url this will lead to disclosure of the Title of your posts. Of course since your articles are still password-protected the articles themselves will still not be vrawled by Google: The Googlebot has the same “permissions” and possibilites as a random guest user.
- - Several Pages will not be included in the sitemap: Section pages and the Home-page often are paginated, those “paginated pages” are not included in the sitemap. There may also be other pages not included that are provided through plugins, for example archive-pages by certain archive plugins etc. Only the pages that are mention in the three-point list above are included in the sitemap.
Last edited by Sencer (2005-09-01 15:18:34)
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
<strike>Ugh, I just saw that you have to “ping” google whenevery you generate the sitemap. The plugin currently does not do this. I will add this feature, before I put this to rest. Feel free to play with the plugin in the meantime.</strike>
Ok, The Plugin supports pinging google in the following way now:
- - Manual creation of the sitemap: It shows you a link you can click (The page will be blank, but the HTTP-response code will be 200 to let you know it worked.)
- - Automatic creation of the sitemap: A http-request will be sent to google. You can track the http headers (X-sitemap: …) to see wether it was successful.
Last edited by Sencer (2005-06-03 22:56:03)
Offline
#3 2005-06-03 23:03:58
- Andrew
- Plugin Author
- Registered: 2004-02-23
- Posts: 730
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
Wow you’re fast. I just spent a little bit creating something similar to yours, however not as an admin plugin. Check it out and let me know if there is anything you need. I wouldn’t mind taking this over for you.
Offline
#4 2005-06-05 11:52:03
- alexandra
- Member
- From: Cologne, Germany
- Registered: 2004-04-02
- Posts: 1,370
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
Hi, i did install this neat plugin, switched it live, went to the extension tab, clicked ->GO and got:
“sitemap.xml.gz does not exist. Create it and make it writable.”
Did i miss a step?
Last edited by alexandra (2005-06-05 11:52:26)
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
Well, no. That’s another step you have to do, I just forgot to mention it. Create a file in the main directory and give it that name and chmod it to 666.
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
>will let you manually create a gzip-compressed sitemap through the extension-tab
by that do you mean the plugins tab?
Yiannis
——————————
NeMe | hblack.art | EMAP | A Sea change | Toolkit of Care
I do my best editing after I click on the submit button.
Offline
#7 2005-06-06 09:15:33
- alexandra
- Member
- From: Cologne, Germany
- Registered: 2004-04-02
- Posts: 1,370
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
> colak wrote:
> >will let you manually create a gzip-compressed sitemap through the extension-tab
by that do you mean the plugins tab?
No colak, Sencer is talking about the extension tab which you find in the main navigation tabs of the admin area.
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
No. When you load the plugin tab, you cannot see the extension tab. This is done on purpose in TXP, so you can uninstall defective plugins. When you click on any other tab, you will see a new extension-tab.
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
got it thannx. works… i think! on clicking the ping to google, a blank page loads, is this right?
Yiannis
——————————
NeMe | hblack.art | EMAP | A Sea change | Toolkit of Care
I do my best editing after I click on the submit button.
Offline
#10 2005-06-06 09:39:56
- alexandra
- Member
- From: Cologne, Germany
- Registered: 2004-04-02
- Posts: 1,370
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
> colak wrote:
> got it thannx. works… i think! on clicking the ping to google, a blank page loads, is this right?
correct. don´t foget to submitt your sitemap to google!
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
Yes, blank page is correct (the HTTP status code tells you wether it was successful)
> don´t foget to submitt your sitemap to google!
After reading up on the Googls-docs – the pages on Google were mostly returning 502 Server errors on the first day – it seems that you don’t have to submit the site to google, pinging will be enough. However it is still a good idea to submit it, because you will get information when they fetched it, and wether they had any problems with the file.
Offline
Re: asy_sitemap: Google-Sitemap (as-is/for developers)
Great job Sencer!. It seems to work ok.
Offline