Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Pages: 1
#1 2007-01-31 20:37:31
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Importer help - volunteers needed
Wordpress 2.1 has again demonstrated the ongoing problem with keeping Textpattern’s database import modules up to date. The database designs of other CMS and blog apps change, and we wind up having to reverse-engineer someone else’s code.
We could use some volunteers to help keep the importers up to date. Here’s what I’d really like people to do:
1. Create a reference set of test articles in MT Import file format. The test import file should represent as many common scenarios as possible: articles with and without comments, categories, different statuses and authors, i18n, markup, and anything else that might be relevant. Brief is good: no need to include long article bodies unless that’s relevant to the test.
2. Import the test data into the current versions of supported applications like Wordpress. Preferrably fresh, stock installs without extra data to confuse us.
3. Extract a copy of the relevant tables (articles, obviously, but perhaps also comments, categories, authors etc) using mysqldump, and send it to the dev team.
Since the mysqldump data all originates from the one input set (the MT import file), we’ll know what we’re looking at. We can test it easily, keep track of changes across versions, and re-test with new versions of Textpattern and of the other applications.
Anyone who’d like to volunteer, please post here.
Last edited by zem (2007-01-31 20:54:13)
Alex
Offline
#2 2007-01-31 20:42:36
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Importer help - volunteers needed
To get things started, here’s the sample MT import file from the spec page:
TITLE: A dummy title
AUTHOR: Foo Bar
DATE: 01/31/2002 03:31:05 PM
PRIMARY CATEGORY: Media
CATEGORY: News
-----
BODY:
This is the body.
-----
EXTENDED BODY:
Here is some more text.
-----
COMMENT:
AUTHOR: Foo
DATE: 01/31/2002 15:47:06
This is
the body of this comment.
-----
COMMENT:
AUTHOR: Bar
DATE: 02/01/2002 04:02:07 AM
IP: 205.66.1.32
EMAIL: me@bar.com
This is the body of
another comment. It goes
up to here.
-----
PING:
TITLE: My Entry
URL: http://www.foo.com/old/2002/08/
IP: 206.22.1.53
BLOG NAME: My Weblog
DATE: 08/05/2002 16:09:12
This is the start of my
entry, and here it...
-----
--------
TITLE: Here is a new entry
AUTHOR: Baz Quux
DATE: 01/31/2002 03:31:05
CATEGORY: Politics
-----
BODY:
This is the body of the second entry. It can
consist of multiple lines.
-----
EXCERPT:
See, this entry does not have an extended piece; but
it does have an excerpt. It is special.
-----
COMMENT:
AUTHOR: Quux
URL: http://www.quux.com/
DATE: 01/31/2002 04:23:01 PM
Here is the first comment on this entry.
-----
--------And here’s a quick test for i18n characters.
--------
AUTHOR: gemal
TITLE: test of æ test
STATUS: Publish
ALLOW COMMENTS: 1
CONVERT BREAKS: __default__
ALLOW PINGS: 0
PRIMARY CATEGORY: Test
CATEGORY: Test
DATE: 01/21/2004 01:18:35 AM
-----
BODY:
body test æ test
-----
EXTENDED BODY:
extended test æ test
-----
EXCERPT:
excerpt test æ test
-----
KEYWORDS:
-----Alex
Offline
Re: Importer help - volunteers needed
Why it should be some test set? Why it can be dump from original database fill with actual stuff?
[ I apologize for my poor english ]
Offline
#4 2007-02-01 07:36:43
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Importer help - volunteers needed
If we start with a known set of data, we can tell if the end result is correct because we have something to compare it to.
If we start with arbitrary data, we can’t, because we don’t know the end result should look like.
We also can’t be sure the data covers all the tricky little gotchas we need to know about.
Last edited by zem (2007-02-01 07:42:38)
Alex
Offline
#5 2007-02-01 07:41:54
- net-carver
- Archived Plugin Author
- Registered: 2006-03-08
- Posts: 1,648
Re: Importer help - volunteers needed
empiryk
Why it can be dump from original database fill with actual stuff?
Well, eventually it could be; but starting small is simply easier to handle for all involved: dev team or those who volunteers here.
Small test sets that are designed to excercise different scenarios, one at a time, for the import routine can help to identify problems much more quickly than large, real-life, data sets.
Importing a few carefully crafted articles and checking the results are as expected can build confidence in the correctness of the import routine much faster than importing and then having to check a thousand real-life articles that either don’t present a problem to the importer — or that do have problems but you cant tell where the problem is because the test cases don’t exercise just one scenario at a time.
For example, look at the difference from the first to the second comment in that test set and you’ll see that the first has author, date and body whilst the second adds IP and email to that. So if the first comment shows up ok and the second doesn’t we already know that it’s most probably the IP or Email import code that’s incorrect.
That’s where the power of well chosen test sets come in.
— Edit —
Looks like zem snuck that post in there faster than I could.
Last edited by net-carver (2007-02-01 07:42:58)
— Steve
Offline
Pages: 1