[archived] Postmaster -- A simple email-on-post / newsletter manager

bancroft · 2007-03-29 22:41:05

Hi Ben,

I’ve sussed the Gmail problem.

In Postmaster at line 620 change this;

$headers = “From: $listAdminEmail” . “\r\n” . “Reply-To: $listAdminEmail” . “\r\n” . ‘X-Mailer: PHP/’ . phpversion() . “\r\n” . “MIME-Version: 1.0” . “\r\n” . ‘Content-Type: text/plain; charset=“UTF-8”’ . “\r\n” . “Content-Transfer-Encoding: 8bit”;

to

$headers = “From: $listAdminEmail” . “\n” . “Reply-To: $listAdminEmail” . “\n” . ‘X-Mailer: PHP/’ . phpversion() . “\n” . “MIME-Version: 1.0” . “\n” . ‘Content-Type: text/plain; charset=“UTF-8”’ . “\n” . “Content-Transfer-Encoding: 8bit”;

It was something to do with line breaks and basically just removing the \r seems to do the trick.

Cheers,

Graham.

Last edited by bancroft (2007-03-29 23:23:42)

benbruce · 2007-03-30 15:57:13

bancroft,

That’s awesome. I thought it might be something small (and hard to find) like that. Thanks for your efforts on this. I’ve added your fix to the Troubleshooting section in the Postmanual, and will make the fix for the next version.

redbot,

If you try bancroft’s fix, does it work?

Ben

Last edited by benbruce (2007-03-30 16:20:02)

redbot · 2007-03-30 18:48:52

Ben,

unfortunately it doesn’t seem to work for me.
I tried with other mail accounts and everything works well…
that’s what I get (I changed the actual site address):

Here is a text warning.

—==Multipart_Boundary_xbbf67edffbb9fc59b12f0be574cf5893x
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
<p>My excerpt</p>

—==Multipart_Boundary_xbbf67edffbb9fc59b12f0be574cf5893x
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<p>My excerpt</p>
<a href=“http://www.mysite.it/index.php?id=142&uid=1e54d5e75b
265e793e515e9f56a0fa02”>http://www.mysite.it/index.php?id=142&
;uid=1e54d5e75b265e793e515e9f56a0fa02</a>

—==Multipart_Boundary_xbbf67edffbb9fc59b12f0be574cf5893x

did you notice the 2 “- -” before “==Multipart_Boundary…”? maybe they mean something??

benbruce · 2007-03-30 19:29:12

redbot,

what does your form look like?
you are sending mails to a list that contains several addresses, and when the emails arrive in other inboxes besides Gmail, they are fine, but in Gmail they appear as you described above, is that right?
is this a different outcome from what was happening BEFORE you made bancroft’s fix? or did it change the outcome?

It is something to do with the way PM has set up it’s headers, and the way Gmail reads headers (at least, that’s what happened with bancroft’s fix). We’ll keep poking.

Ben

johnnie · 2007-03-30 21:02:54

Hmm something’s definately going wrong here. I installed the plug-in, the library and the zem_contact_reborn. I activated all, but upon going to extensions->postmaster I get several warnings, such as:

Warning: BLOB/TEXT column ‘listDescription’ can’t have a default value CREATE TABLE IF NOT EXISTS bab_pm_list_prefs ( `listID` int(4) NOT NULL auto_increment, `listName` varchar(100) NOT NULL default ‘’, `listDescription` longtext NOT NULL default ‘’, `listAdminEmail` varchar(100) NOT NULL default ‘’, `listUnsubscribeUrl` varchar(100) NOT NULL default ‘’, `listEmailForm` varchar(100) NOT NULL default ‘’, `listSubjectLine` varchar(128) NOT NULL default ‘’, `catchall` longtext NOT NULL default ‘’, PRIMARY KEY (`listID`) ) ENGINE=MyISAM CHARACTER SET = utf8 in F:\www\commercieel\pokerforstudents\textpattern\lib\txplib_db.php on line 84

benbruce · 2007-03-30 21:06:06

johnnie,

I think that’s all one warning. What version of MYSQL is on your server? That’s my first thought, that the query generally works, but you’re using a newer version of MYSQL that demands something slightly different.

Ben

johnnie · 2007-03-30 21:29:26

The version I am running is: MySQL 5.0.27-community-nt. Here’s the full list of warnings:

Warning: BLOB/TEXT column ‘listDescription’ can’t have a default value CREATE TABLE IF NOT EXISTS bab_pm_list_prefs ( `listID` int(4) NOT NULL auto_increment, `listName` varchar(100) NOT NULL default ‘’, `listDescription` longtext NOT NULL default ‘’, `listAdminEmail` varchar(100) NOT NULL default ‘’, `listUnsubscribeUrl` varchar(100) NOT NULL default ‘’, `listEmailForm` varchar(100) NOT NULL default ‘’, `listSubjectLine` varchar(128) NOT NULL default ‘’, `catchall` longtext NOT NULL default ‘’, PRIMARY KEY (`listID`) ) ENGINE=MyISAM CHARACTER SET = utf8 in F:\www\commercieel\pokerforstudents\textpattern\lib\txplib_db.php on line 84

Warning: BLOB/TEXT column ‘subscriberCustom1’ can’t have a default value CREATE TABLE IF NOT EXISTS bab_pm_subscribers ( `subscriberID` int(4) NOT NULL auto_increment, `subscriberName` varchar(100) NOT NULL default ‘’, `subscriberEmail` varchar(100) NOT NULL default ‘’, `subscriberLists` varchar(100) NOT NULL default ‘’, `subscriberCustom1` longtext NOT NULL default ‘’, `subscriberCustom2` longtext NOT NULL default ‘’, `subscriberCustom3` longtext NOT NULL default ‘’, `subscriberCustom4` longtext NOT NULL default ‘’, `subscriberCustom5` longtext NOT NULL default ‘’, `subscriberCustom6` longtext NOT NULL default ‘’, `subscriberCustom7` longtext NOT NULL default ‘’, `subscriberCustom8` longtext NOT NULL default ‘’, `subscriberCustom9` longtext NOT NULL default ‘’, `subscriberCustom10` longtext NOT NULL default ‘’, `subscriberCatchall` longtext NOT NULL default ‘’, `flag` varchar(100) NOT NULL default ‘’, `unsubscribeID` varchar(100) NOT NULL default ‘’, PRIMARY KEY in F:\www\commercieel\pokerforstudents\textpattern\lib\txplib_db.php on line 84

Warning: Table ‘poker4students.bab_pm_list_prefs’ doesn’t exist INSERT INTO bab_pm_list_prefs values (‘1’,‘default’,‘All subscribers’,’‘,’‘,’‘,‘Notification: A new article has been posted at <txp:site_url />’,’‘) in F:\www\commercieel\pokerforstudents\textpattern\lib\txplib_db.php on line 84

Warning: Table ‘poker4students.bab_pm_subscribers’ doesn’t exist INSERT INTO bab_pm_subscribers values (‘1’,‘Test’,‘test@test’,‘default’,‘custom1’,’‘,’‘,’‘,’‘,’‘,’‘,’‘,’‘,‘custom10’,’‘,’‘,‘a0a6b1f85aa97b50ae88a898648e819e’) in F:\www\commercieel\pokerforstudents\textpattern\lib\txplib_db.php on line 84

Last edited by johnnie (2007-03-30 21:30:18)

benbruce · 2007-03-30 23:26:10

johnnie,

you are getting all of those errors from the same problem (i think). It’s not liking the MYSQL queries that Postmaster uses to create the database tables — then it’s giving you errors because the tables don’t exist.

I think this is similar to an issue noted above, and that is that the query is using '' when it should be NULL for MYSQL 5.0. In other words, there is no issue when using 4.3, but something changed with 5.0 and it needs to be updated.

If you are at all comfortable with code, there’s a pretty simple find and replace you could do to try to fix it yourself:

About 3/4 of the way into the code, you’ll find “// —— CREATE TABLES —————————” and beneath that, two blocks of code you need to update.

The first:

	$create_sql1[] = safe_query("CREATE TABLE IF NOT EXISTS $bab_pm_PrefsTable (
		`listID` int(4) NOT NULL auto_increment,
		`listName` varchar(100) NOT NULL default '',
		`listDescription` longtext NOT NULL default '',

Change …

`listDescription` longtext NOT NULL default '',

… to …

`listDescription` longtext NOT NULL,

The second:

	$create_sql2[] = safe_query("CREATE TABLE IF NOT EXISTS $bab_pm_SubscribersTable (
		`subscriberID` int(4) NOT NULL auto_increment,
		`subscriberName` varchar(100) NOT NULL default '',
		`subscriberEmail` varchar(100) NOT NULL default '',
		`subscriberLists` varchar(100) NOT NULL default '',
		`subscriberCustom1` longtext NOT NULL default '',
		`subscriberCustom2` longtext NOT NULL default '',
		`subscriberCustom3` longtext NOT NULL default '',
		`subscriberCustom4` longtext NOT NULL default '',
		`subscriberCustom5` longtext NOT NULL default '',
		`subscriberCustom6` longtext NOT NULL default '',
		`subscriberCustom7` longtext NOT NULL default '',
		`subscriberCustom8` longtext NOT NULL default '',
		`subscriberCustom9` longtext NOT NULL default '',
		`subscriberCustom10` longtext NOT NULL default '',
		`subscriberCatchall` longtext NOT NULL default '',

In each case where you see “longtext”, change the line as above. The first example is …

`subscriberCustom1` longtext NOT NULL default '',

… to …

`subscriberCustom1` longtext NOT NULL,

If that doesn’t work, you could repeat the steps above, removing the “NOT NULL” parts and see if that works. So in the first example, you would change …

`listDescription` longtext NOT NULL default '',
… to …

`listDescription` longtext,

Let me know how it goes (or if you don’t want to attempt it).

Thanks,

Ben

Last edited by benbruce (2007-03-30 23:27:53)

redbot · 2007-03-31 14:17:21

benbruce wrote:

redbot,

what does your form look like?

you are sending mails to a list that contains several addresses, and when the emails arrive in other inboxes besides Gmail, they are fine, but in Gmail they appear as you described above, is that right?

is this a different outcome from what was happening BEFORE you made bancroft’s fix? or did it change the outcome?
It is something to do with the way PM has set up it’s headers, and the way Gmail reads headers (at least, that’s what happened with bancroft’s fix). We’ll keep poking.

Ben

Ben,
1) my form is the same present in the Postmanual for a “multipart mime email” with the only addiction of “<txp:bab_pm_data display=“excerpt” />” instead of “Here is the text version” and “<txp:bab_pm_data display=“excerpt_html” />” instead of “Here is the HTML version”.

Here is a text warning.
<txp:bab_pm_mime type=“text” />
<txp:bab_pm_data display=“excerpt” />
<txp:bab_pm_mime type=“html” />
<txp:bab_pm_data display=“excerpt_html” />
<txp:bab_pm_unsubscribeLink type=“html” />
<txp:bab_pm_mime type=“end” />

2)exactly, with the only exception that in my post I was refferring to virgilio.it (wich is one of the most used in Italy) not gmail

3)apparently there’s no difference with or without bancroft’s fix. The outcome is the same

In the next days I’ll do a massive testing of the plugin trying various combinations of tags in my form and checking the results both in outlook and webmail.

If I discover something useful I’ll post it immediatly

Last edited by redbot (2007-03-31 14:28:58)

redbot · 2007-03-31 18:26:14

Ben,
I’ve read my first post and maybe it was not so clear: I’ll try to be more precise now, after some testing.
Theese are the results I got.

The form I’m using is:

Here is a text warning.
<txp:bab_pm_mime type=“text” />
<txp:bab_pm_data display=“excerpt” />
<txp:bab_pm_mime type=“html” />
<txp:bab_pm_data display=“excerpt_html” />
<txp:bab_pm_unsubscribeLink type=“html” />
<txp:bab_pm_mime type=“end” />

1) email@virgilio.it
in both webmail and outlook the MIME type is treated as text and so are html tags (see my previous post)

2) email@gmail.com
Using the webmail I’m receiving only the sender and the object of the email!! no other text is displayed – the email is blank
If using outlook it looks right except an annoyng blank text attachment (ATT[somerandomnumber].txt).

3) every other mail address
all works well when the message is seen via webmail.
If using outlook it looks right except an annoyng blank text attachment (ATT[somerandomnumber].txt).

P.S.

I confirm that bancroft’s fix doesen’t work for me: the results are the same.

For the record, I’ve even thought it could be an antivirus issue (no, it wasn’t), or a server configuration issue (but my txp diagnostics are fine), or tiny_mce plugin (no, tried disabling it and nothing changes)

Last edited by redbot (2007-03-31 18:45:13)

redbot · 2007-04-01 17:33:22

Hi Ben,
My testing continues. I’m now using the simplest form

<txp:bab_pm_data />

everithyng seems to go well with every mail address. I receive no more attachments with outlook but there is another problem…
html tags are treated like simple text and I’m quite sure it depends from the hak_tinymce plugin.
I’ve tried the strip_html attribute and nothing changes.

In short, if I use a multipart MIME I have all the symptoms described in my previous post and if I use a text MIME html is not stripped.
Since I’d really like to use this plugin I wonder if it will be possible in a future release to manage to strip html tags from articles created with hak_tinymce so at least I can use it for text emails.
I hope this all makes sense.

Thanks again for your work. I think I’ll put an end to my compulsive posting now ;-)

Last edited by redbot (2007-04-01 17:34:49)

benbruce · 2007-04-02 18:04:18

There’s a couple of tangled issues here, redbot, if I’m reading everything correctly.

First, what you place between the <txp:bab_pm_mime type=“text” />
and <txp:bab_pm_mime type=“html” /> tags in your form, will need to be straight text. If it has HTML tags in it (or Textile markup), that will appear in the email. In your example, you have <txp:bab_pm_data display=“excerpt” /> — for that article, do you:

have “don’t use Textile” set for the Excerpt field?
Is it straight text or have you added any tags or Textile markup?

If you want to use the Excerpt field for both emails, try this:

<txp:bab_pm_mime type=“text” />
<txp:bab_pm_data display=“excerpt_html” strip_html="yes" />
<txp:bab_pm_mime type=“html” />
<txp:bab_pm_data display=“excerpt_html” />
<txp:bab_pm_unsubscribeLink type=“html” />
<txp:bab_pm_mime type=“end” />

Confusing. Again, that “strip_html” attribute needs testing to make sure the emails come out correctly.

The second part is getting your HTML emails to render correctly in many different email programs. This is the root of the “text vs. HTML emails” argument, which I fall on the “send text, point to an HTML web page” solution because of the incredible difficulty. See this article for more. Your efforts to work it out on your systems and email readers may help others, though. We’ll have to start a collection of working versions of forms.

Can folks post their tweaks to the forms to make emails arrive correctly in particular email readers?

Ben

Last edited by benbruce (2007-04-02 18:06:24)

redbot · 2007-04-03 14:53:02

Ben,
thanks for the clarification. Anyway, for what concerns text only emails, I think I’ve discovered why it didn’t manage to strip html tags from excerpts.
As you said I should be using:
<txp:bab_pm_data display=“excerpt_html” strip_html=“yes” />
well… it was simply not working for me and html tags were not stripped at all.
So I had a look at your code and despite my deep ignorance of php I think I found an error in postmaster library.
Around line 236 I changed

if ($display == 'excerpt_html') {
		if (!$Excerpt_html) { return; } else {
			return $Excerpt_html;
		}
	}

to

if ($display == 'excerpt_html') {
		if (!$Excerpt_html) { return; } else {
			if ($strip_html == 'yes') {
				$Excerpt_html = strip_tags(deGlyph($Excerpt_html));	
			}
			return $Excerpt_html;
		}

and now, at least for text only emails, it works perfectly!

Last edited by redbot (2007-04-03 14:55:23)

benbruce · 2007-04-03 21:44:22

redbot,

Well done! Yeah, that’s an error, and you’ve figured the fix. I’ll add that bit to my notes for the next version.

That doesn’t resolve your HTML emails rendering correctly, though.

I wish there was a better way to ask everyone who’s using the program a question like that, instead of just posting it here in the forum thread (which not everyone would be checking, obviously).

Ben

andyflan · 2007-04-06 07:24:20

Hi Ben, great plugin, thanks. I’m going to use this as a snail mail mailing list too, using the custom fields as address fields. I’m going to write some extra code for this so I can export the data to a csv that can then be used in a mail merge locally. Here’s some example SQL that I hope to employ:

SELECT a,b,a+b INTO OUTFILE ‘/tmp/result.txt’ FIELDS TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY ‘”’ LINES TERMINATED BY ‘\n’ FROM test_table;

Thought I’d just mention that, just in case you’re thinking of including such functionality anyway. Carry on the great work ;)

Textpattern CMS

Textpattern CMS support forum

#196 2007-03-29 22:41:05

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#197 2007-03-30 15:57:13

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#198 2007-03-30 18:48:52

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#199 2007-03-30 19:29:12

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#200 2007-03-30 21:02:54

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#201 2007-03-30 21:06:06

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#202 2007-03-30 21:29:26

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#203 2007-03-30 23:26:10

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#204 2007-03-31 14:17:21

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#205 2007-03-31 18:26:14

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#206 2007-04-01 17:33:22

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#207 2007-04-02 18:04:18

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#208 2007-04-03 14:53:02

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#209 2007-04-03 21:44:22

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

#210 2007-04-06 07:24:20

Re: [archived] Postmaster -- A simple email-on-post / newsletter manager

Board footer