Textpattern's setup process

Bloke · 2012-07-12 17:44:12

As of today, Textpattern has a shiny new setup procedure which, as well as being a lot prettier thanks to Hive, changes as follows:

A progress meter along the top so you know what stage of the setup you have reached
More error trapping so things like blank user names and password cannot be submitted
Better feedback through error messages and improved language
A Back button when errors are encountered
Improved language fallback: if a string isn’t translated you get the English string instead of the_raw_value
Choice of admin-side theme in the final step. You can choose from the core three, or if you upload custom themes as part of your FTP bundle then you can choose from those too (also, as of Txp 4.5, you may put custom install code into /update/custom/post-update*.php which can be used to alter the DB on first install)

As a result, the language strings under the ‘setup’ area have changed, with many additions and quite a few alterations to the existing ones. In the coming few days, once the dust has settled and people have reported feedback on the process, please would translators jump in and align their translations with en-gb. Since the language strings for the setup process are “compiled” into Textpattern at release time, and these language strings are not updated as part of the usual language update procedure, the more translations we can get done before release, the better the process feels for non-English users. I’ll update the en-gb strings tonight before the nightly goes live so translators have a new base to work from.

The improved setup process will be in today’s nightly and we would appreciate feedback on the process here, so we can make sure it works across languages and on all manner of inputs.

Many thanks in advance for any feedback and translations.

els · 2012-07-12 19:56:53

Are translators automatically being notified of new language strings? I’m not sure if there still is a Dutch translator, haven’t seen kees-b on the forum for ages. If necessary I can lend a hand, should I contact kees-b about that?

Bloke · 2012-07-12 20:24:58

Els wrote:

Are translators automatically being notified of new language strings?

Honestly, I don’t know, as I’m a boring monoglot. Hopefully one of the translators can shed some light on this.

I’m not sure if there still is a Dutch translator, haven’t seen kees-b on the forum for ages. If necessary I can lend a hand, should I contact kees-b about that?

You could ask if you don’t mind. Failing a response, do you have an RPC account? Would you like one?

els · 2012-07-12 20:50:38

Bloke wrote:

You could ask if you don’t mind. Failing a response, do you have an RPC account? Would you like one?

I will contact Kees first. Don’t want to get in his way :)

Bloke · 2012-07-12 22:05:13

Right, I’ve deleted all the unused setup strings from the RPC lang table. The only one I left behind that we don’t actually need is prefix_warning. The only reason I left that one was because that string has been moved to a proper popup Help topic, so you might want to copy and paste the string from the Lang area to the Help area to save yourself a bit of typing. Once all the help topics have been set up, I’ll remove the string, but it does no harm right now as it’s already been translated in all languages.

There are currently five popup help topics during setup. Three of them are new:

The restrictions on characters in the Login Name field (which used to be in the setup_login lang string). There are actually no restrictions at all (apart from that it must not be blank), though the help message can imply that it would make it simpler for admins if the login name didn’t contain awkward control sequences.
The fact that the password / passphrase chosen should be a strong password, as it is a Publisher account that is being created. In the en-gb help topic I’ve gone mad and given tips and examples of good passwords, but of course you don’t have to go that far if you don’t want to! As long as people know that the password needs to be strong, that’s good enough.
The aforementioned table prefix warning / assistance message.

The other two ‘site URL’ and ‘admin theme’ are the ones used on the Prefs panel so they should already be set up.

As you test this, if anybody thinks the setup process would benefit from more popup help topics, just say so and we’ll see about putting them in.

Gocom · 2012-07-12 23:46:57

Bloke wrote:

The restrictions on characters in the Login Name field (which used to be in the setup_login lang string). There are actually no restrictions at all (apart from that it must not be blank), though the help message can imply that it would make it simpler for admins if the login name didn’t contain awkward control sequences.

There are certain limitation and issues that should be taken care of, also on the admin-side Add new user editor:

Foremost a login name cannot contain commas. Textpattern’s login cookie’s value is split by commas and using a comma (,) prevents the cookie from being processed correctly (ugh). Better solution probably would be to fix the login and cookie code to work properly with commas than prevent use of commas.

The username can not be longer than 64 characters. The limitation will be 64 bytes (not characters) if the collation/table is non-UTF8 or older MySQL version is used. You need to check MySQL version when checking the login name length. Also applies to the Users panel.

The name can not end or start with any character that is considered as white-space by PHP. These include space, \t, \n, \r, \o, \x. Any white-space will cause issues if not trimmed before adding. Textpattern uses comma-separate list of values that allow extra spacing.

Types and PHP’s type handling can cause issues too. Since Textpattern and it’s plugin relay heavily on arrays and comparisons that mostly ignore types (due to Tag Markup), the names shouldn’t be a integer or otherwise numeric value which PHP will fuck up. “Numeric” usernames should be blocked unless you make sure each and every bit of code and plugin uses types when comparing values (=== operator, in_array()’s third argument etc). Otherwise you get 0.1’s real name reported as Joe Doe while his name is Jack. That can be made sure by checking that is_numeric($user) == false and otherwise prevent saving. Also applies to the Users panel. If the username used validates as numeric you will have potential security issue in your hands.

LIKE wildcards % or _ may not be wanted as well as those are used for LIKE statements in SQL queries. Note that in Textpattern LIKE statement’s wildcards are openly processed from client’s input. If you those are allowed in usernames, you can’t match the username value correctly for instance when used in a custom field. For instance <txp:article_custom designer_in='<txp:author />' /> would match every user starting with J if the author’s username is J_%.

Last edited by Gocom (2012-07-12 23:50:54)

Gocom · 2012-07-13 00:22:19

Gocom wrote:

a login name cannot contain commas

Please see issue report 221. Contains a patch that resolves the issues.

Bloke · 2012-07-13 00:45:14

Gocom wrote:

There are certain limitation and issues that should be taken care of, also on the admin-side Add new user editor <snip>

Excellent list of issue, thanks. You’re right we should check the user name length. The Users panel uses mb_strlen() which I would have expected to catch the byte limit problem, but the MySQL version makes it more complex. Will have to think about that and delve into the archives to find out when things changed in MySQL.

Re: whitespace, I’ll do an additional test to check that the supplied $username == trim($username) and if it doesn’t, refuse it. Will that suffice or do we need to supply trim() with explicit characters? I’ll also add a check for all-numeric login names. We’ll have to think about the MySQL wildcard chars. As far as I recall, the admin side code escapes them all when doing LIKE comparisons, but yes the tag parser won’t, which does become a problem in the situation you gave.

We’re trying to balance letting international strings through (that may be encoded with potentially “illegal” characters), yet forbid anything that might cause problems. My original approach was to sanitize the login name, but that would have caused too many support requests. So now we need to tread carefully and forbid anything that might cause grief later on, without pissing off people who legitimately might be trying to use ‘legal’ characters that get encoded into ‘illegal’ ones by some intermediary (primarily a browser). Not sure if that’s a valid concern. If not, then we should be safe to outlaw % and _ directly.

The cookie problem is a good spot, thanks for the patch. Will review tomorrow. when I’m more alert.

Last edited by Bloke (2012-07-13 00:49:02)

Gocom · 2012-07-13 01:24:53

That said about commas, considering Textpattern uses comma-separated value lists, commas should probably be blocked too.

Gocom · 2012-07-13 02:09:10

Bloke wrote:

Re: whitespace, I’ll do an additional test to check that the supplied $username == trim($username) and if it doesn’t, refuse it.

What about something like this:

Index: lib/txplib_misc.php
===================================================================
--- lib/txplib_misc.php	(revision 3959)
+++ lib/txplib_misc.php	(working copy)
@@ -1838,6 +1838,33 @@
 			(safe_field('COUNT(DISTINCT('.doSlash($col).'))', doSlash($table), '1=1') <= 1);
 	}

+/**
+ * Validates username
+ * @param	string	$name
+ * @return	bool
+ */
+	function is_valid_username($name)
+	{
+		return $name && (string) sanitizeForUsername($name) === (string) $name;
+	}
+
+/**
+ * Sanitizes a string for username
+ * @param	string	$name
+ * @return	string
+ */
+	function sanitizeForUsername($name)
+	{
+		$name = trim(str_replace(array(',', '_', '%', "\n", "\t", "\r", "\0", "\x0B"), '', $name));
+
+		if (is_numeric($name) || !$name)
+		{
+			return '';
+		}
+
+		return $name;
+	}
+
 // --------------------------------------------------------------
 	function EvalElse($thing, $condition)
 	{

$username == trim($username) […] do we need to supply trim() with explicit characters

That would be enough. Just add one more = and cast $username as a string. Otherwise it potentially returns true even when the supplied values are not the same.

Last edited by Gocom (2012-07-13 02:09:40)

tye · 2012-07-13 06:13:29

That’s pretty cool – top work fellas :)

I did notice a couple of things:

1 – When I made an error on purpose to use the Back buttons, in Firefox on a mac I kept getting:

Document Expired
This document is no longer available.
The requested document is not available in Firefox’s cache. As a security precaution, Firefox does not automatically re-request sensitive documents. Click Try Again to re-request the document from the website.

Clicking “Try again” worked and this is probably a browser thing, but just thought I’d mention it.

2 – If you don’t make a config.php file and place it in the textpattern directory, then there is no error message, it just doesn’t let you continue.

Gocom · 2012-07-13 06:35:43

tye wrote:

Clicking “Try again” worked and this is probably a browser thing, but just thought I’d mention it.

Yes it is. As the message states the page wasn’t found from your cache, and since Firefox doesn’t automatically repeat HTTP POST requests in normal circumstance (i.e. sending form twice would cause issues, like double posting and saves of the form etc), the browser is unable to go back.

That said, all multi-step forms, including these type of setup installations, will require their own go back/forward system. When you chain HTTP POST events/requests you will need to provide your own navigation method to go back, since you can’t relay on the browser. In case of documents requested via HTTP POST, the browser can only go back if the document is still in cache, which it may not be and may never have been especially if the form contains a password field or was used to transmit files.

As caching of form elements go, if there are form controls added via JavaScript, those will cause caching issues when moving back if own system/hooks aren’t provided. For instance, Firefox fills in wrong fields (node indexes don’t match on initial page load) and some other may not do a thing.

Textpattern CMS

Textpattern CMS support forum

#1 2012-07-12 17:44:12

Textpattern's setup process

#2 2012-07-12 19:56:53

Re: Textpattern's setup process

#3 2012-07-12 20:24:58

Re: Textpattern's setup process

#4 2012-07-12 20:50:38

Re: Textpattern's setup process

#5 2012-07-12 22:05:13

Re: Textpattern's setup process

#6 2012-07-12 23:46:57

Re: Textpattern's setup process

#7 2012-07-13 00:22:19

Re: Textpattern's setup process

#8 2012-07-13 00:45:14

Re: Textpattern's setup process

#9 2012-07-13 01:24:53

Re: Textpattern's setup process

#10 2012-07-13 02:09:10

Re: Textpattern's setup process

#11 2012-07-13 06:13:29

Re: Textpattern's setup process

#12 2012-07-13 06:35:43

Re: Textpattern's setup process

Board footer