Byte-Order Mark

els · 2007-05-23 21:39:26

For the first time ever I see this in my validation results:

Byte-Order Mark found in UTF-8 File.

The Unicode Byte-Order Mark (BOM) in UTF-8 encoded files is known to cause problems for some text editors and older browsers. You may want to consider avoiding its use until it is better supported.

I looked it up, but to be honest it doesn’t make me much wiser. What is it? How do I locate it? Should I get rid of it?

Zanza · 2007-05-23 22:04:06

Did you have this issue in a textpattern generated page?
I had the same warning, but only with a text-editor generated static page. Never with txp.

Anyway, what I’ve understood is that BOM can cause misunderstanding and interfer with some high level protocol. Not to be used IF the character encoding of the page is UTF-8, as reccomended in xhtml.

Maybe this page has some more detail. UTF-8 doesn’t need a BOM, and it makes no difference about big or little endian form of byte, so rule-of-thumb is: do not use in utf-8 xhtml pages.

If someone knows more…

Z-

els · 2007-05-23 22:11:46

Thanks Zanza, I just found that same page ;)

And yes, it’s a Textpattern generated page. All the site’s pages, as a matter of fact. I don’t see anything strange in the source code, but then I don’t know if I should (is this thing visible?).

The only difference with other TXP sites I can think of is that this is the only site I’ve installed the MLP pack on. Maybe I’d better scan that thread…

Last edited by els (2007-05-23 22:12:55)

Zanza · 2007-05-23 22:19:08

No, it isn’t visible. It is an invisible starting character. Text-editors have an option to let you save with or without BOM. So it’s strange. I don’t run MLP, actually it could be a related issue. Let me know if in that thread you can find an answer.

Z-

Last edited by Zanza (2007-05-23 22:19:30)

Mary · 2007-05-24 02:55:20

It’s “visible” if you’ve got a text editor that can understand them (mine marks them as being used, in the status bar). Textpattern files do not have them though (they are in UTF-8, but without the BOM).

Did you open and save any of them in a text editor?
Can you please post your “High” diagnostics results?

Last edited by Mary (2007-05-24 02:56:31)

ruud · 2007-05-24 08:09:08

The english language file (lang directory) has a BOM.

els · 2007-05-24 20:17:36

Mary wrote:

It’s “visible” if you’ve got a text editor that can understand them (mine marks them as being used, in the status bar). Textpattern files do not have them though (they are in UTF-8, but without the BOM).

Did you open and save any of them in a text editor?

Do you mean open and save pages or files in a text editor? Never mind, I did neither.

Can you please post your “High” diagnostics results?

Sure. I now notice this: clean_url_data_failed, which I think wasn’t there before I installed the MLP pack. Clean url’s work though.

Textpattern version: 4.0.4 (r1973)
Last Update: 2006-10-12 14:57:57/2007-05-20 13:04:13
Document root: /home/<username>/public_html
$path_to_site: /home/<username>/public_html
Textpattern path: /home/<username>/public_html/textpattern
Permanent link mode: section_title
Temporary directory path: /home/<username>/public_html/textpattern/tmp
Site URL: skippyworks.net
PHP version: 4.4.6
Register globals: 1
Server Local Time: 2007-05-24 16:03:43
MySQL: 4.1.21-standard
Locale: en_GB.UTF-8
Server: Apache/1.3.37 (Unix) mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 FrontPage/5.0.2.2635.SR1.2 mod_ssl/2.8.28 OpenSSL/0.9.7a PHP-CGI/0.1b
PHP Server API: cgi
RFC 2616 headers: 
Server OS: Linux 2.6.9-42.0.10.ELsmp
Active plugins: gbp_admin_library-0.4.581, rss_admin_db_manager-4.0.2, zem_nth-0.1, zem_contact_reborn-4.0.3.19, l10n-0.6.600, zem_contact_lang-mlp-4.0.3-MLP, sed_copyright-1.3, sed_plugin_library-0.3

Pre-flight check: 
------------------------
Some Textpattern files have been modified: /lib/txplib_db.php
The following PHP functions (which may be necessary to run Textpattern) are disabled on your server: shell_exec, system, passthru
clean_url_data_failed: 
------------------------

.htaccess file contents: 
------------------------
#DirectoryIndex index.php index.html
#Options +FollowSymLinks
#RewriteBase /relative/web/path/

<IfModule mod_rewrite.c>
 RewriteEngine On
 RewriteCond %{REQUEST_FILENAME} -f [OR]
 RewriteCond %{REQUEST_FILENAME} -d
 RewriteRule ^(.+) - [PT,L]

 RewriteRule ^(.*) index.php
</IfModule>

#php_value register_globals 0

------------------------

Charset (default/config): latin1/utf8
character_set_client: utf8
character_set_connection: utf8
character_set_database: latin1
character_set_results: utf8
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/share/mysql/charsets/
17 Tables: -

PHP extensions: xslt, xmlrpc/0.51, xml, wddx, tokenizer/0.1, swf, standard/4.4.6, sockets, snmp, session, pspell, posix, overload, mysql, mhash, mcrypt, mbstring, imap, gettext, gd, ftp, exif/1.4 $Id: exif.c,v 1.118.2.37.2.7 2007/01/09 11:38:04 tony2001 Exp $, domxml/20020815, curl, ctype, calendar, bcmath, zlib/1.1, pcre, openssl, ionCube Loader, Zend Optimizer

pretext_data: /c4bf7b80ded069b068d7a01f79228f5e/?txpcleantest=1
array (
  'id' => '',
  's' => '',
  'c' => '',
  'q' => '',
  'pg' => '',
  'p' => '',
  'month' => '',
  'author' => '',
  'request_uri' => '/c4bf7b80ded069b068d7a01f79228f5e/?txpcleantest=1',
  'qs' => 'txpcleantest=1',
  'subpath' => '\\/',
  'req' => '/c4bf7b80ded069b068d7a01f79228f5e/?txpcleantest=1',
)

/include/txp_category.php: r1879 (aee777474b2f67ca07fc25756ba25c15)
/include/txp_plugin.php: r1917 (74184c0d8ed8608f840707a255178617)
/include/txp_auth.php: r1879 (b1dd4072b7daf4e997c6ff65ce3d1b2d)
/include/txp_form.php: r1913 (16ec600b41438b4cca10d2c8a19b2db8)
/include/txp_section.php: r1891 (2959593586ba3e97bc602f369c32e738)
/include/txp_tag.php: r1915 (3b4a7f73d92f9bbbe09985c5aa830d29)
/include/txp_list.php: r1892 (41f4d32fd070234b78f94adefebd5234)
/include/txp_page.php: r1913 (34331a5468bbb18dd9f6a282f3aa11e8)
/include/txp_discuss.php: r1909 (9b9ee934a30f52cd7a4d8cb45c8380ab)
/include/txp_prefs.php: r1946 (05615b6275d8927a2a0d51918d70a896)
/include/txp_log.php: r1919 (ed54d02e865319f2506c642a6bde768b)
/include/txp_preview.php: r1238 (5a4ae3ff0d68f4cb573d6d62a00ce9e8)
/include/txp_image.php: r1955 (20be975e67fa7c4aa9a1a3e51bfaf379)
/include/txp_article.php: r1889 (7749f699c03d0c57e04fafe17dbfa94c)
/include/txp_css.php: r1897 (f5cf1c20badb96a063c7c180e9020359)
/include/txp_admin.php: r1879 (d36dac010d21df7bcf9cf5e242b34d58)
/include/txp_link.php: r1879 (0652287df8bb32c66cfa1b939402404a)
/include/txp_diag.php: r1902 (96697ade63048e517177f4129d47de76)
/include/txp_file.php: r1895 (6ed67b094522e51b028dc88baa07444c)
/include/txp_import.php: r1238 (634e75d1b61958875ff275e3130f23ad)
/lib/admin_config.php: r1747 (1563fcbaffe25b3272b0d85ff9d5571d)
/lib/txplib_misc.php: r1956 (182c50b86195f1abe9dbe15728df3cae)
/lib/taglib.php: r1535 (04806ef864d5b0d2974e0e5f6397a2d7)
/lib/txplib_head.php: r1887 (b110efd071e9a5bb395beea66ced128a)
/lib/classTextile.php: r1943 (2c559991e34738eef1990dc079bd91c4)
/lib/txplib_html.php: r1937 (c206ca9cb9a54a7a95f3355b77fd0fa2)
/lib/txplib_db.php: r1879 (125d6835470af76be4b0b369533e25cf)
/lib/IXRClass.php: r765 (cbe59b59246dce060a4b4a52b4d448d8)
/lib/txplib_forms.php: r1887 (0049a228dc8eb346f8603478a7c1b2e2)
/lib/class.thumb.php: r1955 (12961180eee3add5096e69e0a154284e)
/lib/constants.php: unknown (0e40251c717c52b2b7fe992b62a3e97a)
/lib/txplib_update.php: r1239 (757f8189fcc53a795d7c807f17b2e788)
/lib/txplib_wrapper.php: unknown (584448787b4a3488200722672c0eee0d)
/publish/taghandlers.php: r1949 (3fa1b9ded18e6074b2495a3f4e3c33b5)
/publish/atom.php: r1864 (50602e2f1c443819a0a60f14f39d3093)
/publish/log.php: r1637 (a4a772567079f18101a1752446f3f6d4)
/publish/comment.php: r1951 (a3f803d744fea80808eb27a3f6b28674)
/publish/search.php: r1748 (b0182abc287055fe0932c263b2a5266d)
/publish/rss.php: r1864 (ae43eaa9ebe6b00e63810ae60ca7c6b6)
/publish.php: r1973 (492577e7379e745d7a421c855d148ced)
/index.php: r1948 (adf86f44861797f4969373c708ef48fb)
/css.php: r944 (763fa7658fc19ad23a5b2126fcdf366c)

ruud wrote:

The english language file (lang directory) has a BOM.

But how does the validator see that? That file is not loaded with my pages, is it?

ruud · 2007-05-24 20:45:02

True, the en-gb.txt BOM is not loaded on your pages. I was just pointing out the fact that a BOM does occur in one of the TXP files.

Do you have an URL you can post that has this problem. I’m wondering if the BOM is at the very beginning of the page or somewhere in the middle. If it’s at the beginning, then I’d expect to find it in the template for that page, otherwise it could be in a language string or an article or a form.

els · 2007-05-24 20:51:36

It’s skippyworks.net (you could have found that in my diagnostics ;)) The BOM notice is in the validation results of all pages (I use only one template). The same goes for the MLP demo site, that’s why I think it is related.

ruud · 2007-05-24 21:36:37

Hmm.. I’ve just downloaded the MLP pack.
/textpattern/lib/l10n_base.php has a BOM at the beginning. Because that’s before the <?php tag and assuming that file is loaded on each page load before the template is used, that BOM is sent to the browser before the template is parsed (the BOM is at the very beginning of the page as it arrives in the browser, before the DOCTYPE).

Try the notepad trick with that file: open it in notepad -> save as type ANSI (it should be 3 bytes smaller) -> upload it to your site, replacing the original l10n_base.php file.

Last edited by ruud (2007-05-24 21:46:16)

els · 2007-05-24 21:49:21

Sorry, I don’t quite get it. I didn’t have Notepad so I just installed it, but how do I save a file as ANSI? Do I choose Format and check Encode in ANSI? Do I also have to check UTF-8 without BOM?

els · 2007-05-24 21:55:48

Never mind, I did it (without checking UTF-8 without BOM) and the BOM is gone!

Thanks for your trouble Ruud! Just curious: how on earth do you find and see that BOM? I never saw anything…

Textpattern CMS

Textpattern CMS support forum

#1 2007-05-23 21:39:26

Byte-Order Mark

#2 2007-05-23 22:04:06

Re: Byte-Order Mark

#3 2007-05-23 22:11:46

Re: Byte-Order Mark

#4 2007-05-23 22:19:08

Re: Byte-Order Mark

#5 2007-05-24 02:55:20

Re: Byte-Order Mark

#6 2007-05-24 08:09:08

Re: Byte-Order Mark

#7 2007-05-24 20:17:36

Re: Byte-Order Mark

#8 2007-05-24 20:45:02

Re: Byte-Order Mark

#9 2007-05-24 20:51:36

Re: Byte-Order Mark

#10 2007-05-24 21:36:37

Re: Byte-Order Mark

#11 2007-05-24 21:49:21

Re: Byte-Order Mark

#12 2007-05-24 21:55:48

Re: Byte-Order Mark

Board footer