adi_menu – section hierarchy, section menu and breadcrumb trail

jakob · 2024-02-08 11:12:57

detail wrote #336580:

I seem to recall that my version had an Options panel to the right of the Help, but from memory, that was to install Textpack. That is no longer visible, but I guess it is no longer required.

Yes, as adi’s site is no longer online, the textpack download options made no sense, so that “Options panel” has been removed.

The Menu is in the dropdown Presentation menu, but it has the same issue. It flashes on the page for a millisecond, and the orange “Upgrade required” error message still appears.

That message is coming from the plugin. It’s obviously detecting that something is still not up to date with your installation.

Switching the plugin off and back on again (not deleting, just deactivating and reactivating) should trigger the in-built upgrade function. Try that first.

If you still have no luck, edit the plugin and find these two lines and set the 0 temporarily to 1:

$adi_menu_debug = 0; // display admin debug info
$adi_menu_db_debug = 0; // display database debug info

You’ll then see debug information on the panel, including a line v1_0=, v1_1=, v1_4=, v1_4s=, v1_5t=. If one of those shows some output, that might help us further. You can set them back to 0 to silence the debug information.

It seems that somehow adi_menu is still essentially working off the original version he did for me in 2019, if that is at all possible. The 1.5.0 hasn’t overwritten tables in the database or something.

It should be using the new version. It’s not markedly different and the only table modifications it makes are in the one’s adi has added with successive versions of the plugin. The plugin essentially checks if parts added in later versions are actually in the database and upgrades accordingly. Once upgraded, it should then no longer show.

detail · 2024-02-08 19:17:18

Thanks, Jakob.

This is a great help.

Try later and let you know.

detail · 2024-02-09 01:16:19

I have switched on and off 50 times over the days without success!

But this might be something!

The debug shows this:

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_parent'

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_parent'

adi_menu_upgrade():

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_parent'

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_redirect_section'

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_alt_title'

SHOW FIELDS FROM tnz_txp_section LIKE 'adi_menu_redirect_category'

SHOW FIELDS FROM tnz_adi_menu LIKE 'clone_title'

SHOW TABLE STATUS LIKE 'tnz_adi_menu'

Then there’s the other section below:

TXP charset=utf8, collate=utf8_general_ci, engine=MyISAM ... adi_menu collate=utf8mb3_general_ci, plugin_engine=MyISAM
v1_0=, v1_1=, v1_4=, v1_4s=, v1_5t=1 adi_menu_section_list():

SUPPLIED: sections=''; exclude_list=''; override_exclude='1'

NORMAL QUERY (COMPONENTS): bulkq='page != '''; includeq='page != '''; excludeq=''; override_excludeq=''; sortq=' ORDER BY adi_menu_sort'

NORMAL QUERY (COMPILED): page != '' ORDER BY adi_menu_sort

VIRTUAL QUERY (COMPONENTS): bulkvq='1=1'; includevq='1=1'; excludeq=''; override_excludevq=''; sortq=' ORDER BY adi_menu_sort'

VIRTUAL QUERY (COMPILED): 1=1 ORDER BY adi_menu_sort

NORMAL SECTIONS: default te-araroa routes philosophy planning gear huts-campsites-te-araroa huts-abel-tasman-coastal huts-abel-tasman-inland huts-mt-richmond huts-marlborough huts-heaphy huts-kahurangi huts-nelson-lakes huts-coast huts-st-james huts-harper-pass huts-arthurs-pass huts-south-canterbury huts-hollyford-pyke huts-routeburn-greenstone-caples huts-milford-track huts-kepler-track huts-fiordland huts-rakiura huts burn-measureup kakapo-blog big-rivers-blog oteake-blog fidgit-blog rakiura-2022 alfred-2022 lion-2022 hopkins-2022 palmer-2022 canterbury-01-2022 branch-2021 monowai-2021 rakiura2021-blog south-coast-2021 abel-2021 arthurs2-blog jan-2020-blog anatoki-blog arahura-blog angelus-blog cascade-blog nlnp-2018 rakiura9 south-coast-blog mt-richmond-2-blog stafford-blog hanmer-blog n-to-s-blog lake-daniell mt-fell-blog-2 abel-tasman-blog-3 lewis-lewis kaikoura-boyle frew-toaroha balloon wakamarina-blog alpine-winter blue-lake-blog mt-fell-blog kahurangi-blog-3 kahurangi-2-blog ta-blog tin-range-blog mt-richmond-blog-3 milford-blog hollyford-blog arthurs-blog alpine-route-blog abel-tasman-blog-2 st-james-blog heaphy-blog abel-tasman-blog rakiura-blog kahurangi-blogs nelson-lakes-blogs mt-richmond-blog larapinta-blog blog ebooks reviews contact links privacy sitemap 

VIRTUAL SECTIONS: huts-campsites-te-araroa 

PREFS:
write_tab_select_indent: 1
write_tab_select_format: title
write_tab_select_default: 0
admin_pageby: 96

That should probably shine a light on the situation, but I’m holding a very dim candle!

At least I can confirm it is indeed version 1.5.

Jakob, I appreciate the amount of effort you are putting in to resolve this.

But it has to be one of the most important plug-ins for big or complex sites, so It would be great if it was properly resuscitated so anyone could use it.

jakob · 2024-02-09 09:55:11

The interesting bit is here:

TXP charset=utf8, collate=utf8_general_ci, engine=MyISAM ... 
adi_menu collate=utf8mb3_general_ci, plugin_engine=MyISAM
v1_0=, v1_1=, v1_4=, v1_4s=, v1_5t=1

The v1_5t=1 means that a reason for an upgrade has been found, and that’s what’s causing you to get the upgrade notice. The test that triggers that in the plugin is a mismatch between adi_menu and the overall database Collation and/or Engine settings and the upgrade routine tries to rectify that but that has obviously not happened for you. The additional debug shows that the engine settings match but the collation settings don’t.

My suggestion is you try and rectify that manually in the database, for example using your host’s phpMyAdmin or another similar tool.

First: verify you have a DB backup (you probably made this as part of one of the previous steps)
Open your database in your host’s phpMyAdmin (or your local setup if you’re using that)
In the left-hand list of table names, click on “adi_menu”. The table should show on the right-hand pane.
Click on the “Operations” tab at the top. You should see a section called “Table Options” (if you do this before clicking on the table, you will only see options for the entire database: make sure you clicked on the adi_menu table on the left-hand pane first).
There you will see a dropdown called “Collations”. Choose “utf8_general_ci” and check the tick box “Change all column collations”, then click “Go”. You will likely get a warning about possible data loss. Unless you are using some unusual characters in your menus that are not in the utf8 character set, I think that’s unlikely when going from utf8mb3 to utf8 (but you have your backup). If you are using some special characters in your menus, then we may need instead to bring the whole Textpattern database up to utf8mb4 instead: in that case stop here and get in touch.
If you open the “Console” at the bottom of the main panel, you can see the query that was just executed. It should be something like: ALTER TABLE adi_menu CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
If that all went well, click in the left-hand panel on the database name at the top, and on the right you will see a list of all the tables in textpattern. The table collation for adi_menu should now be utf8_general_ci.
Try now accessing the adi_menu panel. Hopefully you can now see the settings.

These instructions show the process with pictures.

Otherwise, send me a message via the forum and we can see if we can find an overlap in our time zones to resolve it.

detail · 2024-02-10 05:52:17

Thanks, Jakob.

That’s about the clearest instructions for dealing with a new task I’ve ever seen!

Seems quite straightforward. And the pictures in the eukhost help clarify!

However, I’ve been busy helping coordinate an imminent building project in the remote hills, and it’s Saturday evening.

Might be better to start freshly tomorrow morning. Should be able to do this myself.

detail · 2024-02-11 06:39:58

Still some issues with this, Jakob.

The character set currently used on my website database is utf8mb3_general_ci. This is all running okay with the six websites in the database.

Seems utf8 has been depreciated and replaced with utf8mb3. The current release is utf8mb4.

This might be the issue I have been experiencing.

For those people still arriving at this question in 2020 or later, there are newer options that may be better than both of these. For example, utf8_unicode_520_ci.

All these collations are for the UTF-8 character encoding. The differences are in how text is sorted and compared.

MySQL is currently transitioning away from an older, flawed UTF-8 implementation. For now, you need to use utf8mb4 instead of utf8 for the character encoding part, to ensure you are getting the fixed version. The flawed version remains for backward compatibility, though it is being deprecated.

So, utf8 is not able to be chosen in the 100? long list of character sets on the drop-down menu of the collation menu in the database.

When I looked at the database, the adi_menu character set was shown as latin1_swedish_ci for some reason. No other tables were. Everything else was utf8mb3_general_ci.

I changed these to utf8mb3_general_ci in the database, but the same debugging issues remained.

I guess I need to change the character set on the plug-in code instead of in the database to get it all to collate correctly.

I found five instances of reference to UTF8 in the dev branch code.

Line 1654   (adi_menu_breadcrumb)      'escape'               => '',                  // escape HTML entities in section titles: "", "html", "htmlspecial" (default="" - htmlentities/ENT_COMPAT/UTF-8)

Line 1688     (adi_menu_info)        'escape'           => '',        // escape HTML entities in section titles: "", "html", "htmlspecial" (default="" - htmlentities/ENT_COMPAT/UTF-8)

Line 2512  adi_menu_htmlentities. //        "" = default: ENT_COMPAT,'UTF-8'

and Line 2522         $string = htmlentities($string, ENT_COMPAT, 'UTF-8');

Line 2555 (function adi_menu_htmlentities)  'escape'                   => '',               // escape HTML entities in section titles: "", "html", "htmlspecial" (default="" - htmlentities/ENT_COMPAT/UTF-8)

Can I just change these five instances from UTF-8 to UTF-8MB3?

This might need to be addressed in the next version of adi_menu.

jakob · 2024-02-11 09:33:44

@devs: please see below for comment about Textpattern’s collation detection.

detail wrote #336589:

Still some issues with this, Jakob. … The character set currently used on my website database is utf8mb3_general_ci … So, utf8 is not able to be chosen in … the drop-down menu of the collation menu in the database. Seems utf8 has been depreciated and replaced with utf8mb3. The current release is utf8mb4.

Hmm, sorry to hear it wasn’t so straightforward. utf8 is an alias for utf8mb3, which probably explains why you don’t see it in the drop-down. As such, however, utf8mb3 is likewise expected to be deprecated (see here and the highlighted note here).

When I looked at the database, the adi_menu character set was shown as latin1_swedish_ci for some reason. No other tables were. Everything else was utf8mb3_general_ci.

That your table uses the latin1 character set is probably because your site has been around for a long time, and while the other tables have since been converted in previous Textpattern upgrades, the plugin’s table was not. Interesting, however, that the debug information says utf8 rather than latin1 – that bit is detected by textpattern, and I’m guessing you have $txpcfg['dbcharset'] = 'utf8'; set in /textpattern/config.php. Textpattern then makes the assumption that the collation is utf8_general_ci and not what it actually is.

I changed these to utf8mb3_general_ci in the database, but the same debugging issues remained.

You say the debug information says the same as before, although you have now made the collations match. I’m guessing Textpattern’s assumption that charset: utf8 = collation: utf8_general_ci may be why you’re still seeing the “mismatch” upgrade notice.

@devs: Is this assumption still appropriate today? Does this perhaps need updating?

I guess I need to change the character set on the plug-in code instead of in the database to get it all to collate correctly. I found five instances of reference to UTF8 in the dev branch code. Can I just change these five instances from UTF-8 to UTF-8MB3?

No. Aside from the fact that htmlentities doesn’t offer that as an option, the lines you mention apply to something different. Both the checking routine and the upgrade routine in the plugin do not use hard-coded collation values but detect what textpattern says is the database value and what a mysql query returns for the status of the adi_menu table.

This might need to be addressed in the next version of adi_menu.

Before replying to your earlier post, I tried deliberately changing the collation and encoding of the adi_menu table on a test site yesterday, and then running the plugin with the debug options set to 1. I see the “upgrade notice” you saw on the adi_menu panel, and deactivating and reactivating the plugin does successfully run the upgrade routine.

——

Long story short: two ideas:

1. In your /textpattern/config.php add the line:

$txpcfg['table_collation'] = 'utf8mb3_general_ci';

That should override Textpattern’s assumption and may let you progress. If the rest of your tables are all set that already, that should pose no other problems (and you also have your backup, right ;-) )

2. Otherwise, for future compatibility, using utf8mb4 for the database and all tables in the database would be a better option. I’ve found trying to change that manually in the database is sometimes fiddly. I think there was a plugin for that somewhere, but I need to look for it. Otherwise, if you’re willing to let me have a go, you could share it with me, and I can try converting it for you.*

EDIT: rvm_utf8_to_utf8mb4 was the plugin I was thinking of. Read the help first, and make sure you have that DB backup. If it all goes well, you also need to make sure that you then have $txpcfg['dbcharset'] = 'utf8mb4'; in config.php (and remove the $txpcfg['table_collation'] entry should you have added it previously).

*If you want to try it yourself, manually switching from utf8(mb3) to utf8mb4 works okay most of the time using the drop-downs to manually change the settings in phpMyAdmin (or Sequel Ace on the Mac if you use that). The fiddly bit is that you get an error when changing the all-important Body and Title rows in the textpattern table. As Oleg describes here, they use a common index and thus have to be changed together in a single DB query. For that, follow Oleg’s example but with COLLATE utf8mb4_unicode_ci instead of swedish. Do this in the “SQL” tab of phpMyAdmin or the “Query” pane of Sequel Ace:

ALTER TABLE textpattern MODIFY Title VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
    MODIFY Body MEDIUMTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

detail · 2024-02-11 22:13:23

Thanks Jakob!

Magic!

I did the config addition $txpcfg['table_collation'] = 'utf8mb3_general_ci'
and it worked immediately.

The adi_menu menu is working perfectly once again.

Hooray!!

Might need a little time to celebrate with the utf8mb4 upgrade, but it looks like it will be faster and, as you said, future-proof, for a little while.

Have learned a lot with this issue.

Thanks again.

jakob · 2024-02-11 23:11:35

Cool! Glad that aspect is resolved.

So, it turns out Textpattern makes an assumption what the table collation is based on the charset set in config.php rather than what a mysql status query would return. adi_menu attempts to align the two but in your case can’t (because the collation option is not available?) and gets stuck in the upgrade window. The resolution here is to override Textpattern’s assumed utf8_general_ci with an own specified collation. I’m not sure how common that situation will be.

Bloke · 2024-02-12 00:20:32

If there’s something we can do here in core to make fewer assumptions about collation mixes for a smoother ride, I’m all eyes.

jakob · 2024-02-12 07:27:53

Moderator notice: The follow-up question has been split off into a new thread here.

Textpattern CMS

Textpattern CMS support forum

#361 2024-02-08 11:12:57

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

detail wrote #336580:

#362 2024-02-08 19:17:18

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#363 2024-02-09 01:16:19

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#364 2024-02-09 09:55:11

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#365 2024-02-10 05:52:17

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#366 2024-02-11 06:39:58

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#367 2024-02-11 09:33:44

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

detail wrote #336589:

#368 2024-02-11 22:13:23

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#369 2024-02-11 23:11:35

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#370 2024-02-12 00:20:32

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

#371 2024-02-12 07:27:53

Re: adi_menu – section hierarchy, section menu and breadcrumb trail

Board footer