smd_redirect: Is there a limit number of redirects I can create?

SteveG · 2021-04-29 06:44:51

Hi, lovely plugin creators and mostly Bloke (if you see this post). I have been using smd_redirect for more than 2 months now. An astonishing useful plugin that allows me to create Textpattern article links and redirect to everything I post on other websites like youtube or Reddit. And with smd_section_roles , I create a section and an account dedicated to creating redirected articles so they won’t interfere with my main account’s article list.

As I make all my online activities and everything I want to share into a redirect link on my site, the redirect list is becoming longer and longer, which makes me wonder if there’s a limitation of how many redirects I can create.

And also I would like to separate the redirect list into pages. Any suggestions on how to achieve it?

Last edited by SteveG (2021-04-29 06:45:13)

Bloke · 2021-04-29 07:55:40

Glad it’s useful. Yes there’s sadly a limit because the plugin cheats and uses the prefs table to store its content. What that limit is will depend on how big your redirects are, and how many you have. It smooshes them into an array, crushes them if it can then encodes them and puts them in a pref value. It’s ugly. I probably should use a dedicated database table instead one day, but as the plugin’s not optimised for handling many redirects, you might be better off finding a different solution.

If you get close to the limit, I’ve just realised that it’ll break spectacularly. If it can’t store the entire encoded chunk it won’t be able to read them back out so it’ll essentially destroy all your redirects. That’s a problem I’ll need to address. Please periodically back up your txp_prefs table just in case (rss_admin_db_manager will help here). Then at least you can roll back and not lose all your redirects.

Leave it with me and I’ll fix that somehow.

Regarding pagination, I’m not sure. It makes drag and drop reordering hard. The plugin’s not really designed for large swathes of redirects. You can use the search to filter the list if it gets too unwieldy but past that I can’t think of a neat solution right now.

Last edited by Bloke (2021-04-29 07:57:13)

jakob · 2021-04-29 08:26:52

I’m not sure I totally understand your setup so this may not apply, but if you want to have own-domain links that point to other sites, kind of like short links of your own, here are two further ideas:

arc_redirect works a lot like smd_redirect. It’s not as powerful in that it doesn’t support regex or priority ordering but it’s just fine for redirecting one static link to another, either within your site or from your site to another location. It uses its own database table and the regular Textpattern admin-side tabular display with pagination and search etc. so you can slice, sort and filter on the admin-side as usual. It’s great, for example, for making human-readable links to illegible dropbox folder links and the like.
There are open-source tools like YOURLS that you can use explicitly for managing links or creating your own short links (don’t be put off by the old looking site, it is kept up to date; someone else has made a nicer admin for it too).
You can then embed those links in your site manually. You’d typically install that in a subdomain to avoid clashing with Textpattern. If you want to automate that, there might be a way of pulling in links from its database automatically using rah_swap and smd_query. There are now various other open-source short-link tools on GitHub too but I’ve not tried them.

etc · 2021-04-29 09:59:00

SteveG wrote #329993:

the redirect list is becoming longer and longer

Does it mean regexp are not usable in your case? Then why wouldn’t you just employ custom fields for external URLs and construct your links accordingly?

SteveG · 2021-04-29 15:33:00

Thanks for all the suggestions!

@Bloke: I’ll back up the txp_prefs table while waiting for your update.

\@jakob and @etc: In my use case, I do need to create the actual article on my site. Because one of my organization’s Discord members wrote a robot that automatically posts about the new article in Discord chat when it’s created. Since the structure of Textpattern’s public article list is simple, the robot is fairly easy to write. Since I can customize the Textpattern’s article url, I can include some keywords in it to let the robot know if it’s a news/video/discussion/guides/etc and the robot will automatically store the information in its own database for its search function. And save the time to create different rules and monitor dozens of Youtube channels or Reddit users, my Textpattern site now is a hub for all kinds of information.

Bloke · 2021-04-29 16:37:36

*scratches head*

I’m still trying to wrap my head round what you’re doing here, sorry.

So when you publish an article on Textpattern – which incidentally triggers a callback called ‘ping’ that you can hook into and use to do whatever you want – there’s a Discord robot that takes your article URL and splits it into component parts that it uses for indexing its search. Is that right so far?

Thus, Discord users can search and find your content via the keywords that have been scraped from the url-only-title. Then go ooh wow, a new post from SteveG I’ll click to read it. They click and visit your article – which has the same permlink URL as it had when you created it.

So where do the redirects come in? Sorry if I’m being dim.

EDIT: If your post itself contains links to external sources such as Reddit and YouTube etc, these endpoints must be known for you to mention them in the article. So why can’t you just link directly to them, either in the body text via Textile, or (perhaps better) define each one in the Content>Links panel and tie them to the article so your template can spit out the correct set of links based on the article being viewed?

Your users in the section_roles plugin could be granted access to only the Links panel so they could harvest links and prepare them. As long as they ‘group’ the links somehow (maybe use the Sort field, as it’s rarely used, to hold the article id that you want the link to be associated with) then your template can construct the <txp:linklist> for you when the article is viewed. That way, you can add/remove links to articles on the fly by just changing the content in the Links panel and associating each with an article. If (ab)using the Sort field isn’t your idea of fun, jakob has a plugin that adds custom fields to links so you can just make your own field to hold the links back to the article.

Last edited by Bloke (2021-04-29 16:42:04)

SteveG · 2021-04-29 18:05:03

Bloke wrote #330003:

So when you publish an article on Textpattern – which incidentally triggers a callback called ‘ping’ that you can hook into and use to do whatever you want – there’s a Discord robot that takes your article URL and splits it into component parts that it uses for indexing its search. Is that right so far?

Thus, Discord users can search and find your content via the keywords that have been scraped from the url-only-title. Then go ooh wow, a new post from SteveG I’ll click to read it. They click and visit your article – which has the same permlink URL as it had when you created it.

Yes, that’s how it works. The thing is I’m not only posting on the Textpattern site but also on Youtube, Reddit and etc. So for example, if it’s a video when they click on the Textpattern permalink URL, it’ll redirect them to Youtube. I thought about putting the links in the article body, but why not just send them to Youtube directly if I can. This saves users time to land on the article page first then go somewhere else while the robot can still get all the conveniences from the fully customizable URL. And I get a nice archive list of all things I wrote and share on various websites in one place.

Last edited by SteveG (2021-04-29 18:05:30)

Bloke · 2021-04-29 19:38:18

So you’re creating articles inside Textpattern that serve no content purpose other than to redirect visitors to external sites? There has to be a better way! :)

How about this:

Imagine the scenarios where you want to publish something like a video. You do so on your external site. You take the URL of said video and create a Textpattern Content>Link out of it in a ‘video’ category. The link name is what you want the Textpattern URL to be itself. So, for example cat-vs-cactus. The link itself is the endpoint where you want people to end up youtube.com/watch?v=12345998. Optional description if you wish.

So far so good.

Now, what you do is create a bunch of Textpattern sections with the same name as your Link catgeories. So you’d make /video, /music, etc. Each of those sections uses the same page template. That template simply grabs the section name and fashions a link list out of it:

<txp:linklist category='<txp:section/>' />

You can use a form or a container to format the links how you like. But each one, when clicked, will go to the designated endpoint. Thus you have a neat archive of all things you’ve posted – separated by ‘category’ (section) which is nice and easy to digest. Your frontpage template could list them all so you have a complete list of everything you’ve posted in all categories.

Now comes the clever bit.

When someone clicks from an external source to view your-txp-site.com/video/cat-vs-cactus, Textpattern will (rightly) throw a 404. But it calls the error_default page. In the <head> area of that template, you check the section. If it’s one of your section/category mapping sections, it inspects the URL it is given. And if it is given a link name that matches one of the links in that category, it simply redirects to the corresponding link. If not, it carries on and throws a proper 404.

This is made slightly tricky by the fact that Textpattern can’t (for reasons I’ve not fathomed nor ever been inclined to change until right this minute) filter links by name. So you have to go round the houses a bit and iterate over the links in your chosen category until you find one that matches the remainder of the URL. But when you do, you simply grab its URL and redirect. Like this:

<!DOCTYPE html>
<html>
<head>
   <!-- Grab the 1st and 2nd URL components: section/title -->
   <txp:variable name="this_section"><txp:page_url type="1" /></txp:variable>
   <txp:variable name="this_link"><txp:page_url type="2" /></txp:variable>

   <!-- If the intended section was one of the ones you wanted -->
   <txp:if_variable name="this_section" value="articles, video, music" match="any">

      <!-- Loop over the links in the category matching the section -->
      <txp:linklist category='<txp:variable name="this_section" />'>
         <!-- Grab the link name -->
         <txp:variable name="linkname"><txp:link_name /></txp:variable>
         <!-- See if the name matches the second part of the URL -->
         <txp:if_variable name="linkname" value='<txp:variable name="this_link" />'>
         <!-- YES! Redirect to the link endpoint -->
            <txp:header name="Location" value='<txp:link_url />' />
         </txp:if_variable>
      </txp:linklist>
   </txp:if_variable>
   ... rest of head section here for throwing regular 404s
</head>
<body>
   ... rest of 404 page...
</body>
</html>

Something like that anyway. Your mileage may vary. But anyone visiting your ‘fake’ link URLs is swiftly redirected to where you want them to end up. On the side – and for free – you get a categorised list of places you’ve written about, and an optional everything I’ve ever said frontpage list. And you don’t need to manage a separate set of redirects. One less plugin. Native solution.

Does that give you any food for thought?

EDIT: Updated to use <txp:header /> tag.

Last edited by Bloke (2021-04-30 08:37:12)

SteveG · 2021-04-30 02:51:28

Redirect the 404 page is a clever move, though I do have questions on how to trigger the robot if I add a new link but not an article. And also if I understand it correctly, I need to name the link the same as the “fake” url which is sad for not able to create the title differently.

What I’m doing now is, I create a section called “Game News” using “archive” as page style.

Then, for example, I post a video on Youtube. Then I create an article. The title is usually the original title from the video. I’ll write a brief description (basically what the video is about and why it’s worth sharing), modify the URL so the Discord robot can process it correctly. The url follows the rule of “month-day-website-category-keyword1-keyword2-….”

After I post the article, it will show up in the public page like this:

Then I use the redirect plugin to redirect the article url to the Youtube url, so when people click on the title it goes to Youtube directly.
In other cases, like if it’s a news article, game patch notes, or organization events, I just post a normal article; If it’s something I want to discuss, I’ll redirect the article to Reddit where I post the topic (usually under the game’s “official” Subreddit).

Last edited by SteveG (2021-04-30 03:01:14)

etc · 2021-04-30 08:29:38

The point is, you don’t need a plugin for a simple redirection:

<txp:if_custom_field name="url">
    <txp:header name="Location" value='<txp:custom_field name="url" />' />
</txp:if_custom_field>

But I may have misunderstood some subtleties as well.

Bloke · 2021-04-30 08:46:23

Then it’s even simpler. As etc says, the process becomes:

Create an article. The title can be whatever you like as it appears in your site.
Write body description if you like as a teaser that you can show on your landing pages (and for site search if you use it).
Edit the article URL so your robot can index it with whatever parameters you like. Remember that Textpattern now supports category1/category2 URL schemes on a per-section basis, and year/month/day so you could change to one of those and tweak your robot accordingly so it saves you having to put (say) the date in the URL title manually again.
In a custom field called Destination (or URL, as etc suggests), put the endpoint if you want to redirect when the article is viewed.

In your template that handles regular article flow, just detect if the Destination custom field is set, exactly as etc states above when viewing a <txp:if_individual_article>. If it is, use <txp:header> to redirect to the destination URL. If not, carry on and show the article as normal.

SteveG · 2021-04-30 08:58:23

etc wrote #330007:

The point is, you don’t need a plugin for a simple redirection:

<txp:if_custom_field name="url">...

But I may have misunderstood some subtleties as well.

WOW! I never thought that custom fields can do this! Actually, I never know what the custom field is for before you mentioned it.
But on the other hand, the good part of using the plugin is that I can manage all the redirects in one page without click into each article if something needs to be changed.

Textpattern CMS

Textpattern CMS support forum

#1 2021-04-29 06:44:51