smd_remote_file: Manage remote URL downloads via TXP's Files tab

marazmus · 2008-08-20 12:15:28

Thanks for clarifying. I think we are on the same page. Yes, it would be cool to do this. Yes I’d like to do it now I’ve thought about it these past few minutes. Yes, if I can figure it out, I will do it. When that takes place I currently cannot say, sorry. I have this new plugin hanging round my neck which desperately needs finishing, some updates to smd_if planned and I really want to revisit smd_query and smd_vars to get them roadworthy. After that I’ll probably look at this.

Okay, I’m not boring you, this is only “feature request” :)

True, I could put it as a disclaimer, but since TXP itself disallows editing of the filename I should probably do likewise. Will think about it.

In the ideal case, there should be an option in Advanced Settings. Something like “Allow changing paths in file links (on Files tab)”.

Yes. But with a lot of files, this is cumbersome. The ability to add more than one URL and pick randomly was a nice idea I had but it’s incomplete. I did start work a long time ago on a companion plugin called smd_ping that checked remote files for a (set) timeout period and if no response was received would take one action or another. I intended to either use that or build something similar into this plugin so the choice of server was not entirely random but based on a fixed, random, start point and a test for “is the server up”. If not it’d try another random server until it found one that worked or exhausted the list. Yet again, it never made it past dev status before I moved on. One day…!

“Is file live?” ping feature will be really cool :) You can add this as option in smd_file_download_link. Something like choose=“live” :)

Okay, just crossing fingers and waiting for :) Thank you.

kostas45 · 2009-04-27 10:32:29

Hi Stef,

This is a (I hope not pushy) reminder about the bug described as:
URL does not resolve (even if the file is accessible via http://) when the remote file is actually not remote, but at same domain TXP is installed.
Thank you for all your great work :-)

-Kostas

jelle · 2009-06-16 22:10:58

wow :) great combo with dropbox

Last edited by jelle (2009-06-16 22:11:11)

feytus · 2009-07-12 15:21:37

I tried this a couple of days ago hotlinking to some files on 2shared.com, which all worked great – for two days, when 2shared simply decided to delete the files. So, I moved them all over to archive.org, got the hotlinks, like this; http://www.archive.org/download/marsmelons_records/MMx009_Lesbian.zip and added it in the files-tab. But this doesn’t work for some reason – get a 404 trying to download it – even though I did the exact same thing when I uploaded from 2shared. Any ideas on why?

Here’s the link that should be working; http://www.marsmelons.com/file_download/68/MMx009_Lesbian.zip.link

Last edited by feytus (2009-07-12 19:50:20)

Bloke · 2009-07-12 19:54:39

feytus wrote:

get a 404 trying to download it

Did you delete the original .link files in the Files tab before adding the new URLs? If you did not do that then you may still be using the old links.

What happens is that each time you add a URL with the same filename at the end of the URL, it is added to a list of similar files. Thus you can host the same file at different providers and the link you are given when you clidk a file_download inTXP is randomly chosen from all the ones you have added. If you did not delete the old one .link, there is a chance that roughly 50% of the time you’ll get a 404 when it tries to serve the (deleted) file from 2shared.com.

The next version of the plugin — which I have in alpha alpha alpha testing (i.e. most of it hasn’t been written!) — actually shows you a list of all the URLs against each file so you can edit them directly, delete them, reorder them, etc. Until that version hits the streets you have a few options:

in your FTP client, navigate to your files directory and download/edit the .link files directly. Open themn in your favourite text editor and delete any old URLs to 2shared.com, then re-upload the .link files, overwriting the original ones
Use smd_file_download_link and add the choose=2 attribute so it forces the plugin to pick the 2nd URL in the list, which should always be your archive.org entry. This overrides the default ‘load balancing’ system that randomly chooses a remote file

Hopefully that will fix it. If none of that is any use, please give me some more details on what you see in a typical .link file (one that gives a 404) and I’ll se if I can diagnose it further.

feytus · 2009-07-12 20:39:17

Thanks for your reply, man. I tried deleting all the old files now, and am simply using the lesbian.zip (strange example, I know) as test for this. It still doesn’t work though. When accessing the .link file through an FTP client, all the info it holds is the link to the download;

http://www.archive.org/download/marsmelons_records/MMx009_Lesbian.zip

Which does work in itself, just not through your plugin. Very, very strange. And this is the link that should in theory be working;

http://www.marsmelons.com/file_download/69/MMx009_Lesbian.zip.link

Last edited by feytus (2009-07-12 20:40:59)

Bloke · 2009-07-12 21:02:38

feytus wrote:

It still doesn’t work though.

Freaky.

OK, let’s run a test or two. I can visit this URL: http://www.marsmelons.com/file_download/1 and I receive the file. I assume this is because it is a regular file hosted in your Files dir. The same with 2 and 3. But if I try 4, 5, 6… 69 and so on, I receive 404s for all of them. This implies one of two things:

those entries are not in your database
the plugin is broken (which may cause issue 1!)

So, please check a couple more things. First, have a look in MySQL at the contents of the txp_file table. Do you see entries for IDs 4, 5, 6 and so on? Do they look normal to you?

If you see there are files there, try temporarily disabling smd_remote_file and visit http://www.marsmelons.com/file_download/69. You should be served the .link file directly. If you still get a 404 then there’s a chance the file itself has not been created or it has been created with insufficient permissions. A head scratcher for sure…

feytus · 2009-07-12 21:17:42

There’s entries in the mySQL database for all the files that I still have (there’s a huge gap in files between 3 and 27, I’m not quite sure why, but I think it was because of some plugins I fooled around with earlier, but decided to delete). All the entries look normal, as far as I’m concerned (I don’t know much about MySQL in all honesty), they have a status “4”, and trying the method you used, I can download all the existing files.

I de-activated your plugin, and clicked the 69, and it asked me to download the .link file – so maybe this implies that there’s something wrong with the plugin? Maybe I installed it incorrectly? Not quite sure how I could’ve done that though.

Edit; Don’t know if this clarifies anything, but I tried your plugin using another hotlink, from another server, and it worked fine. This one;
http://www.dump.no/files/1723776c1e87/MMx001_Amor_Fati.zip

So maybe it’s something to do with archive.org? Hotlinking directly from them, through html, works great, doing it as a temporary thing right now.

Last edited by feytus (2009-07-12 21:24:42)

Bloke · 2009-07-12 21:57:14

feytus wrote:

I de-activated your plugin, and clicked the 69, and it asked me to download the .link file – so maybe this implies that there’s something wrong with the plugin?

Definitely. Sounds like it’s something to do with the way it redirects to the remote server or the way it checks to see if the remote file is available before serving. It does a check before downloading it to see if it’s being tricked (e.g. some hosts supply an image file to say ‘file not found’ instead of simply returning a correct 404 status code). It appears that archive.org are doing some additional (302) redirection with their so-called ‘direct’ URLs.

Unfortunately, this is where you need to be a bit of a Sherlock. My usual MO is to fire up the awesome LiveHTTPHeaders extension in Firefox, click a TXP remote link and watch where it’s being redirected. In this case, the link is 404ing before we get anywhere, so I tried the direct link:

http://www.archive.org/download/marsmelons_records/MMx009_Lesbian.zip

Guess what? It is being redirected to:

http://ia301515.us.archive.org/0/items/marsmelons_records/MMx009_Lesbian.zip

My guess is that if you put that 2nd URL in the .link file it’ll work fine. Please try it to prove it. But of course you don’t really wanna be doing that for all your files because it’s a pain in the butt, so smd_remote_file needs to go back to school and learn to take additional server-side redirects into account. I’ll see what I can do, but I can’t promise the solution will be “very soon” unfortunately. Thanks for pointing this bug out.

feytus · 2009-07-13 09:35:59

Oh yes, that last link you put there is working. :) How did you find that though? I’m willing to go through the trouble to have a counter on my downloads (as my label has just launched, I guess this is the time most people will download stuff). Edit; man, I just got up, missed the part about ‘LiveHTTPHeaders’, I will check that plugin out! Thanks for the tip! And no worries about the bug, it’s a great plugin none the less. :)

Last edited by feytus (2009-07-13 09:38:09)

feytus · 2011-01-29 18:00:13

I am using Textpattern for my netlabel (www.marsmelons.com), but for my two last releases (a mix and an EP), no downloads have been recorded, even though I know people have downloaded them. I am hosting these files on a remote server and integrating them in textpattern with smd_remote_file. Could the problem be on this plugins end, or might there be some error in my textpattern setup?

Everything I have hosted on a remote server before has been counted and working well (and continue to do so), so its a bit strange. Any ideas on what might be wrong? Any help much appreciated. :)

robhert · 2011-11-13 18:32:18

Hi! Seems like smd_remote_file is not working with TXP 4.4.1 I’ve just installed and I can’t see the input form field in the Files tab.

Regards,

Textpattern CMS

Textpattern CMS support forum

#49 2008-08-20 12:15:28

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#50 2009-04-27 10:32:29

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#51 2009-06-16 22:10:58

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#52 2009-07-12 15:21:37

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#53 2009-07-12 19:54:39

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#54 2009-07-12 20:39:17

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#55 2009-07-12 21:02:38

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#56 2009-07-12 21:17:42

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#57 2009-07-12 21:57:14

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#58 2009-07-13 09:35:59

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#59 2011-01-29 18:00:13

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

#60 2011-11-13 18:32:18

Re: smd_remote_file: Manage remote URL downloads via TXP's Files tab

Board footer