smd_access_keys: secure, limited access to content

sacripant · 2011-07-18 13:40:30

I received your patch, thank you.
Some new feedback:

As I can not lock a section, I switched my tests to the context category: http://site-url/category/file/a-file-category/
That makes more sense since the files to be transmitted will necessarily be classified with the categories.

With the version 0.10:

I created a category file. I name it “private-file.”
In my default page I add:

<txp:if_category type="file">			
	<txp:smd_access_protect expires="0" trigger="mickey" force="1">
		<h1>Mickey</h1>
		<p>::: trigger : <txp:smd_access_info item="trigger"/>:::</p>
		<p>::: page : <txp:smd_access_info item="page"/>:::</p>
		<p>::: expires : <txp:smd_access_info item="expires"/>:::</p>	
	</txp:smd_access_protect>

	<txp:smd_access_protect expires="0" trigger="donald" force="1">
		<h1>Donald</h1>
		<p>::: trigger : <txp:smd_access_info item="trigger"/>:::</p>
		<p>::: page : <txp:smd_access_info item="page"/>:::</p>
		<p>::: expires : <txp:smd_access_info item="expires"/>:::</p>	
	</txp:smd_access_protect>
</txp:if_category>

If I created an access-key for the donald trigger http://site-url/categorie/file/private-file/donald/key...
I have no more 404 error… But nothing, juste Home page. May be a consequence of the install in a sub-folder.

With version 0.11:

I kept the same code and just made the update plug’in.
And this time it’s OK: I access the protected content for trigger “Donald”.

————————————————-

I come feedback to post the test in the context section.
But for the project I’m working now, I’ll use the context category.

jayrope · 2011-07-19 21:27:23

I am trying to use smd_access_keys to inject time-limited, obfuscated links to mp3 files into an mp3 player plugin (jnm_audio).

I want the player (and a download link) to work with an obfuscated mp3 link for a limited time only to protect the links to the files
against those sites, that sell bot-harvested download links to customers.

Can i go to mysite.com/mp3player, generate the obfuscated play/download link upon entering the page and use it right away on that exact same page? I can’t seem to be able to do this.

If i do <txp:jnm_audio url="."><txp:smd_access_key url="full/path/to/file.mp3" /></txp:jnm_audio>
then the original path shows up within the player code, added with the desired “/2f3336fb9ef085f3b3eb6e7ddcb88ee025283a0d/4e25f215” in the end, but well:
- The original path is not obfuscated, and the complete link does neither play within the player code, nor does it let me download the file directly.

What am i doing wrong? Would really appreciate a short road map.
My mp3 files do reside in the files folder now, by the way, but that could be changing. They also have not been uploaded using the file upload menu, as they are simply way too big (100 mb++).

Thank you very much in advance, and excuse my slowlearnerism, please…

EDIT: Great idea of a plugin in any case!

Last edited by jayrope (2011-07-19 21:41:06)

Bloke · 2011-07-20 09:22:04

jayrope wrote:

I am trying to use smd_access_keys to inject time-limited, obfuscated links to mp3 files into an mp3 player plugin (jnm_audio).

The plugin does not obfuscate links. I figured there was no need because if the resource is protected by smd_access_keys then it is not possible to get access to the file without the key in the URL (providing they’re in a non-web-accessible location or your .htaccess forbids direct download via site.com/files/name-of-file.mp3). You just get a 4xx message.

What you probably don’t want to do is automatically generate valid keys to the resource inside the jnm attributes. This is simply because a bot could visit the page and retrieve a valid key, then visit the resource using that valid key and use it to obtain access. Even if you set the maximum number of downloads to 1 or 2 — which is sensible in this example — the bot could still download it, even though it would be fairly pointless to repost the link. Stripping off the access key portion of the link and posting the remains would also fail (again, if your .htaccess prevents direct access) because smd_access_keys would throw a 4xx message.

Is there any way you can modify the workflow such that the key is obtained via, I dunno, an AJAX call that then pops up the link embedded in the player? I don’t know of many bots that respond to js in that way (but I could be wrong). Or you could maybe e-mail the link off? Though that adds a step to the user’s workflow, which may not be desirable.

Can i go to mysite.com/mp3player, generate the obfuscated play/download link upon entering the page and use it right away on that exact same page?

It’s possible, though perhaps undesirable as mentioned above. If you do go that route, I would be tempted to do it in two steps. For example:

<txp:file_download_list>
   <txp:variable name="key">
      <txp:smd_access_key url='<txp:file_download_link />' trigger="file_download" />
   </txp:variable>
   <txp:jnm_audio url='<txp:variable name="key" />' />
   <p><a href="<txp:variable name='key' />">Direct download link</a></p>
</txp:file_download_list>

The timeout for file downloads is governed by a plugin preference. Default is 3600 seconds (1 hour to you and me!) After the hour is up, the keys on the page all expire. Revisiting the link gives you new keys, thus everyone gets their own key — including bots — but you have to be quick to repost them before they become invalid! I’d consider lowering it still further to, say, 15 minutes.

The downside to this approach, btw, is that you’ll get a lot of keys generated so you’ll need to keep on top of them by deleting them from the admin side interface.

To help with that I’m thinking of putting in an auto-delete option which purges expired keys after some configurable time has elapsed since expiry (e.g. a key is considered deletable 1 week after it has expired or its max download limit has been reached — that gives you time to check it in the logs before it disappears). There could be a few triggers for this to occur:

visiting the admin side Access Keys tab and clicking the purge button removes all expired keys
someone trying to access a resource with a key will remove only that key if its expiry window has been exceeded

I’d leave it up to you to decide which one(s) you wish to use. Since smd_access_keys v0.10 only has resource expiry (not key expiry: that is a 0.11 feature) then I can’t tell which keys have expired until a resource access is attempted. file_download keys however, are different: the expiry window is known because of the pref setting so these can be auto-deleted fairly easily.

I’ll have to refine the mechanics a bit because I’m not sure how feasible it is, but that’s the plan. How does that sound?

Last edited by Bloke (2011-07-20 09:25:28)

jayrope · 2011-07-20 12:46:07

Hi Stef,
if i use your example i still expose the generated key inside the variable “key” <txp:jnm_audio url='<txp:variable name="key" />' /> – and the original file path remains visible within the generated path. Also

A generated link holding the key on
http://domain.com/musicplayer
still looks like
http://domain.com/musicalities/test.mp3/test/5bdbc2abd391174b397dea9b1e207bd8c2cdb092/4e26b80d
and i get a 404 on direct access of this link whilst the mp3 player says “file not found”. Rrrr.

Note: http://domain.com/musicalities/test.mp3 does exist.
The directory “musicalities” is not protected otherwise.
(I will set up a .htaccess protection to test as soon as i figure out a working obfusctaed link – however using ONLY this to protect the files would still expose a)

What am i doing wrong there?

Of course idealistically the link should look similar to
http://domain.com/musicplayer/test/5bdbc2abd391174b397dea9b1e207bd8c2cdb092/4e26b80d

Looking at your current URL/key scheme i think there is not much missing, besides what i needed was, that the full download path disappears from the key’d URL.

Note: I am deriving this whole idea from http://radioartnet.net/11/2011/07/05/samuel-beckett-words-and-music/
When you look into the source there you see that the mp3 link looks like this:
aHR0cDovL3VidW1leGljby5jZW50cm8ub3JnLm14L3NvdW5kL2JlY2tldHRfc2FtdWVsL3RoZWF0cmVfcGllY2VzL0JlY2tldHQtU2FtdWVsX1RoZWF0cmUtRm9yLVlvdXItTW90aGVyX1dvcmRzK011c2ljLm1wMw

I guess i have no clue how they encode this.

…

The timeout for file downloads is governed by a plugin preference. Default is 3600 seconds […] I’d consider lowering it still further to, say, 15 minutes.

If mp3 files are very long, they will probably still play from the browser buffer provided they downloaded to the player within the time limit. That’s good.
I will have to rethink the pros and cons of individual visitor key creation vs. timed-one-key-for-all-page-visitors creation. i am not expecting loads of visitors (yet), so individual keys will probably not endlessly pile up. But… i will eventually need to sync the key generation with asy_jpcache working at the same time. Sounds like fun, or maybe a cache clearance call from smd_access_keys to asy_jpcache upon creation of a new page key. Only if there is just one key on that page, oh my… ;)

Both outlined automated key deletion options sound good, and the outlined trigger methods. A user-configurable grace period before automatic deletion is nice, as it allows to eventually use the number of downloads in statistics, if the file downloads are not happening from the default files folder.

I’ll have to refine the mechanics a bit [edit jrp: for v. 0.11] because I’m not sure how feasible it is, but that’s the plan. How does that sound?

=> Thank you again for the detailed answer. Can’t wait for 0.11. Only need to fix the link issue…

Last edited by jayrope (2011-07-20 12:47:47)

jayrope · 2011-07-20 13:26:09

What about the option to use an existing asy_jpcache key as a (part of a) key for smd_access_keys?

I’m still thinking, what kind of functionality this could implicate, however, deletion times for both plugins could be synced, when necessary.

Looking around in the jpcache code, will post, if i find something of use.

Reminds me: What happened to smd_remote_file / smd_secure_file ? Found this mentioned here.

Bloke · 2011-07-20 13:30:49

jayrope wrote:

the original file path remains visible within the generated path.

Yes. smd_access_keys does not obfuscate links.

i get a 404 on direct access of this link whilst the mp3 player says “file not found”. Rrrr.

Right. I should explain better. The access key you create is only useful in two situations:

to protect a Txp /section/article (or, in 0.11+, /section URLs are permitted)
to protect Txp file_download URLs

The key itself is just a string of junk: it doesn’t protect the resource itself. That’s the job of <txp:smd_access_protect>. If your musicalities section existed in Txp you could wrap your page content with <txp:smd_access_protect force="1"> tags and then nobody could gain access to it without a key that was generated for that URL.

At the moment, you are trying to protect a resource that is outside Txp’s control so nothing is working. Txp needs to be in the loop somehow. Two ways you can do this with non-web-accessible content:

Method 1

Move musicalities out of webroot
Set the location of your File path (in Advanced Prefs) to the musicalities dir
After uploading your files to that protected area, go to Txp’s Files tab, and under the ‘Existing files’ dropdown, select the file you uploaded and create a real Txp file_download URL for it
Use that site.com/file_download/N/file-name.mp3 as your URL to protect in smd_access_keys

Benefits of that approach:

The real URL path is obfuscated for you as it’s ‘covered’ by the /file_download URL structure
Since the dir is out of web root, no direct access is possible — no need to worry about .htaccess

Method 2

Create a Txp section from which to serve your audio content
Inside your page template (in v0.11, which you can have now if you like: I’ve got one thing to add which isn’t critical) or inside an article in that section (for v0.10) you can put <txp:smd_access_protect> tags around your jnm_audio tags
In the unprotected part of the Page/article, you can put <txp:smd_access_key> tags which allow people to create access keys to the article itself (it’ll do this by default without the url= attribute)
The clever bit is that you can set the trigger to be, I dunno, the ID of the file itself, or part of its name, or whatever — this info can be obtained inside a <txp:file_download_list> tag fairly simply. Thus your URLs might look like this: http://domain.com/protected/access/1/5bdbc2abd391174b397dea9b1e207bd8c2cdb092/4e26b80d (the /1/ is the file ID)

You can probably protect each jnm_audio individually by doing this:

<txp:file_download_list>
   <txp:smd_access_protect trigger='<txp:file_download_id />'>
      <txp:jnm_audio blah blah />
   </txp:smd_access_protect>
</txp:file_download_list>

Not sure if that’ll work, but it might. Failing that you can just generate an access key for the entire article and wrap all your jnm_audio tags inside it: one key to rule them all :-)

Further, if you also shift the musicalities dir out of web root, it doesn’t matter if your jnm_audio tags all reference them directly (as files, not URLs of course, since that will fail) because nobody can access them from the web.

I haven’t looked at radioartnet but I’ll take a gander and see if I can offer obfuscated URLs in a future version.

If mp3 files are very long, they will probably still play from the browser buffer provided they downloaded to the player within the time limit.

Yeah, I expect the access key only cares about the resource at the start of the download: if it takes a week to stream, it shouldn’t care.

Bloke · 2011-07-20 13:33:47

jayrope wrote:

What about the option to use an existing asy_jpcache key as a (part of a) key for smd_access_keys?

I’ve no idea about asy_jpcache. Never used it. But if you can get at the key you can use it as a trigger or inside smd_access_key’s attributes via tags-in-tags. Interesting to know if it works. Thanks in advance for any insight.

What happened to smd_remote_file / smd_secure_file

Remote file is still alive and (barely) kicking. I got partway through smd_secure_file and decided on smd_access_keys and smd_ipn as a better approach because it’s not just limited to files. Currently the other plugin is sitting on the shelf gathering dust.

Last edited by Bloke (2011-07-20 13:36:06)

sacripant · 2011-07-21 17:20:17

Hi Steve,
I continue my tests, but I’m slowed down by some strange behavior of context category.

jayrope · 2011-07-25 12:45:04

Hi Stef, same here, am slowed down by having to work on another project. However, i implemented your last suggestions and they did help a lot. Unfortunately i ran into a totally different issue, which mainly has to do with sending out the proper mime-type headers on access-keyed mp3. Will have to analyze that in detail however. It is possible, that jpcache was the originator of the problem. More later. Thank you!

sacripant · 2011-09-20 18:49:25

Hello Steph,
My feedback for smd_access_key 0.11. Better late than never

the requirement

The user must provide documents to some of these clients. 1 or 2 times a year. No more.
The client does not need a login and password to access just a once a year on a site to download some documents.
So I chose the principle of access keys.

For the user

Create a file category with the name of his client.
- This category should be stored as a child of the category “private”.
- He can create as many customer category they want.
Import the files to be transmitted in the category
Generation of the access key only from the back office and transmitted to the client by mail.

To simplify the process for the user, I searched for a solution that is always the same url (url of the category private) and the trigger is the name of the category (ie the client).

url : site_url/category/file/private/
trigger : category name

1st problem: the translation of urls for categories.

In French, the URL is
site_url/catégorie/file_context/fichiers-prives
But this url does not work and returns a 404 error
We must replace the é with %C3%A9
site_url/cat%C3%A9gorie/file_context/fichiers-prives

This is a bit complicated to explain to my client.

The file “lang FR” has been updated a few days ago and now the url is:
site_url/catégorie/fichier/fichiers-prives
This update may cause redirect problems… I should explain that to my client…

2nd problem: the display of errors (smd_access_error)

I think the plug’in was not designed to operate in a loop category.

<txp:category_list type="file" parent='<txp:category />' exclude='<txp:category />' break="" >
	<txp:smd_access_protect expires="172800" trigger='<txp:category />' force="1">
		<txp:file_download_list category='<txp:category />' sort="created desc" limit="999" break="" />
	<txp:else />
		<txp:output_form form="access-keys-errors" />
	</txp:smd_access_protect>
</txp:category_list>

Whatever the trigger used in the url (the first, second, third loop) is always the output_form “access-keys-error” of the first loop will be used.

For the good output_form be used with the good trigger, I had to juggle with variables (it’s understandable? not sur).

My code (certainly crooked but it works)

cat_list:

<txp:category_list type="file" parent='<txp:category />' exclude='<txp:category />' break="" >
	<txp:smd_access_protect expires="172800" trigger='<txp:category />' force="1">
		<h2 class="clear"><txp:category title="1" /></h2>

		<txp:variable name="maximum" value='<txp:smd_access_info item="maximum"/>' />
		<txp:if_variable name="maximum" value="0">
			<txp:variable name="max-value" value="∞"/>
		<txp:else />
			<txp:variable name="max-value" value='<txp:smd_access_info item="maximum"/>' />
		</txp:if_variable>

		<p>Cette clef d'accès expirera le <strong><txp:smd_access_info item="expires" format="%d %m %Y à %H:%M:%S"/></strong> <br />
			Nombre de connection(s) autorisée&nbsp;: <strong><txp:variable name="max-value" /></strong> <br />
			Nombre de connection(s) enregistrées&nbsp;: <strong><txp:smd_access_info item="accesses"/></strong> </p>

		<txp:variable name="style" value="last" />
		<txp:file_download_list category='<txp:category />' sort="created desc" limit="999" break="" />

		<txp:variable name="error" value="yes" />
	<txp:else />
			<txp:output_form form="access-keys-errors" />
	</txp:smd_access_protect>
</txp:category_list>

output_form ‘access-keys-errors’:

<txp:smd_if_access_error type="smd_akey_err_expired, smd_akey_err_limit, smd_akey_err_bad_token, smd_akey_err_missing_timestamp">
	<txp:variable name="keys-error" value="yes" />
<txp:else />
	<txp:variable name="keys-error" value="no" />
</txp:smd_if_access_error>

<txp:if_variable name="keys-error" value="yes">
	<txp:smd_if_access_error>
		<h2 class="clear">Erreur <txp:smd_access_error item="code"/></h2>
		<txp:smd_if_access_error type="smd_akey_err_forbidden">
		   	<h3><txp:smd_access_error item="message" break="br" message="L'accès à cette page est vérouillée."/></h3>
		<txp:else />
		   <txp:smd_if_access_error type="smd_akey_err_expired, smd_akey_err_limit">
		      	<h3><txp:smd_access_error item="message" break="br" message="Votre clef d'accès a soit expiré soit atteind son nombre de connexion maximum"/></h3>
			<txp:else />
				<txp:smd_if_access_error type="smd_akey_err_bad_token, smd_akey_err_missing_timestamp">
			      	<h3><txp:smd_access_error item="message" break="br" message="Votre clef d'accès n'est pas conforme, vous ne pouvez accéder à cette page"/></h3>				
				<txp:else />
					<h3>Une erreur s'est produite pour accéder à cette page</h3>
				</txp:smd_if_access_error>	
		   </txp:smd_if_access_error>
		</txp:smd_if_access_error>
		<p>N'hésitez pas à <a class="go-contact" href="#contact">nous contacter</a> si vous avez un problème pour accéder à vos informations</p>
	</txp:smd_if_access_error>
	<txp:variable name="error" value="yes" />
</txp:if_variable>

mrdale · 2011-09-20 20:43:12

Hi Stef,

Been meaning to ask.

Could this plugin be used to handle gift-certificate or coupon-code style functionality?, ie code, displays content once only, then the content is no longer available to that user?

jakob · 2011-09-20 21:17:12

@Dale, yes it can :-) The access key URL is a long string but you can set either access duration or number of accesses. After that the resource is no longer accessible.

Textpattern CMS

Textpattern CMS support forum

#13 2011-07-18 13:40:30

Re: smd_access_keys: secure, limited access to content

#14 2011-07-19 21:27:23

Re: smd_access_keys: secure, limited access to content

#15 2011-07-20 09:22:04

Re: smd_access_keys: secure, limited access to content

#16 2011-07-20 12:46:07

Re: smd_access_keys: secure, limited access to content

#17 2011-07-20 13:26:09

Re: smd_access_keys: secure, limited access to content

#18 2011-07-20 13:30:49

Re: smd_access_keys: secure, limited access to content

#19 2011-07-20 13:33:47

Re: smd_access_keys: secure, limited access to content

#20 2011-07-21 17:20:17

Re: smd_access_keys: secure, limited access to content

#21 2011-07-25 12:45:04

Re: smd_access_keys: secure, limited access to content

#22 2011-09-20 18:49:25

Re: smd_access_keys: secure, limited access to content

the requirement

For the user

1st problem: the translation of urls for categories.

2nd problem: the display of errors (smd_access_error)

My code (certainly crooked but it works)

#23 2011-09-20 20:43:12

Re: smd_access_keys: secure, limited access to content

#24 2011-09-20 21:17:12

Re: smd_access_keys: secure, limited access to content

Board footer