rah_unlog_me // Remove your own hits from visitor logs

Hennie · 2012-03-29 10:24:39

Will there be support for wildcards in one of the next versions? I would like that very much.

Gocom · 2012-03-29 13:14:44

Hennie wrote:

Will there be support for wildcards in one of the next versions? I would like that very much.

Maybe. I mean yes.

Gocom · 2012-07-12 00:12:47

Rah_unlog_me v1.4 is now available. This release includes:

Changed: Moved embedded language strings to Textpacks.
Added: Finnish translation.
Added: Excluded IP list supports wildcard. % matches zero or more characters and _ exactly one character.
Now requires PHP5 or newer.

More info and downloads

This release makes Hennie’s requested wildcard excluding possible. Following, when used in the excluded IPs option field, would remove local IPs from logs:

::_%, fc00::_%, 10._%, 127._%, 172.16._%, 192.168.0._%

Gocom · 2012-09-10 15:32:48

Rah_unlog_me v1.4.1 released. Some minor clean up and improvements to the bundled installer.

More info and downloads

etc · 2012-09-11 11:01:49

Like it very much! It would be perfect if it had an option for not logging me at all, could you consider it, please? Something like this may work:

register_callback('rah_nolog', 'log_hit');

function rah_nolog($event, $step)
{
  if(.../*should  not be logged*/) {
    global $nolog;
    $nolog = true;
  }
}

Gocom · 2012-09-11 14:46:54

Oleg, that would be doable. Thanks.

The plugin does (or has done) processing on-demand only on admin-side to avoid adding any type of strain to the public side. IPs will only need to be removed from the logs when you access the panel. That’s the first change you will have to see the records. Given, simple exclusion might have enhancing effects to performance when Textpattern is set to resolve DNS.

Anyway, I’ve implemented preventing to the plugin as of the two latest commits 15f48f and 15b630d.

Preventing does have its few drawbacks. For instance, the preventing can only work consistently for the defined list of IPs. While there is a login cookie for the public side, which can be used to prevent logged in users, I believe the session is only available on a single-site install when both the admin-side and the site itself reside under the same domain.

Then there is that minor fact that the list of IPs support wildcards. Wildcard addresses won’t be prevented, not unless IPs were filtered using MySQL, in which case there is most likely going to be more queries than originally. One for creating temporary record of the current IP and one matching that record using like pattern. There is the option to imitate LIKE syntax using PRCE, but I would rather avoid that, being messy and hackish. In other words, wildcards are still processed the same.

There isn’t any substantial benefit from all this preventing, other than avoiding some rows from appearing in the table. According to my testing, when DNS fetching is turned off, resource-wise there pretty much no affect. Memory usage can increase a bit due to the plugin being loaded and active on public side, and the filtering keeps the runtime almost the same. When automatic excluding is enabled, checking for login sessions keeps query count the same too.

Last edited by Gocom (2012-09-11 14:52:07)

etc · 2012-09-11 16:08:19

I haven’t thought of all this, you are right, the benefit is not so evident except for DNS fetching. Still, it should be doable without much drawback at least for IP lists. Replacing DB writes by DB reads is clearly not an option, but I wouldn’t call imitating LIKE syntax a hack. Also, I do not know how MySQL internally works, but would a query like SELECT '172.16.1.254' LIKE '172.16._%' create any record?

Gocom · 2012-09-11 16:41:16

etc wrote:

Replacing DB writes by DB reads is clearly not an option

Huh? Just running the plugin on public side adds more reads to each page. I’ve already added the active preventing to the plugin in the latest two commits, and it at minimum adds extra two fetched rows to each page (plugin, login session), plus preferences, processing and some more memory taken by the source itself, eval (which are not cached) and keeping the instance in memory. One more read is needed if LIKE patterns are checked.

Also, I do not know how MySQL internally works, but would a query like SELECT '172.16.1.254' LIKE '172.16._%' create any record?

MySQL can do calculations and retrieve computed results. Makes an expensive calculator and/or PCRE runtime. For instance for rah_unlog_me, the LIKE filtering wouldn’t need more than:

getThing("SELECT '".doSlash(remote_addr())."' LIKE ". implode(" OR '".doSlash(remote_addr())."' LIKE ", quote_list(do_list(get_pref('rah_unlog_me_ip')))))

Replacing the current in_array() in rah_unlog_me::block().

Last edited by Gocom (2012-09-11 16:43:11)

etc · 2012-09-11 18:27:55

Gocom wrote:

Just running the plugin on public side adds more reads to each page. I’ve already added the active preventing to the plugin in the latest two commits, and it at minimum adds extra two fetched rows to each page (plugin, login session), plus preferences, processing and some more memory taken by the source itself, eval (which are not cached) and keeping the instance in memory. One more read is needed if LIKE patterns are checked.

Thank you, I will test. More rows and memory – yes, but more disk reads – why? Preferences are already fetched, plugins are loaded anyway. I don’t think the active preventing should check cookies, it’s not very reliable anyway. IP list would suffice (for me) to keep the robots away.

MySQL can do calculations and retrieve computed results. Makes an expensive calculator and/or PCRE runtime.

Well, we are speaking here about maybe 0.1% of txp processing time, don’t frighten plugin users! :)

Gocom · 2012-09-11 19:44:01

etc wrote:

I don’t think the active preventing should check cookies, it’s not very reliable anyway.

What is unreliable? It doesn’t check for any cookie, it checks if the user is logged in to Textpattern. Excluding hits generated by site admins is the main reason why rah_unlog_me exists, thus the name, and it’s been an option in the plugin since the beginning. If the option is enabled, which it is by default, hits generated by logged in users are removed, or as of now — filtered, from logs.

Last edited by Gocom (2012-09-11 19:45:38)

etc · 2012-09-11 21:46:17

Gocom wrote:

What is unreliable? It doesn’t check for any cookie, it checks if the user is logged in to Textpattern. Excluding hits generated by site admins is the main reason why rah_unlog_me exists, thus the name, and it’s been an option in the plugin since the beginning. If the option is enabled, which it is by default, hits generated by logged in users are removed, or as of now — filtered, from logs.

Haven’t seen it was is_logged_in(), sorry, thought you were only checking if txp_login_public cookie is sent. It’s fine on the admin side, but what if an admin visits the site without being logged? I would make the active prevention into an option of “Additional IPs to exclude from the logs” only, to avoid an extra DB query. Or maybe two separate options, this one and one for “Exclude site authors’ IPs from the logs”? Sorry for being fuzzy, and thank you for you time.

Gocom · 2012-09-11 23:21:02

etc wrote:

but what if an admin visits the site without being logged

Then visitor is not logged in, right? Does non. The other reason why there is the clean up function that sweeps the database on log-ins. Albeit, if there weren’t multi-site issues, as of active filtering I would be inclined to remove the clean up feature since addresses are dynamic and such this has the small potential change of removing records it shouldn’t. What is good about filtering is that logs don’t get cluttered by throwaway rows, resulting in slower fragmentation rate caused by deleting.

to avoid an extra DB query

There is no extra query. Just one per page whether the visitor is an admin or not. Without the LIKE pattern support there would be no queries involved for unauthenticated visitors.

Or maybe two separate options, this one and one for “Exclude site authors’ IPs from the logs”?

Nah, I will keep that in mind, but I’m fine how the plugin is now and am inclined to keep preferences as simple as possible.

Textpattern CMS

Textpattern CMS support forum

#13 2012-03-29 10:24:39

Re: rah_unlog_me // Remove your own hits from visitor logs

#14 2012-03-29 13:14:44

Re: rah_unlog_me // Remove your own hits from visitor logs

#15 2012-07-12 00:12:47

Re: rah_unlog_me // Remove your own hits from visitor logs

#16 2012-09-10 15:32:48

Re: rah_unlog_me // Remove your own hits from visitor logs

#17 2012-09-11 11:01:49

Re: rah_unlog_me // Remove your own hits from visitor logs

#18 2012-09-11 14:46:54

Re: rah_unlog_me // Remove your own hits from visitor logs

#19 2012-09-11 16:08:19

Re: rah_unlog_me // Remove your own hits from visitor logs

#20 2012-09-11 16:41:16

Re: rah_unlog_me // Remove your own hits from visitor logs

#21 2012-09-11 18:27:55

Re: rah_unlog_me // Remove your own hits from visitor logs

#22 2012-09-11 19:44:01

Re: rah_unlog_me // Remove your own hits from visitor logs

#23 2012-09-11 21:46:17

Re: rah_unlog_me // Remove your own hits from visitor logs

#24 2012-09-11 23:21:02

Re: rah_unlog_me // Remove your own hits from visitor logs

Board footer