Go to main content

Textpattern CMS support forum

You are not logged in. Register | Login | Help

#11 2021-01-04 10:55:39

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 9,994
Website

Re: Vanilla Textpattern 4.8.4 hiccup

This is indeed intriguing. We switched to cURL recently because anyone whose host had restricted allow_url_fopen would get the ‘failed to check for updates’ message forever, which was a) misleading, and b) annoying.

Now, however, it seems to have caused another potential issue if the crypto stack isn’t fully realised. This may be that cURL assumes (or requires?) a secure connection if the request is secure, or cannot mix insecure (e.g. local http) page requests with subsequent requests to external secure sources.

Not sure exactly what we can do about this. The plan was to introduce or tweak a class that handled ‘connections’ which would do the same thing as we do on the Diagnostics panel now. I was using this panel as a testbed for the idea, with the goal of ultimately always trying cURL first, then falling back on file_get_contents() and then (maybe) trying some other mechanism such as sockets or streams, before failing as gracefully as we can.

EDIT: The solution that jakob found on StackOverflow for the secure streams might be worth investigating but I’m not sure if that can be applied to cURL too. Haven’t grokked the code yet.

If obtaining the new cert bundle works then I guess we can chalk this up to the configured environment and move on, but it would be nice to have a “broken” crypto environment to play in, so if there is any way we can detect this at the application code level, we could bail out more gracefully than a screeching halt.

As far as the closing ?> goes, it’s not been required for, what, maybe a decade or more in PHP (?) and used to cause issues if there was whitespace after it. The primary driver, though, was to simplify instructions for adding stuff to config.php – a task that may become more commonplace in the next version of Txp, with CSP headers and SMTP routing parameters being permitted in the config file.

Last edited by Bloke (2021-01-04 10:58:21)


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

#12 2021-01-07 04:39:26

gomedia
Plugin Author
Registered: 2008-06-01
Posts: 1,303
Website

Re: Vanilla Textpattern 4.8.4 hiccup

colak wrote #327997:

I noticed that too

Thanks Yiannis, and all the best to you too.

Offline

#13 2021-01-07 04:52:36

gomedia
Plugin Author
Registered: 2008-06-01
Posts: 1,303
Website

Re: Vanilla Textpattern 4.8.4 hiccup

Bloke wrote #328011:

… if there is any way we can detect this at the application code level, we could bail out more gracefully than a screeching halt.

Thanks Stef. I understand that it’s my ancient setup which is the fundamental cause but to my mind it’s the way the error manifests itself that’s an issue.

The use of cURL results in a catastrophic failure with no explanation (I’m not seeing entries in PHP error_log either). Whereas, at least with file_get_contents() one gets error messages presented in the browser, and more importantly, the Diagnostics Panel is rendered.

Offline

#14 2021-01-07 05:07:01

gomedia
Plugin Author
Registered: 2008-06-01
Posts: 1,303
Website

Re: Vanilla Textpattern 4.8.4 hiccup

Just for the record, my current setup is:
  • PHP 5.6.21, installed sometime ago from here
  • do have PHP 7.3.11 as well, which came preinstalled with MacOS 10.15 Catalina

My local websites run on 5.6.21 but I can access both PHP CLIs.

Here’s a code snippet (updated to include OPENSSL_VERSION_NUMBER):

<?php
echo "*** PHP ***\n";
echo phpversion()."\n";
echo "*** CURL ***\n";
print_r(curl_version());
echo "*** OPENSSL ***\n";
echo "OPENSSL_VERSION_TEXT = ".OPENSSL_VERSION_TEXT."\n";
echo "OPENSSL_VERSION_NUMBER = ".OPENSSL_VERSION_NUMBER."\n";
echo "*** REGISTERED SOCKET TRANSPORTS ***\n";
print_r(stream_get_transports());
echo "*** TXP LATEST ***\n";
echo file_get_contents('https://textpattern.com/version.json')."\n";
?>

Running it on 5.6.21 I get:

*** PHP ***
5.6.21
*** CURL ***
Array
(
    [version_number] => 470785
    [age] => 3
    [features] => 558621
    [ssl_version_number] => 0
    [version] => 7.47.1
    [host] => x86_64-apple-darwin14.5.0
    [ssl_version] => SecureTransport
    [libz_version] => 1.2.11
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtsp
            [14] => scp
            [15] => sftp
            [16] => smb
            [17] => smbs
            [18] => smtp
            [19] => smtps
            [20] => telnet
            [21] => tftp
        )

)
*** OPENSSL ***
OPENSSL_VERSION_TEXT = OpenSSL 0.9.8zc 19 Mar 2015
OPENSSL_VERSION_NUMBER = 9470431
*** REGISTERED SOCKET TRANSPORTS ***
Array
(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => sslv3
    [6] => sslv2
    [7] => tls
    [8] => tlsv1.0
)
*** TXP LATEST ***
PHP Warning:  file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:
error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version in curl_openssl_test.php on line 11

PHP Warning:  file_get_contents(): Failed to enable crypto in curl_openssl_test.php on line 11

PHP Warning:  file_get_contents(https://textpattern.com/version.json): failed to open stream: operation failed in curl_openssl_test.php on line 11

You can see the various versions and the file_get_contents warnings (I’ve edited them in the above output for brevity).

Running it on 7.3.11 I get:

*** PHP ***
7.3.11
*** CURL ***
Array
(
    [version_number] => 475137
    [age] => 4
    [features] => 7308189
    [ssl_version_number] => 0
    [version] => 7.64.1
    [host] => x86_64-apple-darwin19.0
    [ssl_version] => (SecureTransport) LibreSSL/2.8.3
    [libz_version] => 1.2.11
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtsp
            [14] => smb
            [15] => smbs
            [16] => smtp
            [17] => smtps
            [18] => telnet
            [19] => tftp
        )

    [ares] => 
    [ares_num] => 0
    [libidn] => 
    [iconv_ver_num] => 0
    [libssh_version] => 
    [brotli_ver_num] => 0
    [brotli_version] => 
)
*** OPENSSL ***
OPENSSL_VERSION_TEXT = LibreSSL 2.8.3
OPENSSL_VERSION_NUMBER = 536870912
*** REGISTERED SOCKET TRANSPORTS ***
Array
(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => tls
    [6] => tlsv1.0
    [7] => tlsv1.1
    [8] => tlsv1.2
)
*** TXP LATEST ***
{
  "textpattern-version": {
    "release": "4.8.4",
    "prerelease": null
  }
}

So I reckon if I switch to PHP 7.3.11 all should be well.

Last edited by gomedia (2021-01-09 21:58:23)

Offline

#15 2021-01-07 09:58:42

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 9,994
Website

Re: Vanilla Textpattern 4.8.4 hiccup

Out of curiosity, does wrapping the curl stuff in try... catch() detect this SSL violation?

    if (function_exists('curl_version')) {
        try {
            $ch = curl_init($endpoint);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            $contents = curl_exec($ch);
        } catch (Exception $e) {
            $contents = '';
            print_r($e);
        }
    } else {
        $contents = file_get_contents($endpoint);
    }

If so, we could chuck something like that into 4.8.5 now so at least it’ll show an error message.

Alternatively, what does adding this just before the curl_exec() do:

curl_setopt($ch, CURLOPT_FAILONERROR, true);

Last edited by Bloke (2021-01-07 10:01:09)


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

#16 2021-01-08 02:17:46

gomedia
Plugin Author
Registered: 2008-06-01
Posts: 1,303
Website

Re: Vanilla Textpattern 4.8.4 hiccup

Bloke wrote #328067:

Out of curiosity …

Sorry, to dash your hopes but neither had any effect. Same failure in browser, no Diags panel, and nothing extra in error_log.

Would doing a curl_getinfo($ch); before the curl_exec() help – that way you could check some stuff in advance? I get this:

array (
  'url' => 'https://textpattern.com/version.json',
  'content_type' => NULL,
  'http_code' => 0,
  'header_size' => 0,
  'request_size' => 0,
  'filetime' => 0,
  'ssl_verify_result' => 0,
  'redirect_count' => 0,
  'total_time' => 0,
  'namelookup_time' => 0,
  'connect_time' => 0,
  'pretransfer_time' => 0,
  'size_upload' => 0,
  'size_download' => 0,
  'speed_download' => 0,
  'speed_upload' => 0,
  'download_content_length' => -1,
  'upload_content_length' => -1,
  'starttransfer_time' => 0,
  'redirect_time' => 0,
  'redirect_url' => '',
  'primary_ip' => '',
  'certinfo' => 
  array (
  ),
  'primary_port' => 0,
  'local_ip' => '',
  'local_port' => 0,
)

Hopefully, the values of ssl_verify_result or certinfo provide something useful.

Offline

#17 2021-01-08 12:05:14

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 9,994
Website

Re: Vanilla Textpattern 4.8.4 hiccup

gomedia wrote #328083:

Hopefully, the values of ssl_verify_result or certinfo provide something useful.

Thanks for testing, and good call on that function. I’ll check those values on a working system and see if we can leverage the results so we can come to less of a screeching halt.


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

#18 2021-01-08 13:11:24

gaekwad
Admin
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 3,349

Re: Vanilla Textpattern 4.8.4 hiccup

I think I’ve got it. The penny just dropped. Adi, I think you’ve explained something that’s been bugging me on and off for a few years. Thank you!

Looking at the PHP 5.6.21 + OpenSSL 0.9.8 stack:

(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => sslv3
    [6] => sslv2
    [7] => tls
    [8] => tlsv1.0
)

…that tops out at TLS v1.0, which textpattern.com isn’t supporting (and won’t, at least on my watch), so the connection to https://textpattern.com/version.json will fail. TLS v1.0 is long-since known as being insecure and has been deprecated, along with SSL v2 and v3 (also on the list above).

The PHP 7.3 instance will work because it uses TLS v1.2, which we do use. We currently run textpattern.com on TLS v1.2 and v1.3, and have a legacy set of ciphers to cover the vast majority of browsers & OpenSSL / curl instances without opening ourselves up to compromise with known-broken stuff.

More later, just looking into some stuff.

Offline

#19 2021-01-08 13:25:39

gaekwad
Admin
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 3,349

Re: Vanilla Textpattern 4.8.4 hiccup

Worst case scenario, someone’s running the oldest Textpattern-supported PHP for whatever reason, currently PHP 5.5 on the Textpattern 4.8 branch. The PHP OpenSSL page says (emphasis mine):

In order to use the OpenSSL functions you need to install the OpenSSL library. PHP 5 requires at least OpenSSL >= 0.9.6. However later PHP 5 versions have some compilation issues and should be used at least with OpenSSL >= 0.9.8 which is also a minimal version for PHP 7.0. Other versions (PHP >= 7.1.0) require OpenSSL >= 1.0.1.

So, PHP 5 needs OpenSSL 0.9.8 (released 2005, support ended 2015) or later, PHP 7 needs OpenSSL 1.0.1 (released 2012, support ended 2016) or later. TLS v1.2 (the oldest we use that’s currently considered ‘safe’) was introduced in OpenSSL 1.0.1, so PHP running with OpenSSL 1.0.1 or an off-brand remix with TLS v1.2 support should work just fine.

I have no desire to run broken crypto, that will cause us problems – do we need to consider a fallback to GitHub if someone is using bad TLS? Or perhaps update system requirements to include “PHP with TLS v1.2 support (i.e. OpenSSL 1.0.1 or newer)”, sort of thing?

Offline

#20 2021-01-08 13:35:59

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 9,994
Website

Re: Vanilla Textpattern 4.8.4 hiccup

I’d rather detect the broken crypto stack and bail out slightly more gracefully than we do now, if possible.

Adding the security requirements to the sys reqs page works for me.

If we can detect it, there’s also the possibility of introducing the check very early on – in something like janitor() where we check the PHP version and then (somehow) check that TLS is up-to-spec. We could die with a message that it won’t run right there before going any further.

Will that cause issues with localhost dev though? People who don’t develop in an https environment? If so, we can keep the crypto stack checks at the point of access – namely in our (forthcoming) “connect-to-some-external-stream” class. So each time we try and access either the .json version file, or some third-party stuff we rely on, such as theme or plugin update checks, we’ll verify at that point, and abort the call from within the class if we can’t do it for whatever reason.

The latter sounds safest. But I’m not sure this’ll be ready for 4.8.5 unless we delay a bit longer. If I can hack in the check on the Diagnostics panel as a testbed, we can at least test the waters and see if this is a viable solution for when we do it properly in a class (in 4.9.0).

Last edited by Bloke (2021-01-08 13:38:08)


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

Board footer

Powered by FluxBB