Vanilla Textpattern 4.8.4 hiccup

gomedia · 2021-01-07 04:52:36

Bloke wrote #328011:

… if there is any way we can detect this at the application code level, we could bail out more gracefully than a screeching halt.

Thanks Stef. I understand that it’s my ancient setup which is the fundamental cause but to my mind it’s the way the error manifests itself that’s an issue.

The use of cURL results in a catastrophic failure with no explanation (I’m not seeing entries in PHP error_log either). Whereas, at least with file_get_contents() one gets error messages presented in the browser, and more importantly, the Diagnostics Panel is rendered.

gomedia · 2021-01-07 05:07:01

Just for the record, my current setup is:

PHP 5.6.21, installed sometime ago from here
do have PHP 7.3.11 as well, which came preinstalled with MacOS 10.15 Catalina

My local websites run on 5.6.21 but I can access both PHP CLIs.

Here’s a code snippet (updated to include OPENSSL_VERSION_NUMBER):

<?php
echo "*** PHP ***\n";
echo phpversion()."\n";
echo "*** CURL ***\n";
print_r(curl_version());
echo "*** OPENSSL ***\n";
echo "OPENSSL_VERSION_TEXT = ".OPENSSL_VERSION_TEXT."\n";
echo "OPENSSL_VERSION_NUMBER = ".OPENSSL_VERSION_NUMBER."\n";
echo "*** REGISTERED SOCKET TRANSPORTS ***\n";
print_r(stream_get_transports());
echo "*** TXP LATEST ***\n";
echo file_get_contents('https://textpattern.com/version.json')."\n";
?>

Running it on 5.6.21 I get:

*** PHP ***
5.6.21
*** CURL ***
Array
(
    [version_number] => 470785
    [age] => 3
    [features] => 558621
    [ssl_version_number] => 0
    [version] => 7.47.1
    [host] => x86_64-apple-darwin14.5.0
    [ssl_version] => SecureTransport
    [libz_version] => 1.2.11
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtsp
            [14] => scp
            [15] => sftp
            [16] => smb
            [17] => smbs
            [18] => smtp
            [19] => smtps
            [20] => telnet
            [21] => tftp
        )

)
*** OPENSSL ***
OPENSSL_VERSION_TEXT = OpenSSL 0.9.8zc 19 Mar 2015
OPENSSL_VERSION_NUMBER = 9470431
*** REGISTERED SOCKET TRANSPORTS ***
Array
(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => sslv3
    [6] => sslv2
    [7] => tls
    [8] => tlsv1.0
)
*** TXP LATEST ***
PHP Warning:  file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:
error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version in curl_openssl_test.php on line 11

PHP Warning:  file_get_contents(): Failed to enable crypto in curl_openssl_test.php on line 11

PHP Warning:  file_get_contents(https://textpattern.com/version.json): failed to open stream: operation failed in curl_openssl_test.php on line 11

You can see the various versions and the file_get_contents warnings (I’ve edited them in the above output for brevity).

Running it on 7.3.11 I get:

*** PHP ***
7.3.11
*** CURL ***
Array
(
    [version_number] => 475137
    [age] => 4
    [features] => 7308189
    [ssl_version_number] => 0
    [version] => 7.64.1
    [host] => x86_64-apple-darwin19.0
    [ssl_version] => (SecureTransport) LibreSSL/2.8.3
    [libz_version] => 1.2.11
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtsp
            [14] => smb
            [15] => smbs
            [16] => smtp
            [17] => smtps
            [18] => telnet
            [19] => tftp
        )

    [ares] => 
    [ares_num] => 0
    [libidn] => 
    [iconv_ver_num] => 0
    [libssh_version] => 
    [brotli_ver_num] => 0
    [brotli_version] => 
)
*** OPENSSL ***
OPENSSL_VERSION_TEXT = LibreSSL 2.8.3
OPENSSL_VERSION_NUMBER = 536870912
*** REGISTERED SOCKET TRANSPORTS ***
Array
(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => tls
    [6] => tlsv1.0
    [7] => tlsv1.1
    [8] => tlsv1.2
)
*** TXP LATEST ***
{
  "textpattern-version": {
    "release": "4.8.4",
    "prerelease": null
  }
}

So I reckon if I switch to PHP 7.3.11 all should be well.

Last edited by gomedia (2021-01-09 21:58:23)

Bloke · 2021-01-07 09:58:42

Out of curiosity, does wrapping the curl stuff in try... catch() detect this SSL violation?

    if (function_exists('curl_version')) {
        try {
            $ch = curl_init($endpoint);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            $contents = curl_exec($ch);
        } catch (Exception $e) {
            $contents = '';
            print_r($e);
        }
    } else {
        $contents = file_get_contents($endpoint);
    }

If so, we could chuck something like that into 4.8.5 now so at least it’ll show an error message.

Alternatively, what does adding this just before the curl_exec() do:

curl_setopt($ch, CURLOPT_FAILONERROR, true);

Last edited by Bloke (2021-01-07 10:01:09)

gomedia · 2021-01-08 02:17:46

Bloke wrote #328067:

Out of curiosity …

Sorry, to dash your hopes but neither had any effect. Same failure in browser, no Diags panel, and nothing extra in error_log.

Would doing a curl_getinfo($ch); before the curl_exec() help – that way you could check some stuff in advance? I get this:

array (
  'url' => 'https://textpattern.com/version.json',
  'content_type' => NULL,
  'http_code' => 0,
  'header_size' => 0,
  'request_size' => 0,
  'filetime' => 0,
  'ssl_verify_result' => 0,
  'redirect_count' => 0,
  'total_time' => 0,
  'namelookup_time' => 0,
  'connect_time' => 0,
  'pretransfer_time' => 0,
  'size_upload' => 0,
  'size_download' => 0,
  'speed_download' => 0,
  'speed_upload' => 0,
  'download_content_length' => -1,
  'upload_content_length' => -1,
  'starttransfer_time' => 0,
  'redirect_time' => 0,
  'redirect_url' => '',
  'primary_ip' => '',
  'certinfo' => 
  array (
  ),
  'primary_port' => 0,
  'local_ip' => '',
  'local_port' => 0,
)

Hopefully, the values of ssl_verify_result or certinfo provide something useful.

Bloke · 2021-01-08 12:05:14

gomedia wrote #328083:

Hopefully, the values of ssl_verify_result or certinfo provide something useful.

Thanks for testing, and good call on that function. I’ll check those values on a working system and see if we can leverage the results so we can come to less of a screeching halt.

gaekwad · 2021-01-08 13:11:24

I think I’ve got it. The penny just dropped. Adi, I think you’ve explained something that’s been bugging me on and off for a few years. Thank you!

Looking at the PHP 5.6.21 + OpenSSL 0.9.8 stack:

(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => sslv3
    [6] => sslv2
    [7] => tls
    [8] => tlsv1.0
)

…that tops out at TLS v1.0, which textpattern.com isn’t supporting (and won’t, at least on my watch), so the connection to https://textpattern.com/version.json will fail. TLS v1.0 is long-since known as being insecure and has been deprecated, along with SSL v2 and v3 (also on the list above).

The PHP 7.3 instance will work because it uses TLS v1.2, which we do use. We currently run textpattern.com on TLS v1.2 and v1.3, and have a legacy set of ciphers to cover the vast majority of browsers & OpenSSL / curl instances without opening ourselves up to compromise with known-broken stuff.

More later, just looking into some stuff.

gaekwad · 2021-01-08 13:25:39

Worst case scenario, someone’s running the oldest Textpattern-supported PHP for whatever reason, currently PHP 5.5 on the Textpattern 4.8 branch. The PHP OpenSSL page says (emphasis mine):

In order to use the OpenSSL functions you need to install the OpenSSL library. PHP 5 requires at least OpenSSL >= 0.9.6. However later PHP 5 versions have some compilation issues and should be used at least with OpenSSL >= 0.9.8 which is also a minimal version for PHP 7.0. Other versions (PHP >= 7.1.0) require OpenSSL >= 1.0.1.

So, PHP 5 needs OpenSSL 0.9.8 (released 2005, support ended 2015) or later, PHP 7 needs OpenSSL 1.0.1 (released 2012, support ended 2016) or later. TLS v1.2 (the oldest we use that’s currently considered ‘safe’) was introduced in OpenSSL 1.0.1, so PHP running with OpenSSL 1.0.1 or an off-brand remix with TLS v1.2 support should work just fine.

I have no desire to run broken crypto, that will cause us problems – do we need to consider a fallback to GitHub if someone is using bad TLS? Or perhaps update system requirements to include “PHP with TLS v1.2 support (i.e. OpenSSL 1.0.1 or newer)”, sort of thing?

Bloke · 2021-01-08 13:35:59

I’d rather detect the broken crypto stack and bail out slightly more gracefully than we do now, if possible.

Adding the security requirements to the sys reqs page works for me.

If we can detect it, there’s also the possibility of introducing the check very early on – in something like janitor() where we check the PHP version and then (somehow) check that TLS is up-to-spec. We could die with a message that it won’t run right there before going any further.

Will that cause issues with localhost dev though? People who don’t develop in an https environment? If so, we can keep the crypto stack checks at the point of access – namely in our (forthcoming) “connect-to-some-external-stream” class. So each time we try and access either the .json version file, or some third-party stuff we rely on, such as theme or plugin update checks, we’ll verify at that point, and abort the call from within the class if we can’t do it for whatever reason.

The latter sounds safest. But I’m not sure this’ll be ready for 4.8.5 unless we delay a bit longer. If I can hack in the check on the Diagnostics panel as a testbed, we can at least test the waters and see if this is a viable solution for when we do it properly in a class (in 4.9.0).

Last edited by Bloke (2021-01-08 13:38:08)

gaekwad · 2021-01-08 14:02:25

Bloke wrote #328089:

I’d rather detect the broken crypto stack and bail out slightly more gracefully than we do now, if possible.

+1. I was curious what raw.githubusercontent.com runs in its TLS stack, they currently support TLS v1.0 and v1.1:

It’s a trivial thing to maintain an identical version.json on GitHub, but there are privacy / access considerations for people who are anti-GitHub, of course.

Adding the security requirements to the sys reqs page works for me.

I’ll draft something for review.

If we can detect it, there’s also the possibility of introducing the check very early on – in something like janitor() where we check the PHP version and then (somehow) check that TLS is up-to-spec. We could die with a message that it won’t run right there before going any further.

Will that cause issues with localhost dev though? People who don’t develop in an https environment?

As I understand it, OPENSSL_VERSION_NUMBER is a PHP constant with the version string, would that work? A non-dramatic failure message would be sufficient.

But I’m not sure this’ll be ready for 4.8.5 unless we delay a bit longer. If I can hack in the check on the Diagnostics panel as a testbed, we can at least test the waters and see if this is a viable solution for when we do it properly in a class (in 4.9.0).

+1.

Bloke · 2021-01-08 14:39:39

We could flip-flop to GitHub as fallback if we can get the file_get_contents() to die gracefully enough that we can catch it and failover.

Thing is, there are a few things at play here:

Adi’s server wouldn’t connect via curl_exec() so we need some way to check this code pathway is going to work, without it exploding in a shower of 1s and 0s.
Adi’s server then chokes if file_get_contents() is used too, so even if we manage to detect that cURL will fail, falling back on file_get_contents() will also die in this case. So we need to verify this pathway too (in case curl isn’t installed or it doesn’t meet the crypto requirements).
Relying on the OPENSSL_VERSION_NUMBER constant looks like it won’t work, since Adi’s server reports “OpenSSL 0.9.8z” which meets the minimum spec (if I’ve read your post correctly, Pete).
curl_version() doesn’t return a reliable number. Parsing out the version number from the text isn’t my idea of futureproof fun. Viz…

My Mac, with curl compiled into PHP:

array (
  'ssl_version_number' => 0,
  'ssl_version' => 'OpenSSL/1.0.2o',
...
)

OPENSSL_VERSION_NUMBER: 268443903
OPENSSL_VERSION_TEXT: OpenSSL 1.0.2o 27 Mar 2018

My server without curl compiled in:

OPENSSL_VERSION_NUMBER: 269488255
OPENSSL_VERSION_TEXT: OpenSSL 1.1.1g 21 Apr 2020

So, not sure what to make of all that.

Ideas?

Last edited by Bloke (2021-01-08 14:55:06)

Bloke · 2021-01-08 14:48:04

P.S. curl_getinfo($ch); on my localhost (Mac) returns:

array (
  'url' => 'https://textpattern.com/version.json',
  'content_type' => NULL,
  'http_code' => 0,
  'header_size' => 0,
  'request_size' => 0,
  'filetime' => -1,
  'ssl_verify_result' => 0,
  'redirect_count' => 0,
  'total_time' => 0.0,
  'namelookup_time' => 0.0,
  'connect_time' => 0.0,
  'pretransfer_time' => 0.0,
  'size_upload' => 0.0,
  'size_download' => 0.0,
  'speed_download' => 0.0,
  'speed_upload' => 0.0,
  'download_content_length' => -1.0,
  'upload_content_length' => -1.0,
  'starttransfer_time' => 0.0,
  'redirect_time' => 0.0,
  'redirect_url' => '',
  'primary_ip' => '',
  'certinfo' => 
  array (
  ),
  'primary_port' => 0,
  'local_ip' => '',
  'local_port' => 0,
)

Can’t see anything of use there. Any other threads we can clutch at?

Last edited by Bloke (2021-01-08 14:55:33)

Bloke · 2021-01-08 14:52:03

gaekwad wrote #328086:

I think I’ve got it. Looking at the PHP 5.6.21 + OpenSSL 0.9.8 stack: [8] => tlsv1.0

Missed this post, sorry. I wonder if we can check that stream_get_transports() has a tls value in the array that we support and bail if not. That might be simplest all round, if it’s reliable.

EDIT: This is what I see on my server:

array (
  0 => 'tcp',
  1 => 'udp',
  2 => 'unix',
  3 => 'udg',
  4 => 'ssl',
  5 => 'tls',
  6 => 'tlsv1.0',
  7 => 'tlsv1.1',
  8 => 'tlsv1.2',
)

and localhost (Mac):

array (
  0 => 'tcp',
  1 => 'udp',
  2 => 'unix',
  3 => 'udg',
  4 => 'ssl',
  5 => 'sslv3',
  6 => 'tls',
  7 => 'tlsv1.0',
  8 => 'tlsv1.1',
  9 => 'tlsv1.2',
)

So it does seem as if detecting whether tlsv1.2 is in the array might be a way forward. If not, just give up and say ‘version check not available’ and move on.

There’s the issue of keeping this up to date as things march onward, but if we put a minimum version in the constants then it’s a prompt that we need to attend to it, like we do with the PHP version.

The only issue I can foresee is that, for now, they list all available versions in this array. In future, they might decide to drop older entries so we can’t do a simple “is our version in the list” but also need to check “is our supported version less than the highest one in the list” which complicates things a tad.

Not sure what happens in PHP 8.0.0 either.

Last edited by Bloke (2021-01-08 15:04:15)

Textpattern CMS

Textpattern CMS support forum

#13 2021-01-07 04:52:36

Re: Vanilla Textpattern 4.8.4 hiccup

Bloke wrote #328011:

#14 2021-01-07 05:07:01

Re: Vanilla Textpattern 4.8.4 hiccup

#15 2021-01-07 09:58:42

Re: Vanilla Textpattern 4.8.4 hiccup

#16 2021-01-08 02:17:46

Re: Vanilla Textpattern 4.8.4 hiccup

Bloke wrote #328067:

#17 2021-01-08 12:05:14

Re: Vanilla Textpattern 4.8.4 hiccup

gomedia wrote #328083:

#18 2021-01-08 13:11:24

Re: Vanilla Textpattern 4.8.4 hiccup

#19 2021-01-08 13:25:39

Re: Vanilla Textpattern 4.8.4 hiccup

#20 2021-01-08 13:35:59

Re: Vanilla Textpattern 4.8.4 hiccup

#21 2021-01-08 14:02:25

Re: Vanilla Textpattern 4.8.4 hiccup

Bloke wrote #328089:

#22 2021-01-08 14:39:39

Re: Vanilla Textpattern 4.8.4 hiccup

#23 2021-01-08 14:48:04

Re: Vanilla Textpattern 4.8.4 hiccup

#24 2021-01-08 14:52:03

Re: Vanilla Textpattern 4.8.4 hiccup

gaekwad wrote #328086:

Board footer