PHP RFC: NULL Coercion Consistency

• Version: 0.2
• Voting Start: ?
• Voting End: ?
• RFC Started: 2022-04-05
• RFC Updated: 2022-04-05
• Author: Craig Francis [craig#at#craigfrancis.co.uk]
• Status: Draft
• First Published at: https://wiki.php.net/rfc/null_coercion_consistency
• GitHub Repo: https://github.com/craigfrancis/php-null-coercion-consistency-rfc
• Implementation: ?

PHP 8.1 introduced Deprecate passing null to non-nullable arguments of internal functions. While the consistency is welcome (user-defined vs internal functions), for those not using strict Static Analysis or strict_types=1, the breaking of NULL coercion creates a difficult BC problem.

This RFC does not change anything for strict_types=1 (or static analysis), as strict type checks can be useful. For example, some developers view NULL as a missing/invalid value (not as a value in itself), and passing NULL to a function like htmlspecialchars() might indicate a problem.

There was a short discussion about the original RFC, but with the exception of Craig Duncan, there was no discussion about the BC problems, or the inconsistency of NULL coercion compared to string/int/float/bool coercion, or other contexts like string concatenation, == comparisons, arithmetics, sprintf, print, echo, array keys, etc.

The intention is to also keep Unify PHP's typing modes by George Peter Banyard in mind, with coercions like substr($string, “offset”) and htmlspecialchars(array()) as being clearly problematic; whereas the following is common:

$search = ($_GET['q'] ?? NULL); // Or similar (examples below)
 
echo 'Results for ' . htmlspecialchars($search); // Fatal Error?!?

Roughly 15% of scripts use strict_types=1 (calculation below).

Roughly 33% of developers use static analysis (realistically it's less than this, details below).

According to the documentation, when not using strict_types=1, “PHP will coerce values of the wrong type into the expected scalar type declaration if possible” (ref).

Coercion from NULL is well defined:

1. To String: “null is always converted to an empty string.”
2. To Integer: “null is always converted to zero (0).”
3. To Float: “For values of other types, the conversion is performed by converting the value to int first and then to float”
4. To Boolean: “When converting to bool, the following values are considered false [...] the special type NULL”

String/Int/Float/Bool can be coerced; Arrays, Resources, and Objects (without toString) cannot be coerced (for fairly obvious reasons).

Or, NULL “can be coerced to the type requested by the hint without data loss and without creation of likely unintended data” (ref).

PHP 7.0 introduced the ability for user-defined functions to specify parameter types via the Scalar Type Declarations RFC, where the implementation triggered Type Errors for those using strict_types=1, and otherwise used coercion for string/int/float/bool, but not NULL, PHP 8.1 updated internal function parameters to work in the same way.

print('A');
print(1);
print(1.2);
print(false);
print(NULL); // Fine, still coerced to empty string.
 
echo NULL; // Fine, still coerced to empty string.
 
var_dump(3 + '5' + NULL); // Fine, int(8)
var_dump(NULL / 6); // Fine, int(0)
 
$o = [];
 
$o[] = ('' == '');
$o[] = ('' == NULL); // Fine, still coerced to empty string.
 
$o[] = 'ConCat ' . 'A';
$o[] = 'ConCat ' . 123;
$o[] = 'ConCat ' . 1.2;
$o[] = 'ConCat ' . false;
$o[] = 'ConCat ' . NULL; // Fine, still coerced to empty string.
 
$o[] = sprintf('%s', 'A');
$o[] = sprintf('%s', 1);
$o[] = sprintf('%s', 1.2);
$o[] = sprintf('%s', false);
$o[] = sprintf('%s', NULL); // Fine, still coerced to empty string.
 
$o[] = htmlspecialchars('A');
$o[] = htmlspecialchars(1);
$o[] = htmlspecialchars(1.2);
$o[] = htmlspecialchars(false);
$o[] = htmlspecialchars(NULL); // Deprecated in 8.1, Fatal Error in 9.0?

With user-defined functions, this inconsistency is also noted:

function user_function(string $s, int $i, float $f, bool $b) {
  var_dump($s, $i, $f, $b);
}
 
user_function('1', '1', '1', '1');
  // string(1) "1" / int(1) / float(1) / bool(true)
 
user_function(2, 2, 2, 2);
  // string(1) "2" / int(2) / float(2) / bool(true)
 
user_function(3.3, 3.3, 3.3, 3.3);
  // string(3) "3.3" / int(3), lost precision / float(3.3) / bool(true)
 
user_function(false, false, false, false);
  // string(0) "" / int(0) / float(0) / bool(false)
 
user_function(NULL, NULL, NULL, NULL);
  // Fatal error, Uncaught TypeError x4!

The Scalar Type Declarations RFC says “it should be possible for existing userland libraries to add scalar type declarations without breaking compatibility”, but this is not the case, because of NULL. This has made adoption of type declarations harder, as it does not work like the following:

function my_function($s, $i, $f, $b) {
  $s = strval($s);
  $i = intval($i);
  $f = floatval($f);
  $b = boolval($b);
  var_dump($s, $i, $f, $b);
}
 
function my_function(string $s, int $i, float $f, bool $b) {
  var_dump($s, $i, $f, $b);
}
 
my_function(NULL, NULL, NULL, NULL);

George Peter Banyard notes that “Userland scalar types [...] did not include coercion from NULL for very good reasons”.

The Scalar Type Declarations RFC says “The only exception to this is the handling of NULL: in order to be consistent with our existing type declarations for classes, callables and arrays, NULL is not accepted by default, unless it is a parameter and is explicitly given a default value of NULL”, which goes against the documentation (as noted above) where the coercion from NULL is well defined (i.e. NULL is more like a string/int/float/bool, rather than an object/callable/array).

Common sources of NULL:

$search = (isset($_GET['q']) ? $_GET['q'] : NULL);
 
$search = ($_GET['q'] ?? NULL); // Since PHP 7
 
$search = filter_input(INPUT_GET, 'q');
 
$search = $request->input('q'); // Laravel
$search = $request->get('q'); // Symfony
$search = $this->request->getQuery('q'); // CakePHP
$search = $request->getGet('q'); // CodeIgniter
 
$value = array_pop($empty_array);
$value = mysqli_fetch_row($result);

Examples functions, often working with user input, where NULL has always been accepted/coerced:

$rounded_value = round($value);
 
$search_trimmed = trim($search);
 
$search_len = strlen($search);
 
$search_upper = strtoupper($search);
 
$search_hash = hash('sha256', $search);
 
echo htmlspecialchars($search);
 
echo 'https://example.com/?q=' . urlencode($search);
 
preg_match('/^[a-z]/', $search);
 
exec('/path/to/cmd ' . escapeshellarg($search));
 
socket_write($socket, $search);
 
xmlwriter_text($writer, $search);

And developers have used NULL to skip certain parameters, e.g.

setcookie('q', $search, NULL, NULL, NULL, true, true); // x4
 
substr($string, NULL, 3);
 
mail('nobody@example.com', 'subject', 'message', NULL, '-fwebmaster@example.com');

HTML Templating engines like Laravel Blade suppress this deprecation with null-coalescing (patch); or Symphony Twig which preserves NULL, but it's often passed to echo (which accepts it, despite the echo documentation saying it accepts non-nullable strings).

I'd argue strict type checking (that prevents all forms of coercion) should be done via Static Analysis or via strict_types=1 opt-in, like how a string (e.g. '15') being provided to integer parameter could identify a problem in some development environments.

There are approximately 335 parameters affected by this deprecation.

As an aside, there are also roughly 104 questionable and 558 problematic parameters which probably shouldn't accept NULL or an Empty String. For these parameters, a different RFC could consider updating them to reject both NULL and Empty Strings, e.g. $needle in strpos(), and $characters in trim(); in the same way that $separator in explode() already has a “cannot be empty” Fatal Error.

The only realistic way for developers to find when NULL is passed to these internal functions is to use the deprecation notices (not ideal).

And this does not stop new issues being introduced, as developers may not consider when variables might be NULL (e.g. a browser extension or network issues affecting GET/POST variables).

It is possible to use very strict Static Analysis, to follow every variable from source to sink (to check if a variable could be NULL), but most developers are not in a position to do this (i.e. not using static analysis, or not at a high enough level, or they are using a baseline to ignore).

In the last JetBrains developer survey (with 67% regularly using Laravel), only 33% used Static Analysis (source); where it's fair to say many of these developers would still not identify these possible NULL values (too low level, and/or using a baseline).

As an example, take this simple script:

./src/index.php
<?php
$nullable = ($_GET['a'] ?? NULL);
echo htmlentities($nullable);
?>

This is considered fine by these tools:

composer require --dev "squizlabs/php_codesniffer=*"
./vendor/bin/phpcs -p ./src/
E 1 / 1 (100%)
[...]
 2 | ERROR | Missing file doc comment
[...]

composer require friendsofphp/php-cs-fixer
./vendor/bin/php-cs-fixer fix src --diff --allow-risky=yes
Loaded config default.
Using cache file ".php-cs-fixer.cache".
   1) src/index.php
      ---------- begin diff ----------
--- src/index.php
+++ src/index.php
@@ -1,4 +1,4 @@
 <?php
+
 $nullable = ($_GET['a'] ?? null);
 echo htmlentities($nullable);
-?>
\ No newline at end of file
 
      ----------- end diff -----------
 
Fixed all files in 0.012 seconds, 12.000 MB memory used

composer require --dev phpcompatibility/php-compatibility
sed -i '' -E 's/(PHPCSHelper::getConfigData)/(string) \1/g' vendor/phpcompatibility/php-compatibility/PHPCompatibility/Sniff.php
./vendor/bin/phpcs --config-set installed_paths vendor/phpcompatibility/php-compatibility
 
./vendor/bin/phpcs -p ./src/ --standard=PHPCompatibility --runtime-set testVersion 8.1
. 1 / 1 (100%)

Note: Juliette (@jrfnl) has confirmed that getting PHPCompatibility to solve this problem will be “pretty darn hard to do” because it's “not reliably sniffable” (source).

composer require --dev phpstan/phpstan
./vendor/bin/phpstan analyse -l 9 ./src/
[OK] No errors

composer require --dev phpstan/phpstan-strict-rules
composer require --dev phpstan/extension-installer
./vendor/bin/phpstan analyse -l 9 ./src/
[OK] No errors

Note: There are Stricter Analysis options for PHPStan, but they don't seem to help with this problem.

composer require --dev vimeo/psalm
./vendor/bin/psalm --init ./src/ 4
./vendor/bin/psalm
No errors found!

Note: Psalm can detect this at levels 1, 2, and 3 (don't use a baseline).

Since 21st June 2022, Rector can modify 362 function arguments via NullToStrictStringFuncCallArgRector:

mkdir -p rector/src;
 
cd rector/;
 
composer require --dev rector/rector;
 
echo '<?= htmlspecialchars($var) ?>' > src/index.php;
 
echo '<?php
 
use Rector\Php81\Rector\FuncCall\NullToStrictStringFuncCallArgRector;
use Rector\Config\RectorConfig;
 
return static function (RectorConfig $rectorConfig): void {
    $rectorConfig->paths([__DIR__ . "/src"]);
    $rectorConfig->rule(NullToStrictStringFuncCallArgRector::class);
};
' > rector.php;
 
./vendor/bin/rector process;

This will litter the code with the use of (string) type casting, e.g.

-<?= htmlspecialchars($var) ?>
+<?= htmlspecialchars((string) $var) ?>

For a typical project (which won't be using strict_types=1), expect thousands of changes to be made; and note how this does not improve code quality.

You can disable E_DEPRECATED (as recommended by projects like WordPress).

Alternatively you can use set_error_handler(), with something like:

function ignore_null_coercion($errno, $errstr) {
  // https://github.com/php/php-src/blob/012ef7912a8a0bb7d11b2dc8d108cc859c51e8d7/Zend/zend_API.c#L458
  if ($errno === E_DEPRECATED && preg_match('/Passing null to parameter #.* of type .* is deprecated/', $errstr)) {
    return true;
  }
  return false;
}
set_error_handler('ignore_null_coercion', E_DEPRECATED);

And some developers are simply patching php-src (risky).

While making each change is fairly easy - they are still difficult to find, there are many of them, and the updates used are often pointless, e.g.

• urlencode(strval($name));
• urlencode((string) $name);
• urlencode($name ?? “”);

And often do not make the code easier to read:

 - $result = substr($string, 0, $length);
 + $result = substr($string ?? '', 0, $length);

As noted above - Rector can add (string) type casting automatically, but this does not improve code quality.

Revert the NULL deprecation for parameters (when not using strict_types=1), so NULL coercion continues to work (as it does in other contexts) to avoid the upgrade problems (i.e. Fatal Errors in PHP 9.0).

And, in the spirit of the original RFC to keep user-defined and internal functions consistent, also change user-defined functions so NULL is coerced for non-nullable parameters (when not using strict_types=1).

This means the type “?int” will allow NULL or an integer to be provided to the function; whereas the non-nullable type “int” would coerce NULL to 0; in exactly the same way the string “0” or the boolean false would be.

While the intention of this RFC is to avoid a BC break; for user defined functions to be updated to also coerce NULL (instead of throwing a Type Error), while unlikely, it is possible some non strict_types code may rely the TypeError being thrown, for example:

function my_function(string $my_string) {
  var_dump($my_string);
}
 
try {
  my_function('A');   // string(1) "A"
  my_function(1);     // string(1) "1"
  my_function(1.2);   // string(3) "1.2"
  my_function(true);  // string(1) "1"
  my_function(false); // string(0) ""
  my_function(NULL);  // Throw Type Error
} catch (TypeError $e) {
  // Do something important?
}

PHP 8.2

None known

None known

None known

“it's a bit late” - We only have a deprecation at the moment (which can and is being ignored), it will be “too late” when PHP 9.0 uses Fatal Errors.

The function mt_rand() can be called with no arguments, or with min and max integer arguments. A developer may call mt_rand(NULL, NULL) and expect it to work the same as no arguments (returning a random number between 0 and mt_getrandmax()), but the NULL's would be coerced to 0, so it would always return 0. That said, I cannot find any public examples of this happening (1, 2, 3).

Some function parameters could be updated to rase a Fatal Error when NULL or Empty String is provided; e.g.

1. $needle in strpos()
2. $characters in trim()
3. $method in method_exists()
4. $json in json_decode()

It might be appropriate for coercion and explicit casting/converting to work in the same way, even if they were to become stricter in the values they accept; e.g. intval(“”) and ((int) “”) currently return int(0), whereas (5 + “”) results in a TypeError.

Accept the RFC

TODO

TODO

1. Updating some parameters to accept NULL (details). This was rejected because some developers view NULL as a missing/invalid value that should never be passed to functions like htmlspecialchars() (quiz results).

The 15% of scripts that use strict_types=1 was calculated using grep.app, to “search across a half million git repos”, were each result is a script (not a count of matches, example). We can see 272,871 scripts using strict_types=1, out of 1,842,666. And keep in mind that WordPress only really appears once, it is affected by this deprecation, and is installed/used by many.

In the Scalar Type Declarations RFC for PHP 7.0, scalar types were defined as “int, float, string and bool” - but, despite NULL also being a simple value (i.e. not an array/object/resource), it was not included in this definition. For backwards compatibility reasons this definition is unlikely to change.

Also, note the example quote from Rasmus:

PHP is and should remain:
1) a pragmatic web-focused language
2) a loosely typed language
3) a language which caters to the skill-levels and platforms of a wide range of users