rfc:rng_extension

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
rfc:rng_extension [2021/05/31 13:17]
zeriyoshi back draft
rfc:rng_extension [2022/02/14 10:42]
zeriyoshi update 4.0
Line 1: Line 1:
-====== PHP RFC: Add Random class ====== +====== PHP RFC: Random Extension 4.0 ====== 
-  * Version: 1.0 +  * Version: 4.0 
-  * Date: 2021-05-18 +  * Date: 2022-02-14 
-  * Author: Go Kudo <zeriyoshi@gmail.com> +  * Author: Go Kudo <zeriyoshi@gmail.com> <g-kudo@colopl.co.jp
-  * Status: Draft +  * Status: Under Discussion 
-  * Implementation: https://github.com/php/php-src/pull/7079+  * Implementation: https://github.com/php/php-src/pull/XXXX
   * First Published at: http://wiki.php.net/rfc/object_scope_prng   * First Published at: http://wiki.php.net/rfc/object_scope_prng
  
 ===== Introduction ===== ===== Introduction =====
-PHP is currently having problems with RNG reproducibility. 
  
-PHP's RNG has been unified into an implementation using the Mersenne twisterwith the rand() and srand() functions becoming aliases for mt_rand() and mt_srand() respectively in PHP 7.1.+PHP implements several useful RNGs. Howeverthey are currently only available in the global scope.
  
-Butthese functions still store the state in the global state of PHP and are not easily reproducible. Look at the following example.+Mersenne Twister, PHP's default RNG, provides a function mt_srand() to initialize with a user-specified seed value, but the scope is global, which may cause unintended user behavior.
  
-<code php> +When a user executes mt_srand()one would expect it to only affect result of mt_rand(), however, the following functions implicitly affect the result of mt_rand()
-echo foo(1234function (): void {}) . PHP_EOL; // Result: 1480009472 +
-echo foo(1234function (): void { mt_rand(); }) . PHP_EOL; // Result: 1747253290+
  
-function foo(int $seed, callable $bar): int { +  * shuffle() 
-    mt_srand($seed); +  * str_shuffle() 
-    $result = mt_rand()+  * array_rand()
-    $bar(); +
-    $result += mt_rand(); +
-    return $result; +
-+
-</code>+
  
-As mentioned above, the reproducibility of random numbers can easily be lost if additional processing is added later. +For examplein the following code, the result of the second mt_rand() is not reproducibleThis is because shuffle() uses a MT RNG internallywhich changes the state.
- +
-In addition, the fiber extension was introduced in PHP 8.1. This makes it more difficult to keep track of the execution orderHowever, this problem has existed since the introduced of Generator. +
- +
-There is also the problem of functions that implicitly use the state stored in PHP's global state. shuffle(), str_shuffle(), and array_rand() functions implicitly advance the state of a random number. This means that the following code is not reproducible, but it is difficult for the user to notice this.+
  
 <code php> <code php>
 mt_srand(1234); mt_srand(1234);
-echo mt_rand() . PHP_EOL// Result: 411284887+$next = mt_rand();
  
 mt_srand(1234); mt_srand(1234);
-str_shuffle('foobar'); +$arr = range(0, 9); 
-echo mt_rand() . PHP_EOL; // Result1314500282+shuffle($arr); 
 +$next2 = mt_rand()
 + 
 +die("next: ${next}, next2: ${next2}"); // next: 411284887, next21848274264
 </code> </code>
  
-===== Proposal ===== +These behaviors were unintuitive and often led to unintended execution results, but were not that problematic for general web application use.
-Implements Random class.+
  
-This class implement to ext/standardalways bundled PHP core.+Howeverin more complex and repeatable applications (such as games), this can be a problem.
  
-The PHP code that represents the implementation is as follows:+There is also the issue of state management difficulties with Fiber, which was added in PHP 8.1. Nikita had this to say:
  
-<code php> +https://externals.io/message/115918#115959
-const RANDOM_XORSHIFT128PLUS = 'xorshift128plus'; // 64-bit, fast, default +
-const RANDOM_MT19937 = 'mt19937'; // 32-bit, for backward compatibility +
-const RANDOM_SECURE = 'secure'; // Cryptographically-Secure PRNG +
-const RANDOM_USER = 'user'; // userland implementation+
  
-class Random +In addition, the Mersenne Twister, can only generate 32-bit values
-+In recent years, many of the environments where PHP runs have been migrating to 64-bit platforms
-    /** +In order to generate more secure valuesan RNG that can generate 64-bit wide values should be provided by the language.
-     * Get registered algorithm in array+
-     */ +
-    public static function getAlgos(): array; +
-     +
-    /** +
-     * Get algorithm information in array+
-     * if not registeredreturns null. +
-     */ +
-    public static function getAlgoInfo(string $algo): ?array;+
  
-    /** +===== Proposal =====
-     * Constructor. +
-     * if $seed is null, generating seed by php_random_bytes(), but whether to use depends to algo. +
-     */ +
-    public function __construct(string $algo RANDOM_XORSHIFT128PLUS, ?int $seed null) {}+
  
-    /** +Implement the XorShift128Plus algorithm for generating new 64-bit wide random numbers, along with a random extension that includes an object scope RNG, and bundle it with PHP. 
-     * Returns raw generated number by RNG. +XorShift128Plus is a fast, high-quality RNG that is proven in major web browsers.  
-     * if on the 32-bit machine, then uses 64-bit RNG throw Exception. +Many of the major hardware architectures are now 64-bit, so it makes sense to use this RNG.
-     * but can implicating truncate on random.ignore_generated_size_exceeded=1 INI directive. +
-     */ +
-    public function nextInt(): int {}+
  
-    /** +In addition to the new algorithm, the following classes will be added to fix the global scope issue.
-     * Generates a number within a given range. +
-     * similar random_int() function. +
-     */ +
-    public function getInt(int $min, int $max): int {}+
  
-    /** +  class Random\NumberGenerator\XorShift128Plus 
-     Generates a string within a given range. +  class Random\NumberGenerator\MersenneTwister 
-     similar random_bytes() function. +  class Random\NumberGenerator\CombinedLCG 
-     */ +  class Random\NumberGenerator\Secure (aka php_random_bytes())
-    public function getBytes(int $length): string {}+
  
-    /** +These classes will hold independent RNG state and will not affect the global scope.
-     * Shuffling the given array items. +
-     * similar shuffle(), but non-pass-by-reference. +
-     */ +
-    public function shuffleArray(array $array): array {}+
  
-    /** +An interface Random\NumberGenerator is also added and are implmeneted by the classes above.  
-     * Shuffling the given string characters+This interface has only a single generate() method which makes it possible to switch between RNG implementations depending on the situation, 
-     * similar str_shuffle(). +allowing alternative implementations to be done by PHP in userland. This is useful, for example, for running tests.
-     */ +
-    public function shuffleString(string $string): string {}+
  
-    /** +RNGs other than XorShift128Plus are based on the RNGs currently implemented in PHP.
-     * Serializing state to string if algo's supported. +
-     */ +
-    public function __serialize(): array {}+
  
-    /** +The Random\Randomizer class will be added to manipulate data using these RNGs. 
-     * Unseriialize state from string in algo's supported. +
-     */ +
-    public function __unserialize(array $data): void {}+
  
-    /** +This class provides the following methods:
-     * Must override user-defined RNG class+
-     * only uses inherits from the Random class with RANDOM_USER argument. +
-     */ +
-    protected function next()int {} +
-}+
  
-</code>+  * __constructor(\Random\NumberGenerator $generator = null) [defaults to XorShift128Plus if null] 
 +  * getInt(int $min, int $max): int [replacement for mt_rand() rand()] 
 +  * getBytes(int $length): string [replacement for random_bytes()] 
 +  * shuffleArray(array $array): array [replacement for shuffle()] 
 +  * shuffleString(string $string): string [replacement for str_shuffle()]
  
-This class retrieves and uses the RNG algorithm registered in the core, based on the string passed in the constructor argument $algo.+Method equivalent to array_rand() was not implemented at this time because the implementation is complex and can be easily implemented in userland if necessary.
  
-The bundled RNGs are as follows: +Examples of these uses are as follows:
- +
-  * XorShift128+: 64-bit, reproducible, PRNG. +
-  * MT19937: 32-bit, reproducible, PRNG, compatible mt_srand() / mt_rand(). +
-  * secure: 64-bit, non-reproducible, CSPRNG, uses php_random_bytes() internally. +
-  * user: user defined. +
- +
-By default, XorShift128+ is used. It can generate 64-bit values, is used by major browsers, and is fast and reliable. On the other hand, MT19937 is retained for compatibility. +
- +
-"secure" is practically equivalent to random_int(), but can be used to shuffle arrays and strings. Also, since it is an object, it is easy to exchange implementations. +
- +
-"user" is special, and in order to use it, you need to inherit from the Random class and implement the next(): int method. This can be used to implement a generic RNG in userland, and can be used for unit tests where you want to generate a fixed number.+
  
 <code php> <code php>
 +// Use different RNGs for different environments.
 +$rng = $is_production
 +    ? new Random\NumberGenerator\Secure()
 +    : new Random\NumberGenerator\XorShift128Plus(1234);
  
-class FixedNumberRandom extends Random +$randomizer new Random\Randomizer($rng); 
-+$randomizer->shuffleString('foobar');
-    protected int $current 0; +
- +
-    public function __construct(+
-    { +
-        parent::__construct(RANDOM_USER); +
-    } +
- +
-    protected function next(): int +
-    { +
-        return ++$this->current; +
-    } +
-+
- +
-function foobar(Random $random): void { +
-    for ($i = 0$i < 9; $i++) { +
-        echo $random->nextInt(); +
-    } +
-+
- +
-foobar(new FixedNumberRandom()); // Results: 123456789+
 </code> </code>
  
-Algorithms can be registered and unregistered using PHP's C API, and new implementations can be added via ExtensionThe following APIs are provided:+<code php> 
 +// Safely migrate the existing mt_rand() state.
  
-<code c> +// before 
-/* Register new RNG algo *+mt_srand(1234, MT_RAND_PHP); 
-int php_random_class_algo_register(const php_random_class_algo *algo);+foobar(); 
 +$result = str_shuffle('foobar');
  
-/* Unregister RNG algo *+// after 
-void php_random_class_algo_unregister(const char *ident); +$randomizer = new Random\Randomizer(new Random\NumberGenerator\MersenneTwister(1234, MT_RAND_PHP)); 
- +foobar(); 
-/* Find and get registered algo */ +$result = $randomizer->stringShuffle('foobar');
-const php_random_class_algo* php_random_class_algo_find(const zend_string *ident);+
 </code> </code>
  
-Alsoas with MT, various alternative APIs using object scope RNGs will be provided.+As a side effect of this RFCthe following PHP functions have been moved to the new ext/random extension
  
-<code c> +This is because ext/standard/random.c reserves the name RANDOM and cannot be used by the extension. 
-/* similar php_mt_rand() *+In additionall RNG-related implementations will be moved to the new random extension in order to standardize the RNG implementation.
-uint64_t php_random_class_next(php_random_class *random_classzend_object *obj);+
  
-/similar php_mt_rand_range() */ +  lcg_value() 
-zend_long php_random_class_range(php_random_class *random_class, zend_object *obj, zend_long min, zend_long max);+  srand() 
 +  * rand(
 +  mt_srand() 
 +  mt_rand() 
 +  * random_int() 
 +  * random_bytes()
  
-/* similar php_array_data_shuffle() */ +The following internal APIs will also be moved to the ext/random extension:
-void php_random_class_array_data_shuffle(php_random_class *random_class, zend_object *obj, zval *array);+
  
-/similar php_string_shuffle() */ +  php_random_int_throw() 
-void php_random_class_string_shuffle(php_random_class *random_class, zend_object *obj, char *str, zend_long len); +  php_random_int_silent() 
-</code>+  * php_combined_lcg(
 +  php_mt_srand() 
 +  php_mt_rand() 
 +  php_mt_rand_range(
 +  * php_mt_rand_common() 
 +  * php_srand() 
 +  * php_rand() 
 +  * php_random_bytes() 
 +  * php_random_int()
  
-Random class can be serialized or cloned if the algorithm supports it. This is useful for storing and reusing state.+All of these features are available from the extension by simply including a single ext/random/php_random.h
  
-<code php> +The following header files are left in for extension compatibility. The contents all include ext/random/php_random.h.
-// serialize +
-$foo = new Random(); +
-for ($i = 0; $i < 10; $i++) { $foo->nextInt();+
-var_dump(unserialize(serialize($foo))->nextInt() === $foo->nextInt()); // true+
  
-// clone +  * ext/standard/php_lcg.h 
-$bar = new Random(); +  * ext/standard/php_rand.h 
-for ($i = 0; $i < 10; $i++) { $bar->nextInt();+  * ext/standard/php_mt_rand.h 
-$baz = clone $bar; +  * ext/standard/php_random.h
-var_dump($baz->nextInt() === $bar->nextInt()); // true +
-</code>+
  
-Using this feature, the first example can be rewritten as follows:+===== Future Scope =====
  
-<code php> +These are not within the scope of this RFCbut are worth considering in the future:
-echo foo(1234function ()void {}) . PHP_EOL; // Result: 1480009472 +
-echo foo(1234, function (): void { mt_rand(); }) . PHP_EOL; // Result: 1480009472+
  
-function foo(int $seedcallable $bar): int { +  * Remove old header files for compatibility (php_lcg.hphp_rand.h, php_mt_rand.hphp_random.h
-    $random = new Random(RANDOM_MT19937$seed); +  * Deprecate lcg_value(), mt_srand(), srand()
-    $result = $random->nextInt()+
-    $bar()+
-    $result += $random->nextInt()+
-    return $result; +
-+
-</code>+
  
 ===== Backward Incompatible Changes ===== ===== Backward Incompatible Changes =====
-The following items will no longer be available: 
  
-  - "Random" class name +The following names have been reserved and will no longer be available 
-  "RANDOM_XORSHIFT128PLUSconstant + 
-  "RANDOM_MT19937constant +  * "Random" 
-  "RANDOM_SECUREconstant +  "Random\NumberGenerator
-  "RANDOM_USERconstant+  "Random\NumberGenerator\XorShift128Plus
 +  "Random\NumberGenerator\MersenneTwister
 +  "Random\NumberGenerator\CombinedLCG" 
 +  * "Random\NumberGenerator\Secure"
  
 ===== Proposed PHP Version(s) ===== ===== Proposed PHP Version(s) =====
-8.1+8.2
  
 ===== RFC Impact ===== ===== RFC Impact =====
Line 241: Line 166:
  
 ==== To Existing Extensions ==== ==== To Existing Extensions ====
-none+In the future, it may be necessary to change the included header files to point to ext/random/php_random.h. However, compatibility will be maintained for now.
  
 ==== To Opcache ==== ==== To Opcache ====
Line 247: Line 172:
  
 ==== New Constants ==== ==== New Constants ====
-  * RANDOM_XORSHIFT128PLUS +none
-  * RANDOM_MT19937 +
-  * RANDOM_SECURE +
-  * RANDOM_USER+
  
 ==== php.ini Defaults ==== ==== php.ini Defaults ====
-  * random.ignore_generated_size_exceeded=0 +none
- +
-If 1, implicit truncation will be performed when a 64-bit value is generated in a 32-bit environment. If 0, an exception will be thrown.+
  
 ===== Open Issues ===== ===== Open Issues =====
Line 261: Line 181:
  
 ===== Vote ===== ===== Vote =====
-Voting opens 2021-MM-DD and 2021-MM-DD at 00:00:00 EDT. 2/3 required to accept.+Voting opens 2022-MM-DD and 2021-MM-DD at 00:00:00 EDT. 2/3 required to accept.
  
-<doodle title="Add Random class" auth="zeriyoshi" voteType="single" closed="true"> +<doodle title="Add Random extension" auth="zeriyoshi" voteType="single" closed="true"> 
    * Yes    * Yes
    * No    * No
Line 269: Line 189:
  
 ===== Patches and Tests ===== ===== Patches and Tests =====
-  * https://github.com/php/php-src/pull/7079+  * https://github.com/php/php-src/pull/XXXX
rfc/rng_extension.txt · Last modified: 2022/06/28 02:36 by zeriyoshi