rfc:multibyte_char_handling

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
rfc:multibyte_char_handling [2014/02/02 02:32]
yohgaki
rfc:multibyte_char_handling [2017/09/22 13:28] (current)
Line 1: Line 1:
  
 ====== PHP RFC: Multibyte Char Handling ====== ====== PHP RFC: Multibyte Char Handling ======
-  * Version: 1.2 +  * Version: 1.3 
-  * Date: 2014-01-28+  * Date: 2014-02-03
   * Author: Yasuo Ohgaki, yohgaki@ohgaki.net ​   * Author: Yasuo Ohgaki, yohgaki@ohgaki.net ​
-  * Status: ​Under discussion+  * Status: ​Declined
   * First Published at: http://​wiki.php.net/​rfc/​multibyte_char_handling   * First Published at: http://​wiki.php.net/​rfc/​multibyte_char_handling
  
Line 24: Line 24:
  
 PHP, including released versions, needs secure addslashes()/​var_export()/​stripslashes(). ​ PHP, including released versions, needs secure addslashes()/​var_export()/​stripslashes(). ​
- 
-Compile mbstringp-ng as default compiled module, when mbstring-ng is ready. See following FRC for mbstring-ng details. 
- 
-[[https://​wiki.php.net/​rfc/​altmbstring|Alternative implementation of mbstring using ICU]] 
- 
-Until mbstring-ng is ready, mbstring-ng is provided as EXPERIMENTAL module. 
- 
-mbstring-ng implementation is subject to be changed. Vote for mbstring-ng is done separately. 
  
 ==== Add mb_addslashes()/​mb_var_export()/​mb_stripslashes() to released versions ==== ==== Add mb_addslashes()/​mb_var_export()/​mb_stripslashes() to released versions ====
  
-For PHP 5.and up, add mb_add_slashes()/​mb_var_export()/​mb_strip_slashes() has encoding option.+For PHP 5.and up, add mb_add_slashes()/​mb_var_export()/​mb_strip_slashes() has encoding option.
  
 +<code php>
   string mb_add_slashes(string $str [, string $encoding=internal_encoding])   string mb_add_slashes(string $str [, string $encoding=internal_encoding])
 +  string mb_strip_slashes(string $str [, $encoding=internal_encoding])
 +  string mb_add_cslashes(string $str [, string $encoding=internal_encoding])
 +  string mb_strip_cslashes(string $str [, $encoding=internal_encoding])
   string mb_var_export(mixed $var [, bool $return=FALSE [, string $encoding=internal_encoding]])   string mb_var_export(mixed $var [, bool $return=FALSE [, string $encoding=internal_encoding]])
-  string mb_strip_slashes(string $str [, $encoding=internal_encoding])+</​code>​ 
 + 
 +addcslashes()/​stripcslashs() needs to be multibyte aware for the same reason addslashes()/​stripslashes().
  
 ==== Add mb version of function uses php_mblen() ==== ==== Add mb version of function uses php_mblen() ====
  
-For PHP 5.and up, add mb_escape_shell_arg()/​mb_secape_shell_cmd()/​mb_fget_csv()/​etc that have extra encoding parameter like mb_add_slashes().+For PHP 5.and up, add mb_escape_shell_arg()/​mb_secape_shell_cmd()/​mb_fget_csv()/​etc that have extra encoding parameter like mb_add_slashes().
  
-  * mb_escape_shell_arg +  * mb_escape_shell_arg() 
-  * mb_escape_shsell_cmd +  * mb_escape_shsell_cmd() 
-  * functions/​methods use php_fgetcsv API http://​lxr.php.net/​search?​q=php_fgetcsv&​project=PHP_5_5+  * functions/​methods use [[http://​lxr.php.net/​search?​q=php_fgetcsv&​project=PHP_5_5|php_fgetcsv API]] 
 +    * mb_file_get_csv() 
 +    * mb_file_put_csv() ​
  
 Reference Reference
Line 56: Line 56:
 Functions that should use locale are Functions that should use locale are
  
- ​- ​mb_escape_shell_arg() +  * mb_escape_shell_arg() 
- mb_escape_shell_cmd()+  ​* ​mb_escape_shell_cmd()
  
 These function may override locale by encoding parameter. These function may override locale by encoding parameter.
 Since fgetcsv() uses locale now, do the same for fgetcsv(). Since fgetcsv() uses locale now, do the same for fgetcsv().
  
-==== mbstring ​usage and implementation ​====+==== mbstring.func_overload ​====
  
-For PHP 5.3 and up, all changes done in mbstring.+Some users are annoyed by sloppy multilingual implementations using 
 +this optionThere is feature request from user who want to remove  
 +mbstring.func_overload INI option. 
 + 
 +https://​bugs.php.net/​bug.php?​id=65785 
 + 
 +However, func_overload is extended for now. 
 + 
 + 
 +==== mbstring usage and implementation ====
  
-mbstring is rather large module. Therefore, it is better to be able to build PHP without mbstringAny function uses mbstring feature use "#​if"​so that PHP could be built without mbstring if there is. Note that this RFC only use mbstring feature ​in mbstring ​module.+For PHP 5.4 and upall changes done in mbstring.
  
 mbstring functions have history of remain insecure when single byte version of function'​s issue has been fixed. e.g. mb_prase_str(),​ mb_send_mail() Refactoring is preffered to avoid this issue, but refactoring is postponed until PHP6. i.e. There would be 2 codes that are mostly the same.  mbstring functions have history of remain insecure when single byte version of function'​s issue has been fixed. e.g. mb_prase_str(),​ mb_send_mail() Refactoring is preffered to avoid this issue, but refactoring is postponed until PHP6. i.e. There would be 2 codes that are mostly the same. 
 +
 +In short, if some one fixes related function, do not forget to update mbstring code also.
  
 ===== Note about short and long term resolution ===== ===== Note about short and long term resolution =====
  
-Main objective ​of this RFC is to remove vulnerability like CVE-2014-1239 ​completely.+  * **Short term resolution**:​ Add required function to mbstring 
 +  * **Long term resolution**:​ Replace mbstring with mbstring-ng to provide multibyte aware functions by default. mbstring-ng does not have license issue. 
 + 
 +This RFC is for short term resolution. 
 + 
 +Main objective is to remove vulnerability like CVE-2014-1239.
 To accomplish this objective, we need multibyte aware function by default which we To accomplish this objective, we need multibyte aware function by default which we
-don't have it right now.+don't have it now.
  
-We may compile current mbstring by default, but there is license issue for some +To remove vulnerability like CVE-2014-129 from user scripts, there must be multibyte aware functions by default. ​We may compile current mbstring by default, but there is license issue for some users. mbstring-ng does not have such issue and it is preferred to use it as default with respect to license, but it's far from complete.
-users. mbstring-ng does not have such issue and it is preferred to use it as default +
-with respect to license, but it's far from complete.+
  
-Since there is no feasible option right now, I'm proposing load map for short and long  +Since there is no feasible option right now, short and long term resolution ​is needed.
-term resolution.+
  
-Please take mbstring-ng part as load map for long term resolution. ​ 
-There is "No BC issue" for short term resolution. (Add some functions to mbstring) 
-I agree we would have BC issue for long term resolution. (Replace mbstring by mbstring-ng) 
  
-When mbstring-ng development is finished, we should have vote whether mbstring 
-is replaced by mbstring-ng or not. 
  
 ===== Backward Incompatible Changes ===== ===== Backward Incompatible Changes =====
  
-None for short term resolution. (Adding functions to mbstring+None. (Adding functions to mbstring)
- +
-Some for long term resolution. (Replacing mbstring by mbstring-ng)+
  
 ===== Proposed PHP Version(s) ===== ===== Proposed PHP Version(s) =====
  
-  * PHP 5.and up - Introduce additional mb_*() functions +  * PHP 5.and up - Introduce additional mb_*() functions
-  * PHP 5.6 or up - Introduce mbstring-ng and compile it as default compiled module when it's ready.+
  
-If mbstring-ng is not compiled as default until it's ready. mbstring co-exists until we are confident with mbstring-ng. For PHP 5.6, mbstring-ng will be EXPERIMENTAL module probably.+===== Future Scope =====
  
 +  * mbstring may be replaced by mbstring-ng in future release and mbstring may be moved to PECL.
  
-===== Impact to Existing Extensions =====+There is other RFC for introducing mbstring-ng as a EXPERIMENTAL module.
  
-  * mbstring will be replaced by mbstring-ng ​in future release and mbstring is moved to PECL.+When mbstring-ng development is finished, there will be a vote whether mbstring 
 +is replaced by mbstring-ng ​or not
  
 ===== Open Issues ===== ===== Open Issues =====
- 
-Use of mbstring.func_overload INI for overriding single byte string functions by mbstring functions is left open issue for future releases. 
- 
-Some users are annoyed by sloppy multilingual implementations using 
-this option. There is feature request from user who want to remove ​ 
-mbstring.func_overload INI option. 
- 
-https://​bugs.php.net/​bug.php?​id=65785 
- 
-However, func_overload is extended for now. 
  
 ===== Proposed Voting Choices ===== ===== Proposed Voting Choices =====
  
-Yas/No+Yes/No
  
 ===== Patches and Tests ===== ===== Patches and Tests =====
  
   * Prepared for review after vote.   * Prepared for review after vote.
-  * mbstring-ng is in https://​wiki.php.net/​rfc/​altmbstring 
-  * When mbstring-ng development is finished, it is reviewed and have vote for replacing mbstring. 
  
 ===== Vote ==== ===== Vote ====
  
-VOTE: 2014/01/26 - 2014/02/09+VOTE: 2014/02/10 - 2014/02/17
  
-This vote is only for adding new mb_*() functions to released versions. mbstring-ng vote is done separately.+This vote is only for adding new mb_*() functions to released versions. ​
  
-<doodle title="​Add required mb_*() functions ​and prepare mbstring-ng as a default compiled module" auth="​yohgaki"​ voteType="​single"​ closed="​false">+<doodle title="​Add required mb_*() functions ​to fix vulnerability" auth="​yohgaki"​ voteType="​single"​ closed="​true">
    * Yes    * Yes
    * No    * No
 </​doodle>​ </​doodle>​
 +
 +Thank you for voting. ​
 +
 +**If you vote No for this, please provide alternative short term resolution for CVE-2014-1239.**
  
 ===== Implementation ===== ===== Implementation =====
Line 149: Line 148:
  
 ===== References ===== ===== References =====
 +
 +Discussions
 +  * http://​marc.info/?​l=php-internals&​m=138982990932300&​w=2
 +  * http://​marc.info/?​l=php-internals&​m=139069591127118&​w=2
 +  * http://​marc.info/?​l=php-internals&​m=139018752615166&​w=2
  
 Related RFC Related RFC
rfc/multibyte_char_handling.1391308343.txt.gz · Last modified: 2017/09/22 13:28 (external edit)