rfc:multibyte_char_handling

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
rfc:multibyte_char_handling [2014/02/03 08:59]
yohgaki
rfc:multibyte_char_handling [2017/09/22 13:28] (current)
Line 1: Line 1:
  
 ====== PHP RFC: Multibyte Char Handling ====== ====== PHP RFC: Multibyte Char Handling ======
-  * Version: 1.2 +  * Version: 1.3 
-  * Date: 2014-01-28+  * Date: 2014-02-03
   * Author: Yasuo Ohgaki, yohgaki@ohgaki.net ​   * Author: Yasuo Ohgaki, yohgaki@ohgaki.net ​
-  * Status: ​Under discussion+  * Status: ​Declined
   * First Published at: http://​wiki.php.net/​rfc/​multibyte_char_handling   * First Published at: http://​wiki.php.net/​rfc/​multibyte_char_handling
  
Line 24: Line 24:
  
 PHP, including released versions, needs secure addslashes()/​var_export()/​stripslashes(). ​ PHP, including released versions, needs secure addslashes()/​var_export()/​stripslashes(). ​
- 
-Compile mbstringp-ng as default compiled module, when mbstring-ng is ready. See following FRC for mbstring-ng details. 
- 
-[[https://​wiki.php.net/​rfc/​altmbstring|Alternative implementation of mbstring using ICU]] 
- 
-Until mbstring-ng is ready, mbstring-ng is provided as EXPERIMENTAL module. 
- 
-mbstring-ng implementation is subject to be changed. Vote for mbstring-ng is done separately. 
  
 ==== Add mb_addslashes()/​mb_var_export()/​mb_stripslashes() to released versions ==== ==== Add mb_addslashes()/​mb_var_export()/​mb_stripslashes() to released versions ====
  
-For PHP 5.and up, add mb_add_slashes()/​mb_var_export()/​mb_strip_slashes() has encoding option.+For PHP 5.and up, add mb_add_slashes()/​mb_var_export()/​mb_strip_slashes() has encoding option.
  
 +<code php>
   string mb_add_slashes(string $str [, string $encoding=internal_encoding])   string mb_add_slashes(string $str [, string $encoding=internal_encoding])
   string mb_strip_slashes(string $str [, $encoding=internal_encoding])   string mb_strip_slashes(string $str [, $encoding=internal_encoding])
Line 42: Line 35:
   string mb_strip_cslashes(string $str [, $encoding=internal_encoding])   string mb_strip_cslashes(string $str [, $encoding=internal_encoding])
   string mb_var_export(mixed $var [, bool $return=FALSE [, string $encoding=internal_encoding]])   string mb_var_export(mixed $var [, bool $return=FALSE [, string $encoding=internal_encoding]])
 +</​code>​
  
 addcslashes()/​stripcslashs() needs to be multibyte aware for the same reason addslashes()/​stripslashes(). addcslashes()/​stripcslashs() needs to be multibyte aware for the same reason addslashes()/​stripslashes().
Line 47: Line 41:
 ==== Add mb version of function uses php_mblen() ==== ==== Add mb version of function uses php_mblen() ====
  
-For PHP 5.and up, add mb_escape_shell_arg()/​mb_secape_shell_cmd()/​mb_fget_csv()/​etc that have extra encoding parameter like mb_add_slashes().+For PHP 5.and up, add mb_escape_shell_arg()/​mb_secape_shell_cmd()/​mb_fget_csv()/​etc that have extra encoding parameter like mb_add_slashes().
  
   * mb_escape_shell_arg()   * mb_escape_shell_arg()
Line 68: Line 62:
 Since fgetcsv() uses locale now, do the same for fgetcsv(). Since fgetcsv() uses locale now, do the same for fgetcsv().
  
-==== mbstring ​usage and implementation ​====+==== mbstring.func_overload ​====
  
-For PHP 5.3 and up, all changes done in mbstring.+Some users are annoyed by sloppy multilingual implementations using 
 +this optionThere is feature request from user who want to remove  
 +mbstring.func_overload INI option.
  
-mbstring is rather large moduleThereforeit is better to be able to build PHP without mbstringAny function uses mbstring ​feature use "#​if",​ so that PHP could be built without mbstring if there isNote that this RFC only use mbstring feature ​in mbstring ​module.+https://​bugs.php.net/​bug.php?​id=65785 
 + 
 +Howeverfunc_overload ​is extended for now. 
 + 
 + 
 +==== mbstring ​usage and implementation ==== 
 + 
 +For PHP 5.4 and up, all changes done in mbstring.
  
 mbstring functions have history of remain insecure when single byte version of function'​s issue has been fixed. e.g. mb_prase_str(),​ mb_send_mail() Refactoring is preffered to avoid this issue, but refactoring is postponed until PHP6. i.e. There would be 2 codes that are mostly the same.  mbstring functions have history of remain insecure when single byte version of function'​s issue has been fixed. e.g. mb_prase_str(),​ mb_send_mail() Refactoring is preffered to avoid this issue, but refactoring is postponed until PHP6. i.e. There would be 2 codes that are mostly the same. 
 +
 +In short, if some one fixes related function, do not forget to update mbstring code also.
  
 ===== Note about short and long term resolution ===== ===== Note about short and long term resolution =====
  
-Main objective ​of this RFC is to remove vulnerability like CVE-2014-1239 ​completely.+  * **Short term resolution**:​ Add required function to mbstring 
 +  * **Long term resolution**:​ Replace mbstring with mbstring-ng to provide multibyte aware functions by default. mbstring-ng does not have license issue. 
 + 
 +This RFC is for short term resolution. 
 + 
 +Main objective is to remove vulnerability like CVE-2014-1239.
 To accomplish this objective, we need multibyte aware function by default which we To accomplish this objective, we need multibyte aware function by default which we
-don't have it right now.+don't have it now.
  
-We may compile current mbstring by default, but there is license issue for some +To remove vulnerability like CVE-2014-129 from user scripts, there must be multibyte aware functions by default. ​We may compile current mbstring by default, but there is license issue for some users. mbstring-ng does not have such issue and it is preferred to use it as default with respect to license, but it's far from complete.
-users. mbstring-ng does not have such issue and it is preferred to use it as default +
-with respect to license, but it's far from complete.+
  
-Since there is no feasible option right now, I'm proposing load map for short and long  +Since there is no feasible option right now, short and long term resolution ​is needed.
-term resolution.+
  
-Please take mbstring-ng part as load map for long term resolution. ​ 
-There is "No BC issue" for short term resolution. (Add some functions to mbstring) 
-I agree we would have BC issue for long term resolution. (Replace mbstring by mbstring-ng) 
  
-When mbstring-ng development is finished, we should have vote whether mbstring 
-is replaced by mbstring-ng or not. 
  
 ===== Backward Incompatible Changes ===== ===== Backward Incompatible Changes =====
  
-None for short term resolution. (Adding functions to mbstring+None. (Adding functions to mbstring)
- +
-Some for long term resolution. (Replacing mbstring by mbstring-ng)+
  
 ===== Proposed PHP Version(s) ===== ===== Proposed PHP Version(s) =====
  
-  * PHP 5.and up - Introduce additional mb_*() functions +  * PHP 5.and up - Introduce additional mb_*() functions
-  * PHP 5.6 or up - Introduce mbstring-ng and compile it as default compiled module when it's ready.+
  
-If mbstring-ng is not compiled as default until it's ready. mbstring co-exists until we are confident with mbstring-ng. For PHP 5.6, mbstring-ng will be EXPERIMENTAL module probably.+===== Future Scope =====
  
 +  * mbstring may be replaced by mbstring-ng in future release and mbstring may be moved to PECL.
  
-===== Impact to Existing Extensions =====+There is other RFC for introducing mbstring-ng as a EXPERIMENTAL module.
  
-  * mbstring will be replaced by mbstring-ng ​in future release and mbstring is moved to PECL.+When mbstring-ng development is finished, there will be a vote whether mbstring 
 +is replaced by mbstring-ng ​or not
  
 ===== Open Issues ===== ===== Open Issues =====
- 
-Use of mbstring.func_overload INI for overriding single byte string functions by mbstring functions is left open issue for future releases. 
- 
-Some users are annoyed by sloppy multilingual implementations using 
-this option. There is feature request from user who want to remove ​ 
-mbstring.func_overload INI option. 
- 
-https://​bugs.php.net/​bug.php?​id=65785 
- 
-However, func_overload is extended for now. 
  
 ===== Proposed Voting Choices ===== ===== Proposed Voting Choices =====
  
-Yas/No+Yes/No
  
 ===== Patches and Tests ===== ===== Patches and Tests =====
  
   * Prepared for review after vote.   * Prepared for review after vote.
-  * mbstring-ng is in https://​wiki.php.net/​rfc/​altmbstring 
-  * When mbstring-ng development is finished, it is reviewed and have vote for replacing mbstring. 
  
 ===== Vote ==== ===== Vote ====
  
-VOTE: 2014/02/XX - 2014/02/XX+VOTE: 2014/02/10 - 2014/02/17
  
-This vote is only for adding new mb_*() functions to released versions. mbstring-ng vote is done separately.+This vote is only for adding new mb_*() functions to released versions. ​
  
 <doodle title="​Add required mb_*() functions to fix vulnerability"​ auth="​yohgaki"​ voteType="​single"​ closed="​true">​ <doodle title="​Add required mb_*() functions to fix vulnerability"​ auth="​yohgaki"​ voteType="​single"​ closed="​true">​
Line 146: Line 135:
    * No    * No
 </​doodle>​ </​doodle>​
 +
 +Thank you for voting. ​
 +
 +**If you vote No for this, please provide alternative short term resolution for CVE-2014-1239.**
  
 ===== Implementation ===== ===== Implementation =====
Line 155: Line 148:
  
 ===== References ===== ===== References =====
 +
 +Discussions
 +  * http://​marc.info/?​l=php-internals&​m=138982990932300&​w=2
 +  * http://​marc.info/?​l=php-internals&​m=139069591127118&​w=2
 +  * http://​marc.info/?​l=php-internals&​m=139018752615166&​w=2
  
 Related RFC Related RFC
rfc/multibyte_char_handling.1391417948.txt.gz · Last modified: 2017/09/22 13:28 (external edit)