Table of Contents

Request for Comments: Use default_charset As Default Character Encoding

Introduction

This RFC proposes that use default_charset as default character encoding.

Current PHP does not have default encoding setting. This makes adoption of PHP 5.4 difficult, since PHP 5.4's htmlentities/htmlspecialchars is now default to UTF-8. Some applications are required to set proper encoding for htmlentities/htmlspecialchars for proper character processing. If users mixed ISO-8859-1 and UTF-8 (AND many other multibyte character encodings), it could cause security problem.

There are many encoding setting in php.ini and functions that users simply ignore and leave it alone. However, it is required to handle character encoding properly for secure programs.

Objectives of this proposal are:

  1. Setting charset in HTTP header is recommended since the first XSS advisory in 2000 Feb. by CERT and Microsoft. (Better security)
  2. There are too many encoding settings and it is better to consolidated.
  3. If we have yet another multibyte string module in the future, the new common ini settings can be used. (No more module specific INIs)

Proposal

Set default_charset=“UTF-8” as PHP default for both compiled and php.ini-* option.

Add php.input_encoding, php.internal_encoding and php.output_encoding for encoding related module/functions.

Use default_charset as default for encoding related php.ini settings and module/functions.

Not touched

PHP 5.6 and master, introduce new php.ini setting. Old iconv.*/mbstring.* php.ini parameters will be removed for master PHP6. Use of iconv.*/mbstring.* php.ini parameters raise E_DEPRECATED for 5.6 and up.

PHP 5.5

Precedence of settings

default_charset < php.* < mbstring.*/iconv.* < encoding specified by functions

Encoding name handling

mbstring and iconv have different level of support.

Notes:

Use cases

It simplify i18n applications.

Unifies *.output_encoding/*.internal_encoding/*.input_encoding setting.

Users may check default_charset see if encoding conversion is needed or not. For example, pcre/sqlite only suports UTF-8 and users may check & convert encoding as follows. 

if (ini_get('default_charset') !== 'UTF-8') {
   $str = mb_convert_encoding($str, 'UTF-8'); 
}
preg, sqlite function calls here.

escapeshellcmd/escapeshellarg/fgetcsv or like, are using locale based MBCS support via php_mblen(). These functions are out side of this RFC scope.

Database character encoding is also out side of this RFC scope.

BC issues

None when users already using UTF-8 as their encoding.

Other users may have to change “default_encoding” php.ini setting (leave it empty or set it to desired encoding)

Patch

PoC Patch https://github.com/yohgaki/php-src/compare/master-unified-encoding-setting

Vote Options

Vote

Vote start: 2013/12/20 01:00 UTC

Vote end: 2014/01/10 01:00 UTC

Default Character Encoding
Real name Yes No
ajf (ajf)  
chobieeee (chobieeee)  
indeyets (indeyets)  
lstrojny (lstrojny)  
oaass (oaass)  
pollita (pollita)  
treffynnon (treffynnon)  
willfitch (willfitch)  
yohgaki (yohgaki)  
Final result: 8 1
This poll has been closed.

References

Implementation

Changelog