rfc:default_encoding

This is an old revision of the document!


Request for Comments: Use default_charset As Default Character Encoding

Introduction

This RFC proposes that use default_charset as default character encoding.

Current PHP does not have default encoding setting. This makes adoption of PHP 5.4 difficult, since PHP 5.4's htmlentities/htmlspecialchars is now default to UTF-8. Some applications are required to set proper encoding for htmlentities/htmlspecialchars for proper character processing. If users mixed ISO-8859-1 and UTF-8 (AND many other multibyte character encodings), it could cause security problem.

There are many encoding setting in php.ini and functions that users simply ignore and leave it alone. However, it is required to handle character encoding properly for secure programs.

Objectives of this proposal are:

  1. Setting charset in HTTP header is recommended since the first XSS advisory in 2000 Feb. by CERT and Microsoft. (Better security)
  2. There are too many encoding settings and it is better to consolidated.
  3. If we have yet another multibyte string module in the future, the new common ini settings can be used. (No more module specific INIs)

Proposal

Set default_charset=“UTF-8” as PHP default for both compiled and php.ini-* option.

Add php.input_encoding, php.internal_encoding and php.output_encoding for encoding related module/functions.

  • php.input_encoding (Default: empty)
  • php.internal_encoding (Default: default_charset php.ini setting)
  • php.output_encoding (Default: empty)

Use default_charset as default for encoding related php.ini settings and module/functions.

Not tuoched

  • zend.script_encoding

PHP 5.6 and master, introduce new php.ini settting. Old iconv.*/mbstring.* php.ini parameters will be removed for master.

  • php.input_encoding
  • php.internal_encoding
  • php.output_encoding
  • iconv.input_encoding (Default: php.input_encoding)
  • iconv.internal_encoding (Default: php.internal_encoding)
  • iconv.output_encoding (Default: php.output_encoding)
  • mbstring.http_input (Default: php.output_encoding)
  • mbstring.internal_encoding (Default: php.internal_encoding)
  • mbstring.http_output (Default: php.output_encoding)
  • all functions that take encoding option use php.internal_encording as default (e.g. htmlentities/mb_strlen/mb_regex/etc)

PHP 5.5

  • leave as it is now

Precedence of settings

default_charset < php.* < mbstring.*/iconv.* < encoding specified by functions

Encoding name handling

mbstring and iconv have different level of support.

Notes:

  • iconv does not have API for getting supported encoding and iconv is built with system's iconv library.
  • mbstring has API to check encoding is supported or not.
  • users are responsible to set proper encoding name. e.g. mbstring has SJIS-win, but iconv only has SJIS
  • if encoding names conflicts, users should set module specific ini to valid(non empty) value

Use cases

It simplify i18n applications.

Unifies *.output_encoding/*.internal_encoding/*.input_encoding setting.

Users may check default_charset see if encoding conversion is needed or not. For example, pcre/sqlite only suports UTF-8 and users may check & convert encoding as follows.

if (ini_get('default_charset') !== 'UTF-8') {
   $str = mb_convert_encoding($str, 'UTF-8'); 
}
preg, sqlite function calls here.

escapeshellcmd/escapeshellarg/fgetcsv or like, are using locale based MBCS support via php_mblen(). These functions are out side of this RFC scope.

Patch

Vote Options

  • Yes
  • No

Vote

Not yet

References

Changelog

  • 2013-10-31 Added objectives.
  • 2013-10-29 Update target PHP version to 5.6.0.
  • 2013-06-29 Add PoC patch and update RFC, since PHP 5.5 has been released.
  • 2012-08-31 Initial version. (yohgaki)
rfc/default_encoding.1387228315.txt.gz · Last modified: 2017/09/22 13:28 (external edit)