phpng

PHPNG (next generation)

This page gives short information about development state of a new PHP branch based on refactored Zend Engine.

The phpng branch has been merged into master and has been used as the base for PHP 7.0

Some technical implementation details are available at phpng-int. Information for extension maintainers at phpng-upgrading.

Slides from a talk at ZendCon-2014 phpng-php7.pdf

How to Download, Build, Install and Test

Note: For anonymous checkout, use https://git.php.net/repository/php-src.git instead of git@git.php.net:php-src.git

mkdir ~/tmp
cd ~/tmp
git clone git@git.php.net:php-src.git
cd php-src
./buildconf
./configure \
    --prefix=$HOME/tmp/usr \
    --with-config-file-path=$HOME/tmp/usr/etc \
    --enable-mbstring \
    --enable-zip \
    --enable-bcmath \
    --enable-pcntl \
    --enable-ftp \
    --enable-exif \
    --enable-calendar \
    --enable-sysvmsg \
    --enable-sysvsem \
    --enable-sysvshm \
    --enable-wddx \
    --with-curl \
    --with-mcrypt \
    --with-iconv \
    --with-gmp \
    --with-pspell \
    --with-gd \
    --with-jpeg-dir=/usr \
    --with-png-dir=/usr \
    --with-zlib-dir=/usr \
    --with-xpm-dir=/usr \
    --with-freetype-dir=/usr \
    --with-t1lib=/usr \
    --enable-gd-native-ttf \
    --enable-gd-jis-conv \
    --with-openssl \
    --with-mysql=/usr \
    --with-pdo-mysql=/usr \
    --with-gettext=/usr \
    --with-zlib=/usr \
    --with-bz2=/usr \
    --with-recode=/usr \
    --with-mysqli=/usr/bin/mysql_config
make
make install
mkdir $HOME/tmp/usr/etc
vi $HOME/tmp/usr/etc/php.ini
max_execution_time=600
memory_limit=128M
error_reporting=0
display_errors=0
log_errors=0
user_ini.filename=
realpath_cache_size=2M
cgi.check_shebang_line=0

zend_extension=opcache.so
opcache.enable_cli=1
opcache.save_comments=0
opcache.fast_shutdown=1
opcache.validate_timestamps=1
opcache.revalidate_freq=60
opcache.use_cwd=1
opcache.max_accelerated_files=100000
opcache.max_wasted_percentage=5
opcache.memory_consumption=128
opcache.consistency_checks=0

Now PHP must be installed and configured. The final check.

$ sapi/cli/php -v
PHP 7.0.0-dev (cli) (built: Apr 28 2014 11:13:04) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.7.0-dev, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies

You may run synthetic benchmarks

$ sapi/cli/php Zend/bench.php
$ sapi/cli/php Zend/micro_bench.php

You may measure the performance of some web applications

$ sapi/cgi/php-cgi -T 1000 /var/www/html/wordpress/index.php > /dev/null''

You may do the same with callgrind profiler

$ valgrind --tool=callgrind --dump-instr=yes --separate-recs=1 sapi/cgi/php-cgi -T 100 /var/www/html/wordpress/index.php > /dev/null

Finally, you may run PHP as FastCGI server using CGI SAPI or FPM (it requires additional configuration)

$ PHP_FCGI_CHILDREN=4 PHP_FCGI_MAX_REQUESTS=0 sapi/cgi/php-cgi -b /tmp/fcgi-php

I would recommend build PHP from master and phpng branches and compare their performance

Supported SAPI

  • cli
  • cgi
  • fpm
  • apache (FastCGI and FPM might be significantly faster if mod_php is built as PIC)
  • apache2handler

Supported Extensions

  • bcmath
  • bz2
  • calendar
  • com_dotnet
  • ctype
  • curl
  • date
  • dba
  • dom
  • enchant
  • ereg
  • exif
  • fileinfo
  • filter
  • ftp
  • gd
  • gettext
  • gmp
  • hash
  • iconv
  • imap
  • intl
  • json
  • ldap
  • libxml
  • mbstring
  • mcrypt
  • mysql
  • mysqli
  • mysqlnd
  • odbc (tested with unixODBC and MySQL driver)
  • openssl
  • OPcache
  • pcntl
  • pcre
  • PDO
  • pdo_firebird
  • pdo_mysql
  • PDO_ODBC (tested with unixODBC and MySQL driver)
  • pdo_pgsql
  • pdo_sqlite
  • pgsql
  • Phar
  • posix
  • pspell
  • readline
  • recode
  • Reflection
  • session
  • shmop
  • SimpleXML
  • snmp
  • soap
  • sockets
  • SPL
  • sqlite3
  • standard
  • sysvmsg
  • sysvsem
  • sysvshm
  • tidy
  • tokenizer
  • wddx
  • xml
  • xmlreader
  • xmlwriter
  • xsl
  • zip
  • zlib

Unsupported Extensions (not converted yet)

  • interbase
  • mssql
  • oci8
  • pdo_dblib
  • pdo_oci
  • sybase_ct

Incompatibilities (made on purpose and are not going to be fixed)

  1. PHPNG doesn't keep original values of arguments passed to user functions, so func_get_arg() and func_get_args() will return current value of argument instead of the actually passed. The following code is going to be affected “function foo($x) { $x = 2; return func_get_arg(0);} var_dump(foo(1));”
    • tests/lang/func_get_arg.001.phpt
    • tests/lang/func_get_args.004.phpt
  2. Function parameters with duplicate name are not allowed anymore. Definitions like “function foo($x,$x) {}” will lead to compile time error “Redefinition of parameter”
    • Zend/tests/bug54013.phpt
    • Zend/tests/bug64515.phpt
  3. var_dump() may print recursive data structures differently
    • Zend/tests/closure_026.phpt
  4. Passing scalars by reference may behave differently, because they don't have reference counters anymore

Known Problems

  • foreach may behave differently
    • tests/lang/foreachLoop.013.phpt
    • tests/lang/foreachLoop.014.phpt
    • tests/lang/foreachLoop.015.phpt
    • ext/standard/tests/serialize/serialization_arrays_005.phpt
  • user code in error handler may cause destruction of array, element of which is currently processed
    • Zend/tests/bug54265.phpt
  • _toString() may behave differently
    • Zend/tests/bug60825.phpt (it was a bogus behavior in php5.6 and below)
  • Internal object constructors can't set $this to NULL
    • ext/fileinfo/tests/bug61173.phpt
    • ext/fileinfo/tests/finfo_open_error.phpt
    • ext/pdo_mysql/tests/pdo_mysql_construct.phpt
    • ext/pdo_mysql/tests/pdo_mysql_construct_options.phpt
    • ext/intl/tests/breakiter_construct_error.phpt
    • ext/intl/tests/bug62017.phpt
    • ext/intl/tests/dateformat_construct_bad_tz_cal.phpt
    • ext/intl/tests/formatter_format2.phpt
    • ext/intl/tests/gregoriancalendar_construct_error.phpt
    • ext/intl/tests/resourcebundle_create.phpt
  • Internal object constructors can't set $this to NULL when called indirectly - parent::__construct()
    • ext/date/tests/bug67118.phpt
    • ext/date/tests/bug67118_2.phpt
  • PDP Persistent connection handling
    • ext/pdo_mysql/tests/bug63176.phpt
    • ext/pdo_mysql/tests/pdo_mysql_pconnect.phpt
    • ext/pdo_sqlite/tests/bug43831.phpt
  • Passing scalars by reference may behave differently, because they don't have reference counters anymore
    • Zend/tests/bug35393.phpt
    • ext/pdo_firebird/tests/pdo_005.phpt
    • ext/pdo_mysql/tests/pdo_005.phpt
    • ext/pdo_odbc/tests/pdo_005.phpt
    • ext/pdo_pgsql/tests/pdo_005.phpt
    • ext/pdo_sqlite/tests/pdo_005.phpt
  • We can't catch situations when argument passed to internal function was modified by callback
    • ext/standard/tests/array/unexpected_array_mod_bug.phpt
    • ext/standard/tests/strings/bug55871.phpt (it was a bogus behavior in php5.6 and below)
  • Memory or resource leaks
    • ext/standard/tests/http/bug60570.phpt
  • Inconsistent reference behaviour <?php $a['x'] = 'foo'; $a['x'] = &$a; $a = array(); $a['x'] = $a; var_dump($a); ?>
    • ext/wddx/tests/bug48562.phpt
  • Bug in GC
    • Zend/tests/bug63635.phpt (visible with valgrind)
  • GC doesn't destroy inner-loops
    • ext/standard/tests/array/compact_variation1.phpt
  • Improper resource destruction
    • ext/standard/tests/streams/bug61115.phpt (visible with valgrind)
  • Arrays are not allowed in class constants
    • Zend/tests/bug67368.phpt
    • Zend/tests/constant_expressions_self_referencing_array.phpt
  • Improperly detected resource leaks during pharcmd build
  • Memory leak if pdo persistent dbh lost connection to server
  • call_user_func accepts some arguments for by-ref params that it did not previously. Reproduce code: $i = 1; $a = array(4, 3, 2, $i); var_dump(call_user_func("sort", $a));

Future Engine Optimization Ideas

  • Split IS_BOOL into IS_FALSE and IS_TRUE.
  • Introduce immutable arrays. They don't need to be copied and might be used directly from opcache shared memory. They might be also useful for APCU.
  • Bundle libpcre with JIT support. ext/pcre must use it out of the box.
  • Think about parameter passing/receiving. Currently parameters passed to user functions are copied twice. First by SEND and then by RECV.
  • Refactor zend_parse_parameters() API (It's slow like a hell. for ord() function it takes 90% of CPU time). See PHP RFC: Fast Parameter Parsing API
  • Convert some simple and often used internal functions into more efficient opcodes - e.g. defined(), strlen(), is_array()…
  • Replace Zend Memory Manager with xx_malloc (it must give us at least additional 2% improvement)
  • Use interned zend_strings for often used strings in C code.
  • arg_info→name should be zend_string*, but we have to initialize it from C code.
  • Do we need HashTable for op_array→static_variables? May be we can use a plain C array instead.
  • Think about CALL/RETURN related optimizations. Function call/return code is the main consumer of CPU time now.
  • Try to optimize VM instructions dispatching. May be use global register for “execute_data”. May be also keep “opline” in CPU register?
  • move zend_op.lineno into a separate compressed debug_info table.
  • Try to reduce amount of data copied from OPCache SHM into process memory on each request
  • Try to replace ext/json with pecl/jsond

Performance Evaluation

We constantly measure the performance evaluation using the following 4 tests

  • bench.php (sec) - time taken to execute Zend/bench.php
sapi/cli/php ../Zend/bench.php
  • bench.php (instr) - number of CPU instructions for Zend/bench.php execution
valgrind --tool=callgrind sapi/cli/php ../Zend/bench.php
  • WordPress (sec) - time taken to perform 1000 requests to wordpress-3.6.0 home page
sapi/cgi/php-cgi -T 1000 /var/www/html/bench/wordpress-3.6/index.php > /dev/null
  • WordPress (instr) - number of CPU instructions for 100 requests to wordpress-3.6.0 home page
valgrind --tool=callgrind sapi/cgi/php-cgi -T 100 /var/www/html/bench/wordpress-3.6/index.php > /dev/null
date bench.php (sec) bench.php (instr) WordPress (sec) WordPress (instr) Comments
20.01.2014 2.115 11,066,515,044 26.756 9,413,106,833Master branch before split
18.02.2014 1.898 11,120,470,518 Refactored engine is able to run bench.php wthout OPCache
21.02.2014 1.830 10,969,942,729
28.02.2014 1.941 11,273,346,956
07.03.2014 1.906 11,280,441,681
14.03.2014 1.941 11,251,497,687
21.03.2014 1.941 11,309,008,450
28.03.2014 1.821 10,388,772,172 Engine and core extensions mainly work
04.04.2014 1.571 8,794,394,256 23.676 7,472,466,470OPCache works without fast_shutdown
11.04.2014 1.559 8,666,353,378 22.592 7,186,015,829
18.04.2014 1.515 8,568,559,126 21.503 6,649,116,129
25.04.2014 1.492 8,306,525,457 20.857 6,170,736,000merged with mainstream master
05.05.2014 1.414 8,168,362,858 18.957 5,090,427,768ext/json support
18.05.2014 1.490 8,203,816,181 18.913 5,131,213,25117 extensions left
26.05.2014 1.432 8,204,890,934 18.245 4,992,096,207zend_hash API optimization
03.06.2014 1.492 8,193,509,303 17.403 4,747,230,312immutable arrays
09.06.2014 1.497 8,187,248,085 16.540 4,307,808,050refcounting, PCRE with JIT
30.06.2014 1.412 7,833,554,758 15.940 4,054,164,255arena, mysqlnd, call frame
09.07.2014 1.367 7,596,383,636 15.850 3,994,171,799call frame optimization
14.07.2014 1.407 7,601,792,205 14.810 3,627,440,773fast_zpp, built-ins
15.08.2014 1.350 7,602,520,254 14.864 3,641,452,019merged into master
02.09.2014 1.302 7,534,391,801 14.150 3,407,124,896new MM, int64, AST, zend_ini
07.10.2014 1.219 7,255,184,064 13.890 3,234,526,595many small optimizations
21.11.2014 1.201 7,171,414,831 13.430 3,138,706,949many small optimizations
31.12.2014 1.159 6,867,449,070 12.629 2,899,707,051
phpng.txt · Last modified: 2015/01/31 03:55 by philstu