The PHP CLI benchmark is a script that runs several algorithms and test runs and gives back results in forms of memory consumption or cache misses depending on which tool that is used. The algorithms that are used are similar (or clones) to the ones used in the V8 javascript benchmark and the Sunspider javascript benchmark.

Current available tools:

cachegrind, memusage

The whole benchmark requires PEAR.

For the tool memusage, linux with kernel 2.6.14 or newer is required. (/proc/PID/smaps)

For the tool cachegrind, the software valgrind (http://valgrind.org/) is required.

bench.php --tool memusage --complog -v

This would do a benchmark run in two default test-directories (microtests and tests) and print a total memory usage for the run. It will also (with the -v) display each tests memory usage. --complog will create a serialized log file that will be useful when comparing runs. The comparison log file will look like “bench_PHPVERSION_TOOL_PID.txt”.

bench.php --tool memusage --path microtests -q --complog

This run will just include the microtests path and be quiet (-q, not produce output). There will be a logfile created after this run that can be used for comparison.

bench.php --tool cachegrind --complog

This will do a cachegrind benchmark and show results of the valgrind tool cachegrind. This requires valgrind.

There are also other options available such as:

• --php PATH

Where PATH is a path to a specific PHP Binary. This is useful when running several php versions on the same machine. Default is just “php”. All files in this path that should be tested need to be names test_XXX.php.

• --memory-limit MEM

Where MEM is the maximum memory usage for the test files to consume (individually). The value is in MB, the default is 128.

• --debug

A switch to set to a slightly more verbose debug mode.

• --help

Print the default help message.

• --q

Quite mode (No output)

• --tool TOOL

Select which TOOL that is used. Default is none and current available options are memusage and cachegrind.

• --log FILE

Produces an additional logfile of the output. Default is benchlog.txt, if FILE is set, it will write to FILE.

• --complog

Writes a serialized log file for comparing runs. Looks like bench_PHPVERSION_TOOL_PID.txt

• --comparison FILES

An individual option that compares the FILES selected. FILES can be “bench_*_memusage_*.txt” or selected files, separated by spaces. They must be benched with the same tool.

The comparison output might be changed, but this is how it looks like currently. The number in the parenthesis is the percentage of the “best” value. If the best runtime is 10s, 10s would become 100%. Since I've only used 5.3.0RC4 for comparing, all results are pretty much the same.

To compare the results it's possible to use the --comparison option. In example, if you would like to compare results from 5.X with memory usage the command could be following:

bench.php --comparison bench_5.*_memusage_*

The results will look like this:

bench.php --comparison bench_5.*_memusage_*
--------------- bench.php Comparison --------------
PHP version     PHP uname                           Runtime           Private RSS    
5.3.0RC4        Linux 2.6.28-11-generic i686        147s (101%)       117.5MB (100%)  
5.3.0RC4        Linux 2.6.28-11-generic i686        147.6s (101%)     117.5MB (100%)  
5.3.0RC4        Linux 2.6.28-11-generic i686        146.3s (101%)     117.5MB (100%)  
5.3.0RC4        Linux 2.6.28-11-generic i686        145.4s (100%)     117.6MB (100%)  
--------------- bench.php End Comparison -----------

Private RSS is the summary of the processes memory usage and runtime is just the total runtime.

This is how the cachegrind comparison looks like currently:

bench.php --comparison bench_5.*_cachegrind_*
--------------- bench.php Comparison --------------
PHP version     PHP uname                           Branch Mispred.        Instructions         Cache misses        
5.3.0RC4        Linux 2.6.28-11-generic i686        1.394690e+7 (100%)     7.312482e+9 (100%)   2.419207e+7 (100%)  
5.3.0RC4        Linux 2.6.28-11-generic i686        1.394695e+7 (100%)     7.312483e+9 (100%)   2.419216e+7 (100%)  
--------------- bench.php End Comparison -----------