This is an old revision of the document!
PHP CLI benchmark
Current available tools:
cachegrind, memusage
Requirements
The whole benchmark requires PEAR.
For the tool memusage, linux with kernel 2.6.14 or newer is required. (/proc/PID/smaps)
For the tool cachegrind, the software valgrind (http://valgrind.org/) is required.
Example of usage
bench.php --tool memusage --complog -v
This would do a benchmark run in two default test-directories (microtests and tests) and print a total memory usage for the run. It will also (with the -v) display each tests memory usage. --complog will create a serialized log file that will be useful when comparing runs. The comparison log file will look like “bench_PHPVERSION_TOOL_PID.txt”.
bench.php --tool memusage --path microtests -q --complog
This run will just include the microtests path and be quiet (-q, not produce output). There will be a logfile created after this run that can be used for comparison.
bench.php --tool cachegrind --complog
This will do a cachegrind benchmark and show results of the valgrind tool cachegrind. This requires valgrind.
List of available options
There are also other options available such as:
- --php PATH
Where PATH is a path to a specific PHP Binary. This is useful when running several php versions on the same machine. Default is just “php”. All files in this path that should be tested need to be names test_XXX.php.
- --memory-limit MEM
Where MEM is the maximum memory usage for the test files to consume (individually). The value is in MB, the default is 128.
- --debug
A switch to set to a slightly more verbose debug mode.
- --help
Print the default help message.
- --q
Quite mode (No output)
- --tool TOOL
Select which TOOL that is used. Default is none and current available options are memusage and cachegrind.
- --log FILE
Produces an additional logfile of the output. Default is benchlog.txt, if FILE is set, it will write to FILE.
- --complog
Writes a serialized log file for comparing runs. Looks like bench_PHPVERSION_TOOL_PID.txt
- --comparison FILES
An individual option that compares the FILES selected. FILES can be “bench_*_memusage_*.txt” or selected files, separated by spaces. They must be benched with the same tool.
Result comparison
The comparison output might be changed, but this is how it looks like currently. The number in the parenthesis is the percentage of the “best” value. If the best runtime is 10s, 10s would become 100%. Since I've only used 5.3.0RC4 for comparing, all results are pretty much the same.
To compare the results it's possible to use the --comparison option. In example, if you would like to compare results from 5.X with memory usage the command could be following: bench.php --comparison bench_5.*_memusage_*
The results will look like this:
--------------- bench.php Comparison -------------- PHP version PHP uname Runtime Private RSS 5.3.0RC4 Linux 2.6.28-11-generic i686 147s (101%) 117.5MB (100%) 5.3.0RC4 Linux 2.6.28-11-generic i686 147.6s (101%) 117.5MB (100%) 5.3.0RC4 Linux 2.6.28-11-generic i686 146.3s (101%) 117.5MB (100%) 5.3.0RC4 Linux 2.6.28-11-generic i686 145.4s (100%) 117.6MB (100%) --------------- bench.php End Comparison -----------
Private RSS is the summary of the processes memory usage and runtime is just the total runtime.
This is how the cachegrind comparison looks like currently:
--------------- bench.php Comparison -------------- PHP version PHP uname Branch Mispred. Instructions Cache misses 5.3.0RC4 Linux 2.6.28-11-generic i686 1.394690e+7 (100%) 7.312482e+9 (100%) 2.419207e+7 (100%) 5.3.0RC4 Linux 2.6.28-11-generic i686 1.394695e+7 (100%) 7.312483e+9 (100%) 2.419216e+7 (100%) --------------- bench.php End Comparison -----------