This is an old revision of the document!
Running the new version of run-tests.php
Extract the code
Checkout the code from the PHP git repository like this:
git clone http://git.php.net/repository/phpruntests.git
Under the phpruntests directory you will find all the source code under the directory /src and all of the unit tests under the directory /tests. The main program is called run-tests.php. The main program is located in the directory /src.
Create a test runner
Create a script called 'runtests' in your bin directory (or any directory that is on your PATH) with this content:
#!/bin/sh export TEST_PHP_EXECUTABLE=/location/of/php_to_test/php export TEST_PHP_CGI_EXECUTABLE=/location/of/php_cgi_to_test/php-cgi $TEST_PHP_EXECUTABLE /where/you/put/runtests/run-tests.php $*
In the same way as the original script, this version demands that the PHP executable is set before it will run. You do not need to specify the CGI executable, the run-tests.php code will attempt to guess its location (as the current code does) from the location of the cli.
Run the code
At the moment the code will run a single test, eg:
runtests my_test.phpt
All the tests under a given directory, eg:
runtests ext/standard/tests/
The output format is different to run-tests.php. If a single test is run a short summary is printed to stdout. If one or more directories of tests are run the output is printed after each group (directory) of tests has been run. Again, a summary is printed to stdout but details are written to a file.
runtests --help
will give you all of the possible command line options.
QA for run-tests.php
In the QA directory there is a tar file with a standard set of tests. A README in the same directory has instructions on how to use this to check the code. Developers are expected to run this kind of check (in addition to running unit tests) when making significant changes to the code.
A n explanation of the current differences between the old versions of run-tests.php and this one for the set of QA tests is maintained here.
There are a few tests which fail occasionally both with the new and the old version of the code. In the standard set of QA tests these are:
- ext/phar/tests/zip/phar_stub.phpt
- ext/standard/tests/mail/mail_variation2.phpt
- ext/standard/tests/mail/mail_basic2.phpt
Wider ranging tests indicate that there are some tests for the sockets extension that do the same thing
- ext/sockets/tests/ipv4loop.phpt
- ext/sockets/tests/socket_create_listen.phpt
- ext/sockets/tests/socket_set_block-retval.phpt
- ext/sockets/tests/socket_set_nonblock-retval.phpt
We don't have an explanation for this at the moment.
Configure options
The current QA checks use the standard (no options) configure. However as we develop we are extending this to be able to run as many options as possible. This page has a set of configure options that have been tried.
current state:
./configure --prefix=/Users/g2/php/ --with-config-file-path=/Users/g2/php/php53/ --with-pear=/opt/local/ --with-iconv=/opt/local --with-bz2=/opt/local/ --with-zlib=/opt/local/ --with-libxml-dir=/opt/local --with-gettext=/opt/local --with-libexpat-dir=/opt/local --with-freetype-dir=/opt/local --with-openssl=/opt/local/ --with-t1lib=/opt/local/ --with-tidy=/opt/local --with-xsl=/opt/local --with-curl=/opt/local --with-pcre-regex=/opt/local --with-gd --with-jpeg-dir=/opt/local --with-png-dir=/opt/local --with-mhash=/opt/local --with-mcrypt=/opt/local --with-readline=/opt/local --enable-fastcgi --disable-short-tags --enable-pcntl --enable-soap --enable-exif --enable-sockets --enable-sysvsem --enable-sysvshm --with-xml --enable-sysvmsg --enable-calendar --enable-exif --with-xmlrpc --enable-ftp --enable-wdd --enable-soap --enable-bcmath --enable-mbstring --enable-dba --enable-zip --with-mime-magic --enable-gd-native-ttf --enable-xdebug
Further Information on configure
phpruntests on Windows
At frist, make sure to use the correct php-version as described here: http://windows.php.net/download/
A important difference to unix is that you have to execute the script from the directory were your test-cases are stored.
We've created a short bat-file which you can use:
cd c:\QA set TEST_PHP_CGI_EXECUTABLE=c:\PHP\php-5.3.0-nts-Win32-VC6-x86\php-cgi.exe c:\PHP\php-5.3.0-nts-Win32-VC6-x86\php.exe c:\workspace\phpruntests\src\run-tests.php -p c:\PHP\php-5.3.0-nts-Win32-VC6-x86\php.exe %*