Differences

This shows you the differences between two versions of the page.

--- qa:runtests:howtorun [2009/07/25 11:04] – created zoe
+++ qa:runtests:howtorun [2017/09/22 13:28] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
-====== Running the standard version of run-tests.php ======
-run-tests.php can invoked in a number of different ways. Developers tend to invoke it using make, like this:
-<code>
-make test TESTS=path/to/test
-</code>
-Or like this from the top level directory in the PHP source tree
-<code>
-TEST_EXECUTABLE_PHP=auto sapi/cli/php run-tests.php path/to/test
-</code>
-People running tests tend to use a script file in their bin directory like this:
-<code>
-#!/bin/sh
-export TEST_PHP_EXECUTABLE=/mnt/workspace/PHP/php53/sapi/cli/php
-$TEST_PHP_EXECUTABLE /mnt/workspace/PHP/php53/run-tests.php $*
-</code>
 ====== Running the new version of run-tests.php ======
 ===== Extract the code =====
-Checkout the code from the PHP svn repository like this:
+Checkout the code from the PHP git repository like this:
 <code>
-svn co http://svn.php.net/repository/php/phpruntests/
+git clone http://git.php.net/repository/phpruntests.git
 </code>
 Under the phpruntests directory you will find all the source code under the directory /src and all of the unit tests under the directory /tests.
 The main program is called run-tests.php. The main program is located in the directory /src.
 ===== Create a test runner =====
 Create a script called 'runtests' in your bin directory (or any directory that is on your PATH) with this content:
@@ Line 36: / Line 17: @@
 <code>
 #!/bin/sh
-export TEST_PHP_EXECUTABLE=/location/of/php_to_test/sapi/cli/php
+export TEST_PHP_EXECUTABLE=/location/of/php_to_test/php
+export TEST_PHP_CGI_EXECUTABLE=/location/of/php_cgi_to_test/php-cgi
 $TEST_PHP_EXECUTABLE /where/you/put/runtests/run-tests.php $*
 </code>
 In the same way as the original script, this version demands that the PHP executable is set before it will run.
+You do not **need** to specify the CGI executable, the run-tests.php code will attempt to guess its location (as the current code does) from the location of the cli.
 ===== Run the code =====
-At the moment the code will run a single test, eg:
+Run a single test, eg:
-   * runtests my_test.phpt
+<code>
+ runtests my_test.phpt
+</code>
-All the tests under a given directory, eg:
+Run all the tests under a given directory, eg:
-   * runtests ext/standard/tests/
+<code>
+  runtests ext/standard/tests/
+</code>
-The output format is different to run-tests.php. If a single test is run a short summary is printed to stdout. If one or more directories of tests are run _all_ of the output is saved till the end of the run. Again, a summary is printed to stdout but details are written to a file.
+The output format is different to run-tests.php. If a single test is run a short summary is printed to stdout. If one or more directories of tests are run the output is printed after each group (directory) of tests has been run. Again, a summary is printed to stdout but details are written to a file.
-See --help for options.
+Run all the tests under path_to_php/ext in parallel on 4 processors
+<code>
+  runtests path_to_php/ext -z 4
+</code>
+Generate XML formatted output to a file:
-====== Running the unit tests for run-tests.php ======
+<code>
+  runtests -o xml -s /tmp/somefile path_to_php/tests
-   * Install PHPUnit
+</code>
-   * Install phing
-   * Install Xdebug
-There is a build.xml file in the source which will run all the tests and produce a code coverage report (but only if you have Xdebug installed). In the phpruntests directory, type:
+Get more verbose results to the screen:
 <code>
-phing test
+  runtests -vvv path_to_php/tests
 </code>
-Tests can be run one at a time using PHPUnit. I do this using the Eclipse command line.
+Note: there are three levels of verbosity, -v, -vv and -vvv. But default all you will see is a '.' for each test.
-Note: Some tests require a PHP executable to run. The executable is found using this:
-$this->php = trim(shell_exec("which php"));
+<code>
+runtests --help
-Which works on Unix but will not work on Windows, it might if cygwin is installed but I haven't tried.
+</code>
-====== QA for run-tests.php ======
-In the QA directory there is a tar file with a standard set of tests. A README in the same directory has instructions on how to use this to check the code. Developers are expected to run this kind of check (in addition to running unit tests) when making significant changes to the code.
+will give you all of the possible command line options.
-A n explanation of the current differences between the old versions of run-tests.php and this one for the set of QA tests is maintained [[/qa/runtests/difference|here]].
-There are a few tests which fail occasionally both with the new and the old version of the code. In the standard set of QA tests these are:
-  * ext/phar/tests/zip/phar_stub.phpt
-  * ext/standard/tests/mail/mail_variation2.phpt
-  * ext/standard/tests/mail/mail_basic2.phpt
-Wider ranging tests indicate that there are some tests for the sockets extension that do the same thing
-  * ext/sockets/tests/ipv4loop.phpt
+======run-tests.php on Windows======
-  * ext/sockets/tests/socket_create_listen.phpt
-  * ext/sockets/tests/socket_set_block-retval.phpt
-  * ext/sockets/tests/socket_set_nonblock-retval.phpt
-We don't have an explanation for this at the moment.
+Running the code should be the same, you will need a runtests.bat file rather than a shell script.

Differences

Page Tools