qa:runtests:documentation
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
qa:runtests:documentation [2009/09/01 19:13] – zoe | qa:runtests:documentation [2009/09/01 20:04] – zoe | ||
---|---|---|---|
Line 3: | Line 3: | ||
*[[http:// | *[[http:// | ||
- | A list of the options available in the hold version of run-tests and information about whether that are (or will be) re-implemented. | + | A list of the options available in the old version of run-tests and information about whether that are (or will be) re-implemented. |
====== Introduction ====== | ====== Introduction ====== | ||
- | runtests is a test environment written in PHP. It is used to tests PHP itself. | + | runtests is a test environment written in PHP. It is used to test PHP itself. |
It is not a replacement for unit test frameworks like PHPUnit. | It is not a replacement for unit test frameworks like PHPUnit. | ||
The basic principle of automated tests is very simple. Compare a computed | The basic principle of automated tests is very simple. Compare a computed | ||
result with some pre-calculated "known good", expected value. | result with some pre-calculated "known good", expected value. | ||
- | If both match, the test has passed. If they do not match, the tests failes, and | + | If both match, the test has passed. If they do not match, the tests fails, and |
there is probably a bug in PHP, or some precondition for the test was not met. | there is probably a bug in PHP, or some precondition for the test was not met. | ||
Testing PHP is a far more complex issue than it may seem at first glance, | Testing PHP is a far more complex issue than it may seem at first glance, | ||
because each test may require an individual PHP configuration, | because each test may require an individual PHP configuration, | ||
- | at operting | + | at operating |
services (like databases, LDAP servers or IMAP servers). | services (like databases, LDAP servers or IMAP servers). | ||
Line 54: | Line 54: | ||
We have already mentioned that tests need an individual environment. Not all | We have already mentioned that tests need an individual environment. Not all | ||
- | tests can be run at the command line, for example. If a test need to make | + | tests can be run at the command line, for example. If a test needs to make |
sure that PHP returns correct HTTP header, or processes GET or POST input, | sure that PHP returns correct HTTP header, or processes GET or POST input, | ||
the test probably requires the CGI SAPI to run. | the test probably requires the CGI SAPI to run. | ||
+ | |||
+ | The implementation of runtests that is described in these pages is specific to PHP5.3 and beyond. Many of the implementation details are similar to the previous version of runtests but the overall code structure is very different. | ||
+ | |||
+ | ====== Overall code structure ====== | ||
+ | The most important classes in classes in the code are shown in the figure below and described in the following paragraphs. | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===== Test Run ===== | ||
+ | The main class is rtPhpTestRun. This class is responsible for the overall running of one or more test cases. | ||
+ | |||
+ | rtPhpTestRun instantiates the run configuration class (rtRuntestsConfiguration, | ||
+ | ===== Test Group ===== | ||
+ | A test group (rtPhpTestGroup) is currently defined as all of the tests within a single directory. | ||
+ | |||
+ | There is currently no group configuration class, however, it is possible that one may be required. For example there might be groups of tests that cannot be run at the same time as other groups (tests with REDIRECT? | ||
+ | ===== Test Case ===== | ||
+ | Each test case is executed in it's own process. Many classes are associated with the execution of a single test case. The following subsections give a brief description of the responsibilities of each class. | ||
+ | |||
+ | ===== Run configuration ===== | ||
+ | This group of classes is responsible for setting the configuration for the whole test run. So, for example the name of the name of the PHP executable under test is set in this group as is the name of the PHP CGI exectuable. | ||
+ | |||
+ | ===== Test case configuration ===== | ||
+ | Some settings are specific to each test case and need to be set for a single test. These classes are responsible for settings at the level of individual tests. | ||
+ | |||
+ | ====== Run configuration classes ====== | ||
+ | |||
+ | This sections contains a brief description of each class involved in setting the configuration for the whole test run. | ||
+ | |||
+ | ==== rtRunTestsConfiguration ==== | ||
+ | |||
+ | The main class in this section - all of the other classes are related to setting options in the run configuration object. There should be OS dependent instances for unix, windows etc. | ||
+ | |||
+ | ==== rtCommandLineOptions ==== | ||
+ | Instantiated by rtRuntestsConfiguration. Parses the command line options (argv[]) for runt-tests.php | ||
+ | |||
+ | ==== rtEnvironmentVariables ==== | ||
+ | Get a list of all of the Environment variable and maintains as an array. This is required both establishing configuration settings and to pass to to proc_open() when PHP code is run as part of a test (see rtPhpRunner and testcase/ | ||
+ | |||
+ | ==== rtAddToCommandLine ==== | ||
+ | Somewhat bizarrely (to my mind) argv[] for run-tests can be added to using an Environment variable (TEST_PHP_ARGS). This class is instantiated after rtEnviromentVariables and may (if TEST_PHP_ARGS is used) add other options to an instance of rtCommandLineOptions. This is here for compatibility with the old version of run-tests, if no good reason for it comes to light it should go. | ||
+ | |||
+ | ==== rtIniAsCommandLineArgs ==== | ||
+ | The run-tests code pre-sets a number of ini settings automatically, | ||
+ | |||
+ | php_exectuable | ||
+ | |||
+ | Note that these can be added to using the test case section INI. This class is instantiated from the setting class that builds the php command line, it is also instantiated from the rtTestConfiguration class when a test case contains an INI section. | ||
+ | rtPreCondition list | ||
+ | Maintains a list of all of the pre-conditions that must be satisfied before a test run is attempted. For example - check that a PHP executable has been specified. | ||
+ | |||
+ | ==== rtPrecondition ==== | ||
+ | Parent class for the classes that contain code to check for each pre-condition. See subdirectory ' | ||
+ | |||
+ | ==== rtSetting ==== | ||
+ | Parent class for classes that set run configuration settings. See ' | ||
+ | |||
====== Parallel Execution ====== | ====== Parallel Execution ====== |
qa/runtests/documentation.txt · Last modified: 2017/09/22 13:28 by 127.0.0.1