qa:runtests

This is an old revision of the document!


Requirements

The initial requirements for the project are describedhere. Other requirements will be added as list items here with detailed descriptions (where necessary) added to the requirements documentation.

  • Be able to skip whole directories with one check
  • Provide the same functionality as REDEIRECTTEST, not necessarily using the current design
  • Provide an API so thatthe code can be used n=by other applications (from Greg Beaver).

More documentation and source code can be found here here.

ToDo & who

  • Add a check for 5.3 only (Georg - done, 6th may)
  • Implement valgrind (high priority from Nuno)
  • Implement CGI tests (Zoe)
    • GET (done, 27 April)
    • POST (done 28th April)
    • POST_RAW (done 3rd May)
    • GZIP_POST (done 28th April)
    • DEFLATE_POST (done 28th April)
    • EXPECTHEADERS (done May)
    • COOKIE (done 3rd May)
  • Implement FILE_EXTERNAL (Georg, done)
  • Prototype parallel running (Georg - first prototype committed)
  • Replace the test status array with a status object (Zoe - done, 8th June)

TODOs 30th June

  • Run sequential groups within in Georg's schedulers (G - Done)
  • Write whole object to disk (check timing differences, if none leave as it, otherwise Win/Lin versions) (G - Done)
  • Refactor output writer - and task test group and task receiver. (G - Done)
  • Focus more on QA
    • Assemble a set of standard tests (Z - Done, see QA/QATESTS.tgz + README)
    • Checkin results for running them with old version and current version of our code (Z, done. See LinuxQAJuly18)
    • Write phing task to compare them (S - done)
    • Always run Unit tests before checkin (All)
    • Add new unit test whenever we find a problem (All)
    • Always check output for notices and warnings (All)
  • Always do clean checkout, never use cvs up (All)
  • Use rtException everywhere, get rid of other exception classes (Z - not done)

TODOs 22nd July

  • Investigate CI server (S)
  • Tidy wiki and put together a set of doc pages (Z) (Note: user doc to go on wiki first, final destination undecided)
  • Use rtException everywhere, get rid of other exception classes (Z)
  • Research REDIRECTTEST (S)
  • Run existing code on Windows (S)
  • Fix up any remaining test fails on S and G systems (Z)
  • Start writing documentation for parallel code on wiki (G)
  • Run some more extensive tests/experiments with parallel code (G, Z&S to provide introductions)
  • Implement valgrind (Z)

Record of development decisions

  • Use the PEAR coding standard (22/04/2009 - see php-qa list for concerns)
  • Deviation from the standard: Do not use _ prefix for private members. (Reason: too much re-work to existing code)
  • We prefix classnames with rt (Reason: avoid name conflicts with built-in PHP classes)
  • We do not follow the original PEAR class naming scheme (Reason: we use autoload and name prefixes, so we can keep classnames shorter than DIR_DIR_DIR_CLASS)
  • We will not include both File and Class doc blocks. (Reason: Only ever one class per file)

This page configuration (the address is a bit screwy, but I won't change it as I've posted a link to it) has a list of currently implemented options, and whether we will re-implement or not. Yes means we will - blank means not decided, have asked for input on QA list.

Additional Options

  • Run tests in parallel?
  • XML or CSV or TXT (to replace HTML)
  • Option specify arguments to valgrind

Things that will not be re-implemented

This section is here to record differences between the new and old implementation of run-tests where they affect the excecution of existing tests or the manner in which run-tests.php is executed.

GZIP_POST, DEFLATE_POST

The way these are currently implemented the test has to look like this:

--POST--
Some posted content

--GZIP_POST--
1
--FILE--
etc

This isn't really necessary. In the new implementation the --GZIP_POST-- section will just contain the content to be posted and gzip'd. Same for DEFLATE_POST. This affects two tests:

  • ext/soap/tests/server019.phpt
  • ext/soap/tests/server020.phpt

Subpages

qa/runtests.1248292071.txt.gz · Last modified: 2017/09/22 13:28 (external edit)