qa:phptlessonslearned

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
qa:phptlessonslearned [2008/04/18 15:57] – lsmithqa:phptlessonslearned [2017/09/22 13:28] (current) – external edit 127.0.0.1
Line 24: Line 24:
   * When testing arguments that relate to the length or offset of something, pass negative numbers, as well as numbers less and greater than the length.   * When testing arguments that relate to the length or offset of something, pass negative numbers, as well as numbers less and greater than the length.
   * If a function takes a resource as an argument, test different types of resources (file system, directory, database connection etc.) in case there is no type check.   * If a function takes a resource as an argument, test different types of resources (file system, directory, database connection etc.) in case there is no type check.
 +
  
  
Line 56: Line 57:
   * Watch out for error messages in the expected output:   * Watch out for error messages in the expected output:
     * Change the file location to %s     * Change the file location to %s
-  * Line numbers should be changed to %d, though this can mean bugs don’t get noticed (for example all errors saying “on line 0”). If leaving the line number in, ensure it’s the right one! +    * Line numbers should be changed to %d, though this can mean bugs don’t get noticed (for example all errors saying “on line 0”). If leaving the line number in, ensure it’s the right one! 
-  * Some errors are platform specific, if they’re being generated using ''strerror(errno)'' in the source code then change the error to %s.+    * Some errors are platform specific, if they’re being generated using ''strerror(errno)'' in the source code then change the error to %s. 
  
 ===== Specific Extensions: ===== ===== Specific Extensions: =====
Line 70: Line 72:
 **Anything requiring an external server (e.g. MySQL, IMAP): ** **Anything requiring an external server (e.g. MySQL, IMAP): **
   * Use an include file where users can set their own username, password and server name as variables, then ensure that you use the variables throughout the tests.   * Use an include file where users can set their own username, password and server name as variables, then ensure that you use the variables throughout the tests.
 +  * These variables should be overwritable by enviroment variable so that each tester could specify his own setup e.g.:
 +
 +  $host = !empty($_ENV['TEST_HOST']) ? $_ENV['TEST_HOST'] : 'localhost';
 +
   * Whenever any of these appear in the expected output, remember to replace them with %s (if using EXPECTF).   * Whenever any of these appear in the expected output, remember to replace them with %s (if using EXPECTF).
   * If something doesn’t appear to be working as expected, it could be that the server you’re using doesn’t support this functionality (to save bogus bugs).   * If something doesn’t appear to be working as expected, it could be that the server you’re using doesn’t support this functionality (to save bogus bugs).
   * If possible, test tests on more than one server before committing.   * If possible, test tests on more than one server before committing.
 +
  
 ===== Test Case Generator: ===== ===== Test Case Generator: =====
Line 79: Line 86:
  
 **Use on command Line:** **Use on command Line:**
 +
 ''php.exe generate_phpt.php –h'' ''php.exe generate_phpt.php –h''
 +
 will give a good overview of how to use the test case generator, there are things that I set every time I use it so have that in a batch file. will give a good overview of how to use the test case generator, there are things that I set every time I use it so have that in a batch file.
  
 **gen.bat file:** **gen.bat file:**
 +
   cd <place to store generated tests>   cd <place to store generated tests>
   php.exe <location of generate_phpt.php script> -s <location of source code> -f %*   php.exe <location of generate_phpt.php script> -s <location of source code> -f %*
  
 I then type the function name and the type of test I want, e.g. I then type the function name and the type of test I want, e.g.
 +
 ''  gen.bat array_key_exists –b '' ''  gen.bat array_key_exists –b ''
 +
 This will generate a basic test for array_key_exists. All I need to do now is fill out the EXPECTF section of the file! This will generate a basic test for array_key_exists. All I need to do now is fill out the EXPECTF section of the file!
  
 ** Variation tests: ** ** Variation tests: **
 +
 ''  gen.bat array_key_exists –v'' ''  gen.bat array_key_exists –v''
 +
 will generate variation tests. There will be one for each of the functions argument and an empty test. The numbered variations will iterate over an array of different data types and pass them as an argument, these tests are very useful! Again all that needs doing with these is changing the title (in the -- TEST -- section), add a comment explaining what the test is doing and filling out the EXPECTF section. will generate variation tests. There will be one for each of the functions argument and an empty test. The numbered variations will iterate over an array of different data types and pass them as an argument, these tests are very useful! Again all that needs doing with these is changing the title (in the -- TEST -- section), add a comment explaining what the test is doing and filling out the EXPECTF section.
 With the blank variation test is a template for any other variation tests the function needs. With the blank variation test is a template for any other variation tests the function needs.
 +
 +===== Submitting Tests =====
 +
 +If you concentrate on a certain extension you will probably write several tests for this extension. When it's done you can submit those tests to the  [[https://phptestfest.org/|TestFest website]], but be sure, that every test is working fine. You can do so by running:
 +
 +  make test TESTS=/path/to/php5/ext/yourextension/tests
 +
 +
  
qa/phptlessonslearned.1208534243.txt.gz · Last modified: 2017/09/22 13:28 (external edit)

Page Tools