qa:testfesttalk
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
qa:testfesttalk [2009/03/18 19:37] – dabase | qa:testfesttalk [2009/04/09 20:55] – removed lazy, dabase | ||
---|---|---|---|
Line 13: | Line 13: | ||
* a text editor | * a text editor | ||
* a way to execute the code | * a way to execute the code | ||
+ | |||
+ | Required Packages (debian) | ||
+ | |||
* the make commando available | * the make commando available | ||
- | * bison or flex | + | |
+ | * gcc | ||
+ | | ||
+ | * libxml2-dev | ||
+ | * re2c | ||
+ | * lcov | ||
Line 98: | Line 107: | ||
===== Execute tests ===== | ===== Execute tests ===== | ||
+ | There is a set of things that need to be prepared before you can actually start execute the tests. | ||
+ | |||
+ | ==== Which " | ||
+ | You must use TEST_PHP_EXECUTABLE environment variable to explicitly select the php executable to be used to run the tests. That can either be the CLI or CGI executable. Command "make test" executes " | ||
+ | |||
+ | |||
+ | < | ||
+ | TEST_PHP_EXECUTABLE=sapi/ | ||
+ | </ | ||
+ | and | ||
+ | < | ||
+ | export TEST_PHP_EXECUTABLE | ||
+ | </ | ||
==== All Tests ==== | ==== All Tests ==== | ||
+ | You can re-run the same Tests as executed in '' | ||
- | ==== Single Test ==== | + | < |
- | ===== Write Tests... ===== | + | php run-tests.php |
+ | </ | ||
+ | |||
+ | ==== Single | ||
+ | It is possible to execute a single test or tests residing in one directory. Again run-tests.php helps here. | ||
+ | |||
+ | A single | ||
+ | < | ||
+ | php run-tests.php tests/ | ||
+ | </ | ||
+ | |||
+ | A directory with tests | ||
+ | < | ||
+ | php run-tests.php tests/ | ||
+ | </ | ||
+ | |||
+ | ===== Write Tests ===== | ||
+ | From here on it's easy to add own tests. | ||
+ | |||
+ | < | ||
+ | --TEST-- | ||
+ | Hello world test | ||
+ | --FILE-- | ||
+ | <?php | ||
+ | echo " | ||
+ | ?> | ||
+ | --CLEAN-- | ||
+ | --EXPECT-- | ||
+ | Hello | ||
+ | </ | ||
+ | |||
+ | ===== Write good Tests... ===== | ||
There are certain guidelines you can follow when writing tests. There are several guidelines that you can follow to get working test code that is solid and makes sense. | There are certain guidelines you can follow when writing tests. There are several guidelines that you can follow to get working test code that is solid and makes sense. | ||
+ | Several basics apply in general | ||
+ | |||
+ | * KISS Keep the Test short in general. | ||
+ | * Test one functionality at a time | ||
+ | * Try to make execute the test fast, waiting for the next eclipse to proof the date() function fails will slow down build times | ||
+ | * Document the outcome. People using your tests will not always be experts for that specific tests | ||
+ | * Place Links to resources as they help understand the test. A test is as much about documentation as on prooving fail or working condition of a function | ||
+ | * Extension docs | ||
+ | * Bug reports | ||
+ | * If you are testing on a bug, make sure you got the bottom line of it | ||
==== Mystery Guests | ==== Mystery Guests | ||
Line 122: | Line 186: | ||
* storing data in data files | * storing data in data files | ||
+ | |||
+ | Avoid by | ||
+ | * Store Data in the Test | ||
+ | * Write Setup Code | ||
+ | |||
+ | ==== Resource Optimism ==== | ||
+ | |||
+ | //Test code that makes optimistic assumptions about the ex- | ||
+ | istence (or absence) and state of external resources (such as | ||
+ | particular directories or database tables) can cause non- | ||
+ | deterministic behavior in test outcomes. The situation | ||
+ | where tests run fine at one time and fail miserably the other | ||
+ | time is not a situation you want to find yourself in. Use | ||
+ | Setup External Resource to allocate and/or initialize all | ||
+ | resources that are used.// | ||
+ | |||
+ | Examples: | ||
+ | |||
+ | * Files in /tmp (there is no /tmp in windows) | ||
+ | * extensions installed | ||
+ | * Special variants/ | ||
+ | |||
+ | Avoid By | ||
+ | * Write Setup Code for the resource | ||
+ | * Do not re-use resources through multiple tests | ||
+ | |||
+ | ==== Test Run Wars ==== | ||
+ | |||
+ | //Such wars arise when the tests run fine as long as you are | ||
+ | the only one testing but fail when more programmers run | ||
+ | them. This is most likely caused by resource interference: | ||
+ | some tests in your suite allocate resources such as tempo- | ||
+ | rary files that are also used by others. Apply Make Re- | ||
+ | source Unique (3) to overcome interference. | ||
+ | // | ||
+ | |||
+ | Examples: | ||
+ | * Creating files in a public directory (/tmp/ if choosen right ^^) under a fixed file name | ||
+ | |||
+ | Avoid by: | ||
+ | * Adding Random or user based Value to file name | ||
+ | |||
+ | ==== Eager Test ==== | ||
+ | |||
+ | //When a test method checks several methods of the object to | ||
+ | be tested, it is hard to read and understand, and therefore | ||
+ | more difficult to use as documentation. Moreover, it makes | ||
+ | tests more dependent on each other and harder to maintain. | ||
+ | The solution is simple: separate the test code into test | ||
+ | methods that test only one method using Fowler’s Extract | ||
+ | Method (F:110), using a meaningful name highlighting the | ||
+ | purpose of the test. Note that splitting into smaller methods | ||
+ | can slow down the tests due to increased setup/ | ||
+ | overhead.// | ||
+ | |||
+ | Examples: | ||
+ | * Tests of different behaviour in one test | ||
+ | * More than one method is tested | ||
+ | |||
+ | Avoid | ||
+ | * Split into several smaller tests | ||
+ | |||
+ | |||
+ | |||
==== ... for Coverage ==== | ==== ... for Coverage ==== |
qa/testfesttalk.txt · Last modified: 2017/09/22 13:28 by 127.0.0.1