PEAR QA Continuous Integration and Unit Tests
There are a number of tools qa use to maintain quality.
PHPCS and General QA tools
http://test.pear.php.net/ hosts continuous integration tools
The helper scripts live at https://github.com/pear/phpuc/
Jenkins
Jenkins is a continuous integration platform. Currently, all packages in both PEAR and PEAR2 were scanned for the presence of a 'tests' folder, and corresponding jenkins jobs were created.
Access
Jenkins is configured to authenticate against the github organisations of pear / pear2. You simply need to be associated with one of the organisations to modify builds.
Adding a new job/build
- Create job
- Enter package name
- Enter link to github
- Enter the read-only git url into source management
- Select 'build on push from github' as a trigger
- Add either manual build steps or invoke an ant build.xml target. Manual steps should include executing
- phpunit --coverage-html build/coverage/ --coverage-clover build/logs/clover.xml --log-junit build/logs/junit.xml tests/
- php ~/pyrus.phar package
- Add in Post-build Actions for 'Publish junit test result report'
How do I make sure my tests work?
In general, we encourage the use of an appropriate phpunit.xml - we target PHPUnit 3.6 at this time.
Your package directory layout should allow you to successfully run your test suite as below (assuming svn):
$ cd Foo_Bar/trunk $ phpunit -c phpunit.xml tests/
My directory layout doesn't work
A common pattern with older packages is to have a directory structure which is unlike the installed package layout. IE: Foo_Bar/trunk/Bar.php
. Most authors make use of the baseinstalldir
attribute in the package.xml, or only run tests after installation
To fix this:
$ svn mkdir Foo $ svn mv Bar.php Foo/ # Edit package.xml, change baseinstalldir to / $ vim package.xml # Rebuild package using PEAR_PackageFileManager_CLI $ pfm # Run tests to confirm no fatal errors, using one of $ pear run-tests -r tests/ $ phpunit tests/ # Commit $ svn commit -m "Updated directory layout"
Pyrus won't build my package 1.0
See migration
My unit tests need a (database/server/setup/configuration)
Two strategies exist for this: - Making use of PHPUnit's mocking capabilities or utilising something like HTTP_Request2's Mock Adapter - Checking your environment and skipping the test if it cannot run
There is also a test mysql instance available.
Hi Daniel:
On Wed, Nov 16, 2011 at 08:34:04AM +1030, Daniel O'Connor wrote:
> http://test.pear.php.net:8080/job/DB_DataObject/4/console
>
> This and a few others obviously need a database to be tested properly.
> Certainly that machine has mysql on it; or sqlite is probably available -
> what would be needed to load up an appropriate schema/tear it down after
> the tests?
I agree and mentioned this on pear-qa a while ago. Take a look at what
I committed to DB tests yesterday. It takes advantage of the
MYSQL_TEST_* environment variables, which are used for PHP's tests. We
should create a test user and database and set those environment
variables in the cron job, or whatever.
A similar setup should be made for PostgreSQL in PEAR and PHP's unit
tests. Right now, PHP's tests don't use such environment variables.
SQLite3 tests should use “:memory:” as the database.
Thanks,
--Dan
Build tools
In general, ant is the preferred way of managing a build after a certain level of complexity. See http://jenkins-php.org/ for more detail.
Noteably, we prefer phpdoc/docblox over phpdox.
Some packages use phing
Dependencies
There are two available techniques for managing dependencies without installing system wide libraries.
The first is relatively straight forward - simply pulling the library from source and pushing it into a lib directory.
$ git clone git://path/to/package.git lib/package $ git clone git://path/to/package2.git lib/package2 $ phpunit --include-path lib/package:lib/package2 ...
This can be either configured in the jenkins build or (preferred) in the relevant build.xml and phpunit.xml files.
The second is to utilise composer.phar in combination with a relevant phpunit.xml
Todo
- Put more packages into jenkins
- Investigate a notification task to package maintainers?
- Investigate allowing bootstrapped build.xml files back in (Pyrus)
- Upgrade PHP on test.pear.php.net
- Install xdebug for proper code coverage
How to test a new PEAR release
Before releasing a new version of PEAR, we need to make sure it runs on as many systems as possible. It's pretty easy:
- Install the new PEAR
- Make sure you have the XML_RPC package
- Checkout pear-core from the repository to get the tests
- Run the tests
pear upgrade -f PEAR pear upgrade XML_RPC git clone git://github.com/pear/pear-core.git cd pear-core/tests && pear run-tests -r
Tell pear-qa@lists.php.net if you got failures, and if, which tests did not pass. We will come back to you in that case.
Common pitfalls
- Make sure that PHP is compiled with tokenizer support (--enable-tokenizer). On most systems this is default, but on Gentoo Linux for instance, you need to enable the tokenizer USE flag.
- For some tests the executable php-cgi is needed. In some Linux distributions this is contained in a separate package, but on for instance Gentoo Linux this has to be enabled using the cgi USE flag.
- If running on linux, do not run the tests as root