Buildbot

We have for quite some time now been discussing setting up a build farm to do automated build sanity checks and running the tests on many different operating systems and configurations to detect regressions quicker and help debugging on platforms not everyone have access to.

The idea is to install buildbot to run automated build testing and getting the new run-tests to aggregate the test results and present in a human readable format (cdash could be of help here)?

Making the build farm automatically run the test suite after each commit to php-src and reply to the commit mail if the build broke or if test results aren't identical to the last run would be very helpful to identify which exact commit was broken.

We already have a good chunk of operating system available:

TODO: : These do not have a .php.net DNS yet.

• FreeBSD 6.4 is available on 80.82.121.132
• FreeBSD 7.2 is available on 80.82.121.133
• Debian “lenny” is available on 80.82.121.134
• OpenBSD 4.5 is available on 80.82.121.135
• NetBSD 5.0 is available on 80.82.121.136
• Some PPC boxes, unsure which OS+versions though.

All are just basic installations with no further packages installed.

Developers can get access to these boxes to debug bugs or run tests prior to commits.

Please do not taint the system and run all your testing in your own home directory.

To gain access to these boxes, mail systems@.

After this continuous integration server is completed and finalized, it will be replicated across the other UNIX servers in a slave-master set up.

For now, each slave will be doing pretty much what the master is doing (testing php 5.2, 5.3 and 6.0)

However, the results will only be accessible and stored on the master server.

Here is the rough draft for the implementation details:

Please contact me at iekpo@php.net if you have any questions, comments or recommendations.

The priority as stated by Hannes Magnusson, is to set it up for PHP 5.3 first then PHP 5.2 and 6.0+

So the build bot will only be monitoring commits to the following subversion URLS :

• PHP 5.3: https://svn.php.net/repository/php/php-src/branches/PHP_5_3
• PHP 5.2: https://svn.php.net/repository/php/php-src/branches/PHP_5_2
• PHP 6: https://svn.php.net/repository/php/php-src/trunk

A separate build daemon will be set up for each branch on each UNIX server.

A post commit hook will notify the build system that a commit has occurred (I will need the assistance of systems guys set up this post-commit hook).

A simple web service will be set up at the master to receive these notifications.

The revision number will be entered into a queue and then a queue processing system will take it from there.

The notification will be sent to the master and the other slaves will be notified after that.

I am planning for the master to be the Debian server and the other servers will be the slaves.

The build will occur in very distinct phases/states.

Each stage/state will be fine-tuned or broken down if needed in the future.

We should be able to find out what state a particular job is at any give instant.

Here are the states :

• 2A : The source code will be checked out of SVN.
• 2B : Then the buildconf command will be run.
• 2C : Then the configure script will be run will all the extensions enabled.
• 2D : Then make
• 2E : Then make install

If any of the above steps fail, this particular job is put on HOLD and an email notification will be sent out.

• 2F : Then make test (or any other simpler alternative)

• 2G: The results will then be agregated an analyzed. This will be explained in 4 below

However, we have to take care of the following cases/scenarios to avoid false alarms.

• 1. Removal of phpt scripts (a phpt script that was in the previous revision no longer exists)
• 2. Addition of new phpt scripts (a phpt script that never existed is now introduced)

Whenever a particular build fails (steps 2A-E), human intervention may be needed because the reason could be a missing dependency such as libraries, tools etc.

As for steps 2G, someone will have to fix the source code to make it PASS the test.

If the problem is not in the source code, then the phpt script may have to be adjusted to make it pass the test.

2G1 : Existing phpt files

• If the result changes from PASS to FAIL, a FAILURE notice will be sent out (RED)
• If the result changes from FAIL to PASS, a RESTORED notice will be sent out (GREEN)
• If the result remains at PASS, no notice will be sent out.
• If the result remains at FAIL, no notice will be sent out.

2G2 : Removed phpt files. A REMOVED notice will be sent out (ORANGE)

2G3 : Newly introduced phpt files.

• If the result is FAIL, a FAILURE notice will be sent out (RED)
• If the result is PASS, a PASS notice will be sent out (GREEN)

The results will be stored in a MySQL database that will be archived from time to time.

There will be a web interface to view the results of each test for each server and SVN URL

http://wiki.php.net/internals/buildbot#operating_systems_available

http://www.php.net/svn.php