Table of Contents

Buildbot

We have for quite some time now been discussing setting up a build farm to do automated build sanity checks and running the tests on many different operating systems and configurations to detect regressions quicker and help debugging on platforms not everyone have access to.

The idea is to install buildbot to run automated build testing and getting the new run-tests to aggregate the test results and present in a human readable format (cdash could be of help here)?

Making the build farm automatically run the test suite after each commit to php-src and reply to the commit mail if the build broke or if test results aren't identical to the last run would be very helpful to identify which exact commit was broken.

Operating Systems available

We already have a good chunk of operating system available:

TODO: FIXME: These do not have a .php.net DNS yet.

All are just basic installations with no further packages installed.

Developers access

Developers can get access to these boxes to debug bugs or run tests prior to commits.

Please do not taint the system and run all your testing in your own home directory.

To gain access to these boxes, mail systems@.

Proposed Implementation

After this continuous integration server is completed and finalized, it will be replicated across the other UNIX servers in a slave-master set up.

For now, each slave will be doing pretty much what the master is doing (testing php 5.2, 5.3 and 6.0)

However, the results will only be accessible and stored on the master server.

Here is the rough draft for the implementation details:

Please contact me at iekpo@php.net if you have any questions, comments or recommendations.

0. Target Subversion URLs

The priority as stated by Hannes Magnusson, is to set it up for PHP 5.3 first then PHP 5.2 and 6.0+

So the build bot will only be monitoring commits to the following subversion URLS :

A separate build daemon will be set up for each branch on each UNIX server.

1. Notification of Build System

A post commit hook will notify the build system that a commit has occurred (I will need the assistance of systems guys set up this post-commit hook).

A simple web service will be set up at the master to receive these notifications.

The revision number will be entered into a queue and then a queue processing system will take it from there.

The notification will be sent to the master and the other slaves will be notified after that.

I am planning for the master to be the Debian server and the other servers will be the slaves.

2. Processing items in the queue

The build will occur in very distinct phases/states.

Each stage/state will be fine-tuned or broken down if needed in the future.

We should be able to find out what state a particular job is at any give instant.

Here are the states :

If any of the above steps fail, this particular job is put on HOLD and an email notification will be sent out.

However, we have to take care of the following cases/scenarios to avoid false alarms.

3. Handling Errors and Build-Failures

Whenever a particular build fails (steps 2A-E), human intervention may be needed because the reason could be a missing dependency such as libraries, tools etc.

As for steps 2G, someone will have to fix the source code to make it PASS the test.

If the problem is not in the source code, then the phpt script may have to be adjusted to make it pass the test.

4. Intepretation of Build results

2G1 : Existing phpt files

2G2 : Removed phpt files. A REMOVED notice will be sent out (ORANGE)

2G3 : Newly introduced phpt files.

5. Managing Results Data

The results will be stored in a MySQL database that will be archived from time to time.

There will be a web interface to view the results of each test for each server and SVN URL

References :

http://wiki.php.net/internals/buildbot#operating_systems_available

http://www.php.net/svn.php