How does the QA team work?

Originally: http://pear.php.net/pepr/pepr-proposal-show.php?id=60

In short, QA core group access rights:

1. Max 7 core members that get the core-QA-karma.
2. the core QA team will get more permission to fix issues and make release of packages if reported problems are not fixed in a specific time frame
3. the core QA team assumes lead over orphaned packages
4. the core QA team gets the rights to delete releases of packages if it determines that the release has severe issues
5. the QA team is allowed to add QA related notes to package homepages
6. QA team needs to approve an first stable release for a given major version number (done by votes)
7. the entire QA team will identify QA problems, will help writing QA related documents, will help people resolve their QA problems and such things
8. More focus is needed on writing QA related things in the manual and make then more visible
9. Make a little QA corner at pear.php.net so QA is more visible for people and package devs are more aware of the QA team and how it works.
10. keep track of what QA does regarding packages that QA doesn't have permission to edit through a wiki of some sort

If there are more applicants than seats, there should be an open election, where everyone may cast as many votes as there are seats (only 1 vote may be cast per candidate). Of course, if there are less people than seats, all may be accepted and the remaining seats can be filled by QA group vote. Once the core team is formed all further team members can be voted in as per the rules stated below.

There will be a QA core team with a limited number (no more than 7, with the pear group this should give a sufficient amount of people with enough karma that can react to bad releases) people with extended “core-QA-karma”. In order for the QA core team to react quickly, the core QA team should be spread out across all time zones as good as possible. This rather small number should empower the QA team sufficiently to handle issues but doesn't trivialize access rights.

All other people interested are free to join the QA team. However potential new members for the QA team (core and non core) must be voted in by a 2/3 majority of the existing members of the QA team (core and non core). This is to prevent that the addition of new members is done even though there is a considerable amount of opposition, as this could lead to internal quarrels which could result in reducing the QA teams ability to efficiently address QA issues. Removing a member from either group requires a 2/3 majority as well.

For casual helpers the QA mailing list will of course remain open.

All members of the QA team (core and non core) are eligible to participate in QA team votes.

Any member that leaves the QA team loses all his special karma which he may have received due to his membership. If any member of the core QA team leaves and as a result a time zone gets under staffed, QA should actively try to find a replacement.

This issue needs more discussion since Klaus raised a good point in his post http://news.php.net/article.php?group=php.pear.dev&article=26756 but the problem is that it isn't wise to give many users so much karma ¹⁾).

When voting for a new member voters must be sure that the person has been helping with bug fixes in the past or has the potentials to help with that.

All votes (except membership related votes; see above) require a simple 50% majority. Any member of the QA team may call for a vote at any time. All voting is done on the QA mailinglist.

The core team may overrule any decision (this includes votes on memberships) of the QA team and may decide to hold their own internal vote to make the ultimate decision on how to proceed. Furthermore, the core QA team may also hold votes without even consulting the QA team. However this should be only done in rare cases. All members of the core QA team may make decision on their own if a decision is time critical (like pulling a broken release). However the core QA team should be aware that making too many decisions on their own may lead to alienating the other members and therefore if possible it should always be attempted to at least consult other members of the team.

If any QA related issues are found in a package that QA has not been granted permission to change without consulting the maintainer,the QA team will file a bug report. If the issue remains unresolved for 1 month (2+1+1 weeks; see below), QA may fix it themselves. If the issue isn't fixed with in two weeks QA will email the maintainers about the issue (if the bug system won't have auto bug report notices every X week in the future) and if the problem isn't fixed within one week from that, QA will send yet another email to the maintainers and if no answer nor fix to the problem is provided within one week, all members of the core QA team will get the permission to modify any file needed to fix that QA issue as well as make a new release.

If any major QA issues are found in any package and the QA team doesn't have permission to change that package without contacting the lead first then the QA team will file a bug report. Any major issues can stay open for 2 weeks (5+5+4 days; see below), QA may fix it themselves. If the issue isn't fixed within 5 days QA will email the maintainers about the issue (if the bug system won't have auto bug report notices every X week in the future) and if the problem isn't fixed with in 5 days, from that QA will send yet another email to the maintainers and if no answer nor fix to the problem is provided with in 4 days, all members of the core QA team will have permission to modify any file needed to fix that QA issue as well as make a new release.

The same time frames apply when the QA team want to make a release because of unreleased fixes/improvements which are only available via CVS, or when the QA team wants to determine if a package has been orphaned (using the time frames specified for non major issues).

The QA team can decide what are major issues after discussion on the QA Mailing-list.

The QA team will for this reason keep track of those QA issues in their pear web subsection. This would only be for having overview over which package maintainers need to be emailed for reminders and such things.

The QA team can use the above stated rules to determine if a package is orphaned. The core QA team gets lead rights on all orphaned packages until the original maintainer returns or a new maintainer is found.

The core QA team also has the permission to delete any release that might have any issues that have been found after the release, to reduce the effects of the problem. This right must be exercised with caution.

Package maintainers can also grant QA the rights to make a releases for their packages.

The QA team needs to approve the first stable release of any major version. For this purpose maintainers upload the release to PEAR web and contact the QA team with a link to the release, then QA will decide if the release is ready or not. The core QA team will at this point approve the release through PEAR web and release it for the maintainer.

Notes: The QA team must always inform the current lead maintainers of any changes that are done to a given package using the currently configured email addresses on pear web.

The core QA team may delegate tasks as much as possible to other QA team members. However no access rights will be assigned just for the sake of delegating a task (so members of the QA team may package a new release, but only members of the core QA team will have the necessary rights to upload the release).