* 04 Jan 2004 */ include "includes/header.php"; ?> Main text goes here. As Jojo says, I believe it would make most sense to partition the preflight checklist, and am going to suggest this should be done on a per module and per environment basis. The QA team already have divided up the modules, so this will help each of us focus on a particular area. Per environment is a little more difficult because we have not taken on the responsibility of testing our modules in every possible environment (e.g win32/mysql; linux/pgsql; ...).... hmmm... Anyway, if we take the following basic checklist for any module and more or less enumerate it through all the modules we'll have a starting point that will give us some surface testing. I'm only going to offer the generic module checklist for now, and rely on our QA members to add module specific items as (or before) they test their modules. I imagine we will need a similar checklist for the Xaraya Core, and also for themes. Generic module preflight checklist v0.1: ======================================== Date: _____________ QA tester: _____________ Xaraya version: _____________ Module name: _____________ Module version: _____________ Operating system: _____________ Database (and version): _____________ Browser(s): _____________ PHP version: _____________ Start time: _____________ Date of last test: _____________ Location of last test report: ____________________________ 1.0 Installation ^^^^^^^^^^^^^^^^ __ 1.1 Module installs successfully with a fresh Xaraya installation __ 1.2 Module upgrades successfully from version packaged with last Xaraya release (if the module has been upgraded) __ 1.3 Module upgrades successfully from version packaged with a 2-release old release of Xaraya __ 1.4 Module upgrades successfully from version packaged with a 3-release old release of Xaraya **module specific tests here** 2.0 Item creation / modification ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ __ 2.1 Creation of each item type successful. To pass this test, we need to make sure that: * fields have sensible validation * module handles invalid fields gracefully * addition of an invalid item doesn't break database integrity __ 2.2 Item preview for each item type does not cause any errors __ 2.3 Modification of an instance of each item types is successful according to the same criterea as 2.1 __ 2.4 Deletion of each item type is successful **module specific tests here** 3.0 Blocks ^^^^^^^^^^ __ 3.1 Creation of each block type successful __ 3.2 Configuration of each block type successful (try changing each field, and check that the block reacts correspondingly) __ 3.3 Each block reacts gracefully to invalid configuration __ 3.4 Deletion of each block type **module specific tests here** 4.0 Module configuration ^^^^^^^^^^^^^^^^^^^^^^^^ __ 4.1 Module has sensible defaults __ 4.2 Configuration options have clear meanings __ 4.3 Module reacts gracefully to invalid configuration __ 4.4 Unable to come up with a configuration which renders the module unusable **module specific tests here** 5.0 Hooks ^^^^^^^^^ __ TODO 6.0 Security ^^^^^^^^^^^^ __ TODO 7.0 Usability ^^^^^^^^^^^^^ __ 7.1 Pages render correctly in 'supported' browsers (yet to be defined) 8.0 Documentation ^^^^^^^^^^^^^^^^^ __ 8.1 Module has enough documentation on admin-main to allow a site administrator to make use of the module __ 8.2 Provided hooks on admin-main screen matches hooks shown by modules module __ 8.3 Processes described on admin-main screen can be followed with success 8.4 Locations of additional documentation relating to this module since last test (e.g. newsgroup postings, wiki notes, Xaraya announcements, module developers guide...) ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 8.5 Cut and paste valid documentation locations from last test report: ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 9.0 Targeted testing process ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 9.1 Modifications to module since last tested: (check bk comments? ask developer?) _____________________________________________________________________ _____________________________________________________________________ 9.2 Bugs filed since last tested (any status) _____________________________________________________________________ 9.3 New xartests written during testing or since last test _____________________________________________________________________ _____________________________________________________________________ 9.4 New preflight checklist items for this module _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ 10.0 Module removal ^^^^^^^^^^^^^^^^^^ __ 10.1 Module removes successfully through Modules interface __ 10.2 No module variables left in the database __ 10.3 Dynamic data items and instances gone from the database __ 10.4 Security masks removed from database 11.0 Code inspection ^^^^^^^^^^^^^^^^^^^ see http://xaraya.athomeandabout.com/xardocs/qa-reviewer-checklist.html for now... although we have a few things to add to this from recent newsgroup posts 12.0 API tests ^^^^^^^^^^^^^^ using bk tests 13.0 Important records ^^^^^^^^^^^^^^^^^^^^^ 13.1 Repeated defects (jot down bugs that seem to keep reemerging in this module) _____________________________________________________________________ _____________________________________________________________________ 14.0 Testing postmortem ^^^^^^^^^^^^^^^^^^^^^^ 14.1 REQUIRED actions before release ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 14.2 RECOMMENDED actions before release ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 14.3 For the attention of the module developer(s) ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 14.4 Additional comments ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ Finish time: ____________ Total time testing/reviewing: ____________ Random thoughts: * should code inspection and API tests be first? * should this document be split up into testing and review sections? * maybe this document could go into the BK repo so it can grow properly