and I am not referring to the Kraftwerk song.
One of the hard parts of working with our QA crew is getting a common model of how Sakai operates informing their efforts. So much seems magical / inconsistent / random that they often throw up their hands in the bug reports and trail off into "what did we miss?" and "this happens now and again..."
The practice, with stable code, should be deterministic.
One source of the problem is in basing QA on clones of production databases and using the production integration points. This practice comes from laziness. I can understand it as QA is in reality the second tier of support at Stanford. (there is only 1 support person for our few thousand users. Go cost cutting measures!) One reason QA uses live data is because so often they are pulled in to examine production issues.
As a result the development of formal test cases has been seen as a luxury.
The production of formal test cases would follow an understanding of the Sakai models - and internalizing that is just not going to happen in such an environment. One would need a couple more QA people focusing solely on QA, and not "where is my worksite? oh under that pulldown?" issues.