Tuesday, November 23, 2010

running Sakai3 Ruby tests against remote hosts

The Sakai 3 checkout comes with a nice set of Ruby test coverage. Reviewing tests is a good way to learn the landscape of a given system, which is why I'm poking around with this Ruby and Selenium stuff.

All the Ruby tests can be run via a utility script: tools/runalltests.rb

The tests run on localhost.

If you want to run the existing tests against a remote host you can manually modify the Ruby testing setup script to point at your target host. You'll get some errors, but later for that ;)

Or you can use this script to modify the testing framework, temporarily pointing it at an arbitrary host and then run all the tests. When it's done it returns the framework to the original state.


#!/bin/bash
# runall_elsewhere.sh
#
# edit test.rb to point to a remote server
# and then via tools/runalltests.rb run all tests
# found in:
#   SlingRuby/kerns
#   SlingRuby/tests
#
# caseyd@czwxllc.com

# make sure we have an argument

if [ $# -ne 1 ]
then
 echo "Usage: $0 TESTHOST_URL"
 exit 1
fi

TARGET_SERVER=$1

# some variables
# NAK_HOME needs to be set to your checkout/workarea
NAK_HOME=/Users/casey/dev/nak/nakamura
TEST_HOME=${NAK_HOME}/testscripts/SlingRuby

# the current runalltests script in NAK_HOME/tools expects to be run
#     from the SlingRuby directory

pushd $TEST_HOME

cp ./lib/sling/test.rb ./lib/sling/test.rb.bak

# in-place editing fun
sed -i ''  "s!Sling\.new()!Sling\.new('${TARGET_SERVER}')!" ./lib/sling/test.rb

# stash the result away for later inspection, or not.
cp ./lib/sling/test.rb /tmp/test.rb.EDIT

# run all the tests in TEST_HOME/kerns and TEST_HOME/tests
#  except now the test setup will point at your TARGET_SERVER

$NAK_HOME/tools/runalltests.rb

# repair changes to the testing framework
mv -f ./lib/sling/test.rb.bak ./lib/sling/test.rb

# go back to where ever you were
popd


Remote targeting can be useful for quick A/B comparisons.

 
I suppose this retargeting idea could be built into the current runalltests.rb script.
What I didn't see was a way to easily mongle server retargeting into the current test framework.

You will experience errors.
Tests were not written with this kind of use in mind, so you will get funky race conditions and conflicting states. Also some of the tests are hardcoded to use localhost, so they'll have trouble.

If the various ID generators were slightly tweaked to generate more unique IDs, and the use of localhost removed, these tests would run against arbitrary hosts.

You could use this technique to crowdsource load against a common target server. You and your evil pals would fire up a flock of shells on your local machines and point the test suite at the target server.

Localhost tests:

kern-1105.rb
kern-568.rb
kern-577.rb
kern-797.rb
kern-854.rb
kern-854.rb
kern-857.rb
kern-857.rb
kern-891.rb
basiclti-test.rbmessage-test.rb
message-test.rb
send-email-message.rb

OK time for me to cook a stack of gluten free pancakes for my crew!

No comments: