Split cluster node startup and cluster node configuration

The process of installation of ODL to a cluster node takes
way too long, leading to the state when the first node being
installed finishes booting before the second and third nodes
even get around to starting the boot. This is not a supported
cluster startup scenario and thus leads to numerous spurious
test failures which may appear and disappear depending on the
network load on the Nexus server (e.g. data transfer rate
drops to 100 kb/s, causing the ODL download to take 5 minutes
-> fail).

This change splits the setup script to 2 scripts. The first one
performs the ODL deployment and configuration and the second one
just runs bin/start. This allows all nodes to be started at
roughly the same time (plus/minus a few seconds), eliminating
the unsupported boot scenario.

Change-Id: If8b32309a996451fbc4e1a021fdebfe635e0f916
Signed-off-by: Jozef Behran <jbehran@cisco.com>

diff --git a/jjb/integration/include-raw-integration-configure-clustering.sh b/jjb/integration/include-raw-integration-configure-clustering.sh
index 264f659b3e384a6cc5f828cf28a2ff210df42555..90921512a415fa3079b13c1312fff2afcbd4ea8a 100644 (file)
--- a/jjb/integration/include-raw-integration-configure-clustering.sh
+++ b/jjb/integration/include-raw-integration-configure-clustering.sh
@@ -120,6 +120,11 @@ cat ${MODULESCONF}
 echo "Dump module-shards.conf"
 cat ${MODULESHARDSCONF}
 
+EOF
+
+# Create the startup script to be run on controllers.
+cat > ${WORKSPACE}/startup-script.sh <<EOF
+
 echo "Starting controller..."
 /tmp/${BUNDLEFOLDER}/bin/start
 
@@ -135,4 +140,13 @@ do
     ssh ${!CONTROLLERIP} "bash /tmp/configuration-script.sh ${i}"
 done
 
+# Copy over the startup script to each controller and execute it.
+for i in `seq 1 ${NUM_ODL_SYSTEM}`
+do
+    CONTROLLERIP=ODL_SYSTEM_${i}_IP
+    echo "Starting member-${i} with IP address ${!CONTROLLERIP}"
+    scp ${WORKSPACE}/startup-script.sh ${!CONTROLLERIP}:/tmp/
+    ssh ${!CONTROLLERIP} "bash /tmp/startup-script.sh"
+done
+
 # vim: ts=4 sw=4 sts=4 et ft=sh :