BGP ingest suites: Make options distinguished 05/29205/16
authorVratko Polak <vrpolak@cisco.com>
Thu, 19 Nov 2015 12:56:00 +0000 (13:56 +0100)
committerGerrit Code Review <gerrit@opendaylight.org>
Tue, 1 Dec 2015 07:56:26 +0000 (07:56 +0000)
All 4 suites are run within one job, but their requirements
for options (especially timeout-related) are different.
Also, scenario with 600 peers is interesting,
and this change would allow only one suite to test that,
while the other would keep testing 2 peers.

Suite order in testplan has changed,
prefix count suites are before change count ones.
Still, single peer suite block is before many peer one.

Timeout computation uses float numbers to avoid rounding errors.
Teardown does not fail if SSH connection to tools system has dropped.

Utils.Execute Controller Karaf Command With Retry On Background
keyword tries to re-establish connection to Karaf console,
as during long test cases (30 minutes) it drops.
Utils.Log Message To Controller Karaf now uses that keyword.

Logical off-by-one error in WaitUtils is fixed.

Unified Karaf log level handling between bgpingest suites.

Change-Id: I99b39b0f78da4ee810c46f71a5389ffd0058456f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
csit/libraries/KarafKeywords.robot
csit/libraries/WaitUtils.robot
csit/suites/bgpcep/bgpingest/manypeers_changecount.robot
csit/suites/bgpcep/bgpingest/manypeers_prefixcount.robot
csit/suites/bgpcep/bgpingest/singlepeer_changecount.robot
csit/suites/bgpcep/bgpingest/singlepeer_prefixcount.robot
csit/testplans/bgpcep-bgp-ingest.txt

index 9b58d5d32f769fc3ef8eca847f87ba89428c2ab8..84e62513f9f98346aed9d570c6d46bfc66ec9d50 100644 (file)
@@ -138,10 +138,20 @@ Execute Controller Karaf Command On Background
     BuiltIn.Run Keyword If    '${status_wait}' != 'PASS'    BuiltIn.Fail    Failed to see prompt after sending the command: ${command}
     [Return]    ${message_wait}
 
+Execute Controller Karaf Command With Retry On Background
+    [Arguments]    ${command}
+    [Documentation]    Attemp to send command to karaf, if fail then open connection and try again.
+    ${status}    ${message}=    BuiltIn.Run Keyword And Ignore Error    Execute Controller Karaf Command On Background    ${command}
+    BuiltIn.Return_From_Keyword_If    '${status}' == 'PASS'    ${message}
+    # TODO: Verify this does not leak connections indices.
+    Open Controller Karaf Console On Background
+    ${message}=    Execute Controller Karaf Command On Background    ${command}
+    [Return]    ${message}
+
 Log Message To Controller Karaf
     [Arguments]    ${message}
     [Documentation]    Send a message into the controller's karaf log file. Do not change current SSH connection.
-    ${reply}=    Execute Controller Karaf Command On Background    log:log "ROBOT MESSAGE: ${message}"
+    ${reply}=    Execute Controller Karaf Command With Retry On Background    log:log "ROBOT MESSAGE: ${message}"
     [Return]    ${reply}
 
 Log Test Suite Start To Controller Karaf
index b205e98ba708bdf1359548be8bdb7d62a01c9c57..8d5252b7b05ec03ba093aecc595bf2ed635700c2 100644 (file)
@@ -198,7 +198,7 @@ Wait_For_Getter_And_Safe_Stateful_Validator_Consecutive_Success
     ${result} =    BuiltIn.Set_Variable    No result yet.
     ${state} =    BuiltIn.Set_Variable    ${initial_state}
     # The loop for failures.
-    : FOR    ${try}    IN RANGE    1    ${maximum_tries}
+    : FOR    ${try}    IN RANGE    1    ${maximum_tries}+1    # If maximum_tries is 3, for will go through 1, 2, and 3.
     \    ${state}    ${status}    ${result} =    Getter_And_Safe_Stateful_Validator_Have_To_Succeed_Consecutively_By_Deadline    date_deadline=${date_deadline}    period_in_seconds=${period_in_seconds}
     \    ...    count=${count}    getter=${getter}    safe_validator=${safe_validator}    initial_state=${state}
     \    # Have we passed?
@@ -221,7 +221,7 @@ Wait_For_Getter_Error_Or_Safe_Stateful_Validator_Consecutive_Success
     ${result} =    BuiltIn.Set_Variable    No result yet.
     ${state} =    BuiltIn.Set_Variable    ${initial_state}
     # The loop for failures.
-    : FOR    ${try}    IN RANGE    1    ${maximum_tries}
+    : FOR    ${try}    IN RANGE    1    ${maximum_tries}+1    # If maximum_tries is 3, for will go through 1, 2, and 3.
     \    ${state}    ${status}    ${result} =    Getter_And_Safe_Stateful_Validator_Have_To_Succeed_Consecutively_By_Deadline    date_deadline=${date_deadline}    period_in_seconds=${period_in_seconds}
     \    ...    count=${count}    getter=${getter}    safe_validator=${safe_validator}    initial_state=${state}
     \    # Have we passed?
index 6eaec8b2969f44de7bc5d68df76d5b3ae04333e7..711b0b1d6ddcd783e4cc3cd7554d4a61a877c998 100644 (file)
@@ -59,22 +59,32 @@ Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
 Resource          ${CURDIR}/../../../libraries/SSHKeywords.robot
 
 *** Variables ***
+${BGP_TOOL_LOG_LEVEL}    info
 ${BGP_VARIABLES_FOLDER}    ${CURDIR}/../../../variables/bgpuser/
-${HOLDTIME}       180
-${HOLDTIME_CHANGE_COUNT}    ${HOLDTIME}
+${CHECK_PERIOD}    1
+${CHECK_PERIOD_CHANGE_COUNT}    ${CHECK_PERIOD}
+${CHECK_PERIOD_CHANGE_COUNT_MANY}    ${CHECK_PERIOD_CHANGE_COUNT}
 ${COUNT}          1000000
 ${COUNT_CHANGE_COUNT}    ${COUNT}
+${COUNT_CHANGE_COUNT_MANY}    ${COUNT_CHANGE_COUNT}
 ${FIRST_PEER_IP}    127.0.0.1
+${HOLDTIME}       180
+${HOLDTIME_CHANGE_COUNT}    ${HOLDTIME}
+${HOLDTIME_CHANGE_COUNT_MANY}    ${HOLDTIME_CHANGE_COUNT}
+${KARAF_LOG_LEVEL}    INFO
+${KARAF_BGPCEP_LOG_LEVEL}    ${KARAF_LOG_LEVEL}
+${KARAF_PROTOCOL_LOG_LEVEL}    ${KARAF_BGPCEP_LOG_LEVEL}
 ${MULTIPLICITY}    2    # May be increased after Bug 4488 is fixed.
 ${MULTIPLICITY_CHANGE_COUNT}    ${MULTIPLICITY}
-${CHECK_PERIOD}    1
-${CHECK_PERIOD_CHANGE_COUNT}    ${CHECK_PERIOD}
-${REPETITIONS}    5
-# TODO: Figure out reliable value. Even without Bug 4488 hitting, something
-# (OperationLimiter or garbage collection) is causing failures when low value is used.
+${MULTIPLICITY_CHANGE_COUNT_MANY}    ${MULTIPLICITY_CHANGE_COUNT}
+${REPETITIONS}    1    # Should be increased depending on multiplicity.
 ${REPETITIONS_CHANGE_COUNT}    ${REPETITIONS}
+${REPETITIONS_CHANGE_COUNT_MANY}    ${REPETITIONS_CHANGE_COUNT}
+${TEST_DURATION_MULTIPLIER}    1
+${TEST_DURATION_MULTIPLIER_CHANGE_COUNT}    ${TEST_DURATION_MULTIPLIER}
+${TEST_DURATION_MULTIPLIER_CHANGE_COUNT_MANY}    ${TEST_DURATION_MULTIPLIER_CHANGE_COUNT}
 # TODO: Option names can be better.
-${last_change_count}    -1
+${last_change_count_many}    -1
 
 *** Test Cases ***
 Check_For_Empty_Ipv4_Topology_Before_Talking
@@ -85,10 +95,10 @@ Check_For_Empty_Ipv4_Topology_Before_Talking
 
 Reconfigure_ODL_To_Accept_Connections
     [Documentation]    Configure BGP peer modules with initiate-connection set to false.
-    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY}+1
+    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY_CHANGE_COUNT_MANY}+1
     \    ${peer_name} =    BuiltIn.Set_Variable    example-bgp-peer-${index}
     \    ${peer_ip} =    BuiltIn.Evaluate    str(ipaddr.IPAddress('${FIRST_PEER_IP}') + ${index} - 1)    modules=ipaddr
-    \    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': '${peer_name}', 'IP': '${peer_ip}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT}', 'INITIATE': 'false'}
+    \    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': '${peer_name}', 'IP': '${peer_ip}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT_MANY}', 'INITIATE': 'false'}
     \    ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
     # FIXME: Add testcase to change bgpcep and protocol log levels, when a Keyword that does it without messing with current connection is ready.
 
@@ -100,24 +110,24 @@ Reconfigure_Data_Change_Counter
     [Documentation]    Configure data change counter to count transactions in example-ipv4-topology instead of example-linkstate-topology.
     ChangeCounter.Reconfigure_Topology_Name    example-ipv4-topology
 
-Set_Logging_Levels
-    [Documentation]    For Bug 4488 examination, more verbose logging may be needed.
-    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${BGPCEP_LOG_LEVEL}    protocol_level=${PROTOCOL_LOG_LEVEL}
+Change_Karaf_Logging_Levels
+    [Documentation]    We may want to set more verbose logging here after configuration is done.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_BGPCEP_LOG_LEVEL}    protocol_level=${KARAF_PROTOCOL_LOG_LEVEL}
 
 Start_Talking_BGP_Manager
     [Documentation]    Start Python manager to connect speakers to ODL.
     Store_Change_Count
     # Myport value is needed for checking whether connection at precise port was established.
-    BGPSpeaker.Start_BGP_Manager    --amount=${COUNT_CHANGE_COUNT} --multiplicity=${MULTIPLICITY} --myip=${FIRST_PEER_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT}
+    BGPSpeaker.Start_BGP_Manager    --amount=${COUNT_CHANGE_COUNT_MANY} --multiplicity=${MULTIPLICITY_CHANGE_COUNT_MANY} --myip=${FIRST_PEER_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT}
 
 Wait_For_Stable_Talking_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable. This is done by checking stability of the change counter.
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_MANY}    repetitions=${REPETITIONS_CHANGE_COUNT_MANY}    count_to_overcome=${last_change_count_many}
 
 Check_Talking_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT_MANY}
 
 Kill_Talking_BGP_Speakers
     [Documentation]    Abort the Python speakers. Also, attempt to stop failing fast.
@@ -131,16 +141,16 @@ Kill_Talking_BGP_Speakers
 Wait_For_Stable_Ipv4_Topology_After_Talking
     [Documentation]    Wait until example-ipv4-topology becomes stable again.
     [Tags]    critical
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_MANY}    repetitions=${REPETITIONS_CHANGE_COUNT_MANY}    count_to_overcome=${last_change_count_many}
 
 Check_For_Empty_Ipv4_Topology_After_Talking
     [Documentation]    Example-ipv4-topology should be empty now.
     [Tags]    critical
     PrefixCounting.Check_Ipv4_Topology_Is_Empty
 
-Restore_Logging_Levels
-    [Documentation]    Set logging on bgpcep and protocol to default values.
-    KarafKeywords.Set_Bgpcep_Log_Levels
+Restore_Karaf_Logging_Levels
+    [Documentation]    Set logging on bgpcep and protocol to the global value.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_LOG_LEVEL}    protocol_level=${KARAF_LOG_LEVEL}
 
 Restore_Data_Change_Counter_Configuration
     [Documentation]    Configure data change counter back to count transactions affecting example-linkstate-topology.
@@ -151,7 +161,7 @@ Delete_Bgp_Peer_Configuration
     [Documentation]    Revert the BGP configuration to the original state: without any configured peers.
     [Setup]    SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
     # TODO: Is it useful to extract peer naming logic to separate Keyword?
-    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY}+1
+    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY_CHANGE_COUNT_MANY}+1
     \    ${template_as_string}=    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer-${index}'}
     \    ConfigViaRestconf.Delete_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLESFOLDER}${/}bgp_peer    ${template_as_string}
 
@@ -176,15 +186,17 @@ Setup_Everything
     SSHLibrary.Put_File    ${CURDIR}/../../../../tools/fastbgp/manage_play.py
     SSHKeywords.Assure_Library_Ipaddr    target_dir=.
     # Calculate the timeout value based on how many routes are going to be pushed
-    ${period} =    DateTime.Convert_Time    ${CHECK_PERIOD_CHANGE_COUNT}    result_format=number
-    ${timeout} =    BuiltIn.Evaluate    ${COUNT_CHANGE_COUNT} * 3 / 10000 + ${period} * (${REPETITIONS_CHANGE_COUNT} + 1)
+    ${period} =    DateTime.Convert_Time    ${CHECK_PERIOD_CHANGE_COUNT_MANY}    result_format=number
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_CHANGE_COUNT_MANY} * (${COUNT_CHANGE_COUNT_MANY} * 3.0 / 10000 + ${period} * (${REPETITIONS_CHANGE_COUNT_MANY} + 1))
     Builtin.Set_Suite_Variable    ${bgp_filling_timeout}    ${timeout}
-    ${timeout} =    BuiltIn.Evaluate    ${COUNT_CHANGE_COUNT} * 2 / 10000 + ${period} * (${REPETITIONS_CHANGE_COUNT} + 1)
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_CHANGE_COUNT_MANY} * (${COUNT_CHANGE_COUNT_MANY} * 2.0 / 10000 + ${period} * (${REPETITIONS_CHANGE_COUNT_MANY} + 1))
     Builtin.Set_Suite_Variable    ${bgp_emptying_timeout}    ${timeout}
+    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${KARAF_LOG_LEVEL}
 
 Teardown_Everything
     [Documentation]    Make sure Python tool was killed and tear down imported Resources.
-    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
+    # Environment issue may have dropped the SSH connection, but we do not want Teardown to fail.
+    BuiltIn.Run_Keyword_And_Ignore_Error    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
     ConfigViaRestconf.Teardown_Config_Via_Restconf
     RequestsLibrary.Delete_All_Sessions
     SSHLibrary.Close_All_Connections
@@ -192,4 +204,4 @@ Teardown_Everything
 Store_Change_Count
     [Documentation]    Get the count of changes from BGP change counter. Ignore error or store the value.
     ${status}    ${count} =    BuiltIn.Run_Keyword_And_Ignore_Error    ChangeCounter.Get_Change_Count
-    BuiltIn.Run_Keyword_If    '${status}' == 'PASS'    BuiltIn.Set_Suite_Variable    ${last_change_count}    ${count}
+    BuiltIn.Run_Keyword_If    '${status}' == 'PASS'    BuiltIn.Set_Suite_Variable    ${last_change_count_many}    ${count}
index 28e74d8f8ae835204d5f305fa4b3d4d8c4f0fa5a..eaec63d0725d1198ebac595e620f0e9d36f08d71 100644 (file)
@@ -55,21 +55,31 @@ Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
 Resource          ${CURDIR}/../../../libraries/SSHKeywords.robot
 
 *** Variables ***
+${BGP_TOOL_LOG_LEVEL}    info
 ${BGP_VARIABLES_FOLDER}    ${CURDIR}/../../../variables/bgpuser/
-${HOLDTIME}       180
-${HOLDTIME_PREFIX_COUNT}    ${HOLDTIME}
+${CHECK_PERIOD}    1    # ${MULTIPLICITY*2} recommended for this suite, but keeping the common default.
+${CHECK_PERIOD_PREFIX_COUNT}    ${CHECK_PERIOD}
+${CHECK_PERIOD_PREFIX_COUNT_MANY}    ${CHECK_PERIOD_PREFIX_COUNT}
 ${COUNT}          1000000
 ${COUNT_PREFIX_COUNT}    ${COUNT}
+${COUNT_PREFIX_COUNT_MANY}    ${COUNT_PREFIX_COUNT}
 ${FIRST_PEER_IP}    127.0.0.1
+${HOLDTIME}       180
+${HOLDTIME_PREFIX_COUNT}    ${HOLDTIME}
+${HOLDTIME_PREFIX_COUNT_MANY}    ${HOLDTIME_PREFIX_COUNT}
+${KARAF_LOG_LEVEL}    INFO
+${KARAF_BGPCEP_LOG_LEVEL}    ${KARAF_LOG_LEVEL}
+${KARAF_PROTOCOL_LOG_LEVEL}    ${KARAF_BGPCEP_LOG_LEVEL}
 ${MULTIPLICITY}    2    # May be increased after Bug 4488 is fixed.
 ${MULTIPLICITY_PREFIX_COUNT}    ${MULTIPLICITY}
-${CHECK_PERIOD}    4    # ${MULTIPLICITY*2}
-${CHECK_PERIOD_PREFIX_COUNT}    ${CHECK_PERIOD}
+${MULTIPLICITY_PREFIX_COUNT_MANY}    ${MULTIPLICITY_PREFIX_COUNT}
 ${REPETITIONS}    1
 ${REPETITIONS_PREFIX_COUNT}    ${REPETITIONS}
-${BGPCEP_LOG_LEVEL}    INFO
+${REPETITIONS_PREFIX_COUNT_MANY}    ${REPETITIONS_PREFIX_COUNT}
+${TEST_DURATION_MULTIPLIER}    1
+${TEST_DURATION_MULTIPLIER_PREFIX_COUNT}    ${TEST_DURATION_MULTIPLIER}
+${TEST_DURATION_MULTIPLIER_PREFIX_COUNT_MANY}    ${TEST_DURATION_MULTIPLIER_PREFIX_COUNT}
 # TODO: Option names can be better.
-${last_prefix_count}    -1
 
 *** Test Cases ***
 Check_For_Empty_Ipv4_Topology_Before_Talking
@@ -80,30 +90,30 @@ Check_For_Empty_Ipv4_Topology_Before_Talking
 
 Reconfigure_ODL_To_Accept_Connections
     [Documentation]    Configure BGP peer modules with initiate-connection set to false.
-    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY}+1
+    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY_PREFIX_COUNT_MANY}+1
     \    ${peer_name} =    BuiltIn.Set_Variable    example-bgp-peer-${index}
     \    ${peer_ip} =    BuiltIn.Evaluate    str(ipaddr.IPAddress('${FIRST_PEER_IP}') + ${index} - 1)    modules=ipaddr
-    \    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': '${peer_name}', 'IP': '${peer_ip}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT}', 'INITIATE': 'false'}
+    \    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': '${peer_name}', 'IP': '${peer_ip}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT_MANY}', 'INITIATE': 'false'}
     \    ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
     # FIXME: Add testcase to change bgpcep and protocol log levels, when a Keyword that does it without messing with current connection is ready.
 
-Set_Logging_Levels
-    [Documentation]    For Bug 4488 examination, more verbose logging may be needed.
-    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${BGPCEP_LOG_LEVEL}    protocol_level=${PROTOCOL_LOG_LEVEL}
+Change_Karaf_Logging_Levels
+    [Documentation]    We may want to set more verbose logging here after configuration is done.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_BGPCEP_LOG_LEVEL}    protocol_level=${KARAF_PROTOCOL_LOG_LEVEL}
 
 Start_Talking_BGP_Manager
     [Documentation]    Start Python manager to connect speakers to ODL.
     # Myport value is needed for checking whether connection at precise port was established.
-    BGPSpeaker.Start_BGP_Manager    --amount=${COUNT_PREFIX_COUNT} --multiplicity=${MULTIPLICITY} --myip=${FIRST_PEER_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT}
+    BGPSpeaker.Start_BGP_Manager    --amount=${COUNT_PREFIX_COUNT_MANY} --multiplicity=${MULTIPLICITY_PREFIX_COUNT_MANY} --myip=${FIRST_PEER_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT}
 
 Wait_For_Stable_Talking_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable. This is done by checking stability of prefix count.
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=0
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_MANY}    repetitions=${REPETITIONS_PREFIX_COUNT_MANY}    excluded_count=0
 
 Check_Talking_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT_MANY}
 
 Kill_Talking_BGP_Speakers
     [Documentation]    Abort the Python speakers. Also, attempt to stop failing fast.
@@ -119,22 +129,22 @@ Wait_For_Stable_Ipv4_Topology_After_Talking
     # TODO: Is is possible to have failed at Check_Talking_Ipv4_Topology_Count and still have initial period of constant count?
     # FIXME: If yes, do count here to get the initial value and use it (if nonzero).
     # TODO: If yes, decide whether access to the FailFast state should have keyword or just variable name.
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=${COUNT_PREFIX_COUNT}
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_MANY}    repetitions=${REPETITIONS_PREFIX_COUNT_MANY}    excluded_count=${COUNT_PREFIX_COUNT_MANY}
 
 Check_For_Empty_Ipv4_Topology_After_Talking
     [Documentation]    Example-ipv4-topology should be empty now.
     [Tags]    critical
     PrefixCounting.Check_Ipv4_Topology_Is_Empty
 
-Restore_Logging_Levels
-    [Documentation]    Set logging on bgpcep and protocol to default values.
-    KarafKeywords.Set_Bgpcep_Log_Levels
+Restore_Karaf_Logging_Levels
+    [Documentation]    Set logging on bgpcep and protocol to the global value.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_LOG_LEVEL}    protocol_level=${KARAF_LOG_LEVEL}
 
 Delete_Bgp_Peer_Configuration
     [Documentation]    Revert the BGP configuration to the original state: without any configured peers.
     [Setup]    SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
     # TODO: Is it useful to extract peer naming logic to separate Keyword?
-    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY}+1
+    : FOR    ${index}    IN RANGE    1    ${MULTIPLICITY_PREFIX_COUNT_MANY}+1
     \    ${template_as_string}=    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer-${index}'}
     \    ConfigViaRestconf.Delete_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLESFOLDER}${/}bgp_peer    ${template_as_string}
 
@@ -157,15 +167,17 @@ Setup_Everything
     SSHLibrary.Put_File    ${CURDIR}/../../../../tools/fastbgp/play.py
     SSHLibrary.Put_File    ${CURDIR}/../../../../tools/fastbgp/manage_play.py
     # Calculate the timeout value based on how many routes are going to be pushed.
-    ${period} =    DateTime.Convert_Time    ${CHECK_PERIOD_PREFIX_COUNT}    result_format=number
-    ${timeout} =    BuiltIn.Evaluate    ${COUNT_PREFIX_COUNT} * 3 / 10000 + ${period} * (${REPETITIONS_PREFIX_COUNT} + 1)
+    ${period} =    DateTime.Convert_Time    ${CHECK_PERIOD_PREFIX_COUNT_MANY}    result_format=number
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_PREFIX_COUNT_MANY} * (${COUNT_PREFIX_COUNT_MANY} * 3.0 / 10000 + ${period} * (${REPETITIONS_PREFIX_COUNT_MANY} + 1))
     Builtin.Set_Suite_Variable    ${bgp_filling_timeout}    ${timeout}
-    ${timeout} =    BuiltIn.Evaluate    ${COUNT_PREFIX_COUNT} * 2 / 10000 + ${period} * (${REPETITIONS_PREFIX_COUNT} + 1)
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_PREFIX_COUNT_MANY} * (${COUNT_PREFIX_COUNT_MANY} * 2.0 / 10000 + ${period} * (${REPETITIONS_PREFIX_COUNT_MANY} + 1))
     Builtin.Set_Suite_Variable    ${bgp_emptying_timeout}    ${timeout}
+    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${KARAF_LOG_LEVEL}
 
 Teardown_Everything
     [Documentation]    Make sure Python tool was killed and tear down imported Resources.
-    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
+    # Environment issue may have dropped the SSH connection, but we do not want Teardown to fail.
+    BuiltIn.Run_Keyword_And_Ignore_Error    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
     ConfigViaRestconf.Teardown_Config_Via_Restconf
     RequestsLibrary.Delete_All_Sessions
     SSHLibrary.Close_All_Connections
index 69fe4effea208746592bd4a1377c303c25d625fe..43303921af477190829722a48340e727816db467 100644 (file)
@@ -45,25 +45,33 @@ Resource          ${CURDIR}/../../../libraries/PrefixCounting.robot
 Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
 
 *** Variables ***
+${BGP_TOOL_LOG_LEVEL}    info
 ${BGP_VARIABLES_FOLDER}    ${CURDIR}/../../../variables/bgpuser/
-${HOLDTIME}       180
-${HOLDTIME_CHANGE_COUNT}    ${HOLDTIME}
-${COUNT}          1000000
-${COUNT_CHANGE_COUNT}    ${COUNT}
 ${CHECK_PERIOD}    1
 ${CHECK_PERIOD_CHANGE_COUNT}    ${CHECK_PERIOD}
-${REPETITIONS_CHANGE_COUNT}    1
+${CHECK_PERIOD_CHANGE_COUNT_SINGLE}    ${CHECK_PERIOD_CHANGE_COUNT}
+${COUNT}          1000000
+${COUNT_CHANGE_COUNT}    ${COUNT}
+${COUNT_CHANGE_COUNT_SINGLE}    ${COUNT_CHANGE_COUNT}
+${HOLDTIME}       180
+${HOLDTIME_CHANGE_COUNT}    ${HOLDTIME}
+${HOLDTIME_CHANGE_COUNT_SINGLE}    ${HOLDTIME_CHANGE_COUNT}
 ${INSERT}         1
-${WITHDRAW}       0
+${KARAF_LOG_LEVEL}    INFO
+${KARAF_BGPCEP_LOG_LEVEL}    ${KARAF_LOG_LEVEL}
+${KARAF_PROTOCOL_LOG_LEVEL}    ${KARAF_BGPCEP_LOG_LEVEL}
 ${PREFILL}        0
-${UPDATE}         single
-${BGP_TOOL_LOG_LEVEL}    info
-${CONTROLLER_LOG_LEVEL}    INFO
-${CONTROLLER_BGP_LOG_LEVEL}    DEFAULT
+${REPETITIONS}    1
+${REPETITIONS_CHANGE_COUNT}    ${REPETITIONS}
+${REPETITIONS_CHANGE_COUNT_SINGLE}    ${REPETITIONS_CHANGE_COUNT}
 ${RESULTS_FILE_NAME}    bgp.csv
 ${TEST_DURATION_MULTIPLIER}    1
+${TEST_DURATION_MULTIPLIER_CHANGE_COUNT}    ${TEST_DURATION_MULTIPLIER}
+${TEST_DURATION_MULTIPLIER_CHANGE_COUNT_SINGLE}    ${TEST_DURATION_MULTIPLIER_CHANGE_COUNT}
+${UPDATE}         single
+${WITHDRAW}       0
 # TODO: Option names can be better.
-${last_change_count}    -1
+${last_change_count_single}    -1
 
 *** Test Cases ***
 Check_For_Empty_Ipv4_Topology_Before_Talking
@@ -74,7 +82,7 @@ Check_For_Empty_Ipv4_Topology_Before_Talking
 
 Reconfigure_ODL_To_Accept_Connection
     [Documentation]    Configure BGP peer module with initiate-connection set to false.
-    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'false'}
+    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT_SINGLE}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'false'}
     ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
 
 Wait_For_Data_Change_Counter_Ready
@@ -85,20 +93,24 @@ Reconfigure_Data_Change_Counter
     [Documentation]    Configure data change counter to count transactions in example-ipv4-topology instead of example-linkstate-topology.
     ChangeCounter.Reconfigure_Topology_Name    example-ipv4-topology
 
+Change_Karaf_Logging_Levels
+    [Documentation]    We may want to set more verbose logging here after configuration is done.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_BGPCEP_LOG_LEVEL}    protocol_level=${KARAF_PROTOCOL_LOG_LEVEL}
+
 Start_Talking_BGP_Speaker
     [Documentation]    Start Python speaker to connect to ODL.
     Store_Change_Count
     # Myport value is needed for checking whether connection at precise port was established.
-    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_CHANGE_COUNT} --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
+    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_CHANGE_COUNT_SINGLE} --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
 
 Wait_For_Stable_Talking_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable. This is done by checking the change counter.
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_SINGLE}    repetitions=${REPETITIONS_CHANGE_COUNT_SINGLE}    count_to_overcome=${last_change_count_single}
 
 Check_Talking_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT_SINGLE}
 
 Kill_Talking_BGP_Speaker
     [Documentation]    Abort the Python speaker. Also, attempt to stop failing fast.
@@ -121,7 +133,7 @@ Store_Results_For_Talking_BGP_Speaker
 Wait_For_Stable_Ipv4_Topology_After_Talking
     [Documentation]    Wait until example-ipv4-topology becomes stable again.
     [Tags]    critical
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_emptying_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_emptying_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_SINGLE}    repetitions=${REPETITIONS_CHANGE_COUNT_SINGLE}    count_to_overcome=${last_change_count_single}
 
 Check_For_Empty_Ipv4_Topology_After_Talking
     [Documentation]    Example-ipv4-topology should be empty now.
@@ -130,22 +142,22 @@ Check_For_Empty_Ipv4_Topology_After_Talking
 
 Start_Listening_BGP_Speaker
     [Documentation]    Start Python speaker in listening mode.
-    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_CHANGE_COUNT} --listen --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
+    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_CHANGE_COUNT_SINGLE} --listen --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
 
 Reconfigure_ODL_To_Initiate_Connection
     [Documentation]    Replace BGP peer config module, now with initiate-connection set to true.
     Store_Change_Count
-    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'true'}
+    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_CHANGE_COUNT_SINGLE}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'true'}
     ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
 
 Wait_For_Stable_Listening_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable.
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_SINGLE}    repetitions=${REPETITIONS_CHANGE_COUNT_SINGLE}    count_to_overcome=${last_change_count_single}
 
 Check_Listening_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_CHANGE_COUNT_SINGLE}
 
 Kill_Listening_BGP_Speaker
     [Documentation]    Abort the Python speaker. Also, attempt to stop failing fast.
@@ -168,13 +180,17 @@ Store_Results_For_Listening_BGP_Speaker
 Wait_For_Stable_Ipv4_Topology_After_Listening
     [Documentation]    Wait until example-ipv4-topology becomes stable again.
     [Tags]    critical
-    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_emptying_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT}    repetitions=${REPETITIONS_CHANGE_COUNT}    count_to_overcome=${last_change_count}
+    ChangeCounter.Wait_For_Change_Count_To_Become_Stable    timeout=${bgp_emptying_timeout}    period=${CHECK_PERIOD_CHANGE_COUNT_SINGLE}    repetitions=${REPETITIONS_CHANGE_COUNT_SINGLE}    count_to_overcome=${last_change_count_single}
 
 Check_For_Empty_Ipv4_Topology_After_Listening
     [Documentation]    Example-ipv4-topology should be empty now.
     [Tags]    critical
     PrefixCounting.Check_Ipv4_Topology_Is_Empty
 
+Restore_Karaf_Logging_Levels
+    [Documentation]    Set logging on bgpcep and protocol to the global value.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_LOG_LEVEL}    protocol_level=${KARAF_LOG_LEVEL}
+
 Restore_Data_Change_Counter_Configuration
     [Documentation]    Configure data change counter back to count transactions affecting example-linkstate-topology.
     [Setup]    SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
@@ -203,17 +219,16 @@ Setup_Everything
     # Both TODOs would probably need to update every suite relying on current Variables.
     SSHLibrary.Put_File    ${CURDIR}/../../../../tools/fastbgp/play.py
     # Calculate the timeout value based on how many routes are going to be pushed
-    # TODO: Replace 20 with some formula from period and repetitions.
-    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER} * ${COUNT_CHANGE_COUNT} * 3.0 / 10000 + 20
+    # TODO: Unify formulas with other suites in this directory.
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_CHANGE_COUNT_SINGLE} * (${COUNT_CHANGE_COUNT_SINGLE} * 3.0 / 10000 + 20)
     Builtin.Set_Suite_Variable    ${bgp_filling_timeout}    ${timeout}
     Builtin.Set_Suite_Variable    ${bgp_emptying_timeout}    ${bgp_filling_timeout*3.0/4}
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_LOG_LEVEL}
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_BGP_LOG_LEVEL} org.opendaylight.bgpcep
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_BGP_LOG_LEVEL} org.opendaylight.protocol
+    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${KARAF_LOG_LEVEL}
 
 Teardown_Everything
     [Documentation]    Make sure Python tool was killed and tear down imported Resources.
-    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
+    # Environment issue may have dropped the SSH connection, but we do not want Teardown to fail.
+    BuiltIn.Run_Keyword_And_Ignore_Error    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
     ConfigViaRestconf.Teardown_Config_Via_Restconf
     RequestsLibrary.Delete_All_Sessions
     SSHLibrary.Close_All_Connections
@@ -221,7 +236,7 @@ Teardown_Everything
 Store_Change_Count
     [Documentation]    Get the count of changes from BGP change counter. Ignore error or store the value.
     ${status}    ${count} =    BuiltIn.Run_Keyword_And_Ignore_Error    ChangeCounter.Get_Change_Count
-    BuiltIn.Run_Keyword_If    '${status}' == 'PASS'    BuiltIn.Set_Suite_Variable    ${last_change_count}    ${count}
+    BuiltIn.Run_Keyword_If    '${status}' == 'PASS'    BuiltIn.Set_Suite_Variable    ${last_change_count_single}    ${count}
 
 Store_File_To_Workspace
     [Arguments]    ${src_file_name}    ${dst_file_name}
index a730439df3872022a33f7991d722ad2919d4fa40..5f2b161dd21e068944ff27c89166192a1693d4b5 100644 (file)
@@ -37,25 +37,32 @@ Resource          ${CURDIR}/../../../libraries/PrefixCounting.robot
 Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
 
 *** Variables ***
+${BGP_TOOL_LOG_LEVEL}    info
 ${BGP_VARIABLES_FOLDER}    ${CURDIR}/../../../variables/bgpuser/
-${HOLDTIME}       180
-${HOLDTIME_PREFIX_COUNT}    ${HOLDTIME}
-${COUNT}          1000000
-${COUNT_PREFIX_COUNT}    ${COUNT}
 ${CHECK_PERIOD}    1
 ${CHECK_PERIOD_PREFIX_COUNT}    ${CHECK_PERIOD}
-${REPETITIONS_PREFIX_COUNT}    1
+${CHECK_PERIOD_PREFIX_COUNT_SINGLE}    ${CHECK_PERIOD_PREFIX_COUNT}
+${COUNT}          1000000
+${COUNT_PREFIX_COUNT}    ${COUNT}
+${COUNT_PREFIX_COUNT_SINGLE}    ${COUNT_PREFIX_COUNT}
+${HOLDTIME}       180
+${HOLDTIME_PREFIX_COUNT}    ${HOLDTIME}
+${HOLDTIME_PREFIX_COUNT_SINGLE}    ${HOLDTIME_PREFIX_COUNT}
 ${INSERT}         1
-${WITHDRAW}       0
+${KARAF_LOG_LEVEL}    INFO
+${KARAF_BGPCEP_LOG_LEVEL}    ${KARAF_LOG_LEVEL}
+${KARAF_PROTOCOL_LOG_LEVEL}    ${KARAF_BGPCEP_LOG_LEVEL}
 ${PREFILL}        0
-${UPDATE}         single
-${BGP_TOOL_LOG_LEVEL}    info
-${CONTROLLER_LOG_LEVEL}    INFO
-${CONTROLLER_BGP_LOG_LEVEL}    DEFAULT
+${REPETITIONS}    1
+${REPETITIONS_PREFIX_COUNT}    ${REPETITIONS}
+${REPETITIONS_PREFIX_COUNT_SINGLE}    ${REPETITIONS_PREFIX_COUNT}
 ${RESULTS_FILE_NAME}    bgp.csv
 ${TEST_DURATION_MULTIPLIER}    1
+${TEST_DURATION_MULTIPLIER_PREFIX_COUNT}    ${TEST_DURATION_MULTIPLIER}
+${TEST_DURATION_MULTIPLIER_PREFIX_COUNT_SINGLE}    ${TEST_DURATION_MULTIPLIER_PREFIX_COUNT}
+${UPDATE}         single
+${WITHDRAW}       0
 # TODO: Option names can be better.
-${last_prefix_count}    -1
 
 *** Test Cases ***
 Check_For_Empty_Ipv4_Topology_Before_Talking
@@ -66,22 +73,26 @@ Check_For_Empty_Ipv4_Topology_Before_Talking
 
 Reconfigure_ODL_To_Accept_Connection
     [Documentation]    Configure BGP peer module with initiate-connection set to false.
-    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'false'}
+    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT_SINGLE}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'false'}
     ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
 
+Change_Karaf_Logging_Levels
+    [Documentation]    We may want to set more verbose logging here after configuration is done.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_BGPCEP_LOG_LEVEL}    protocol_level=${KARAF_PROTOCOL_LOG_LEVEL}
+
 Start_Talking_BGP_Speaker
     [Documentation]    Start Python speaker to connect to ODL.
     # Myport value is needed for checking whether connection at precise port was established.
-    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_PREFIX_COUNT} --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
+    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_PREFIX_COUNT_SINGLE} --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --peerport=${ODL_BGP_PORT} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
 
 Wait_For_Stable_Talking_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable. This is done by checking stability of prefix count.
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=0
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_SINGLE}    repetitions=${REPETITIONS_PREFIX_COUNT_SINGLE}    excluded_count=0
 
 Check_Talking_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT_SINGLE}
 
 Kill_Talking_BGP_Speaker
     [Documentation]    Abort the Python speaker. Also, attempt to stop failing fast.
@@ -106,7 +117,7 @@ Wait_For_Stable_Ipv4_Topology_After_Talking
     # TODO: Is is possible to have failed at Check_Talking_Ipv4_Topology_Count and still have initial period of constant count?
     # FIXME: If yes, do count here to get the initial value and use it (if nonzero).
     # TODO: If yes, decide whether access to the FailFast state should have keyword or just variable name.
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=${COUNT_PREFIX_COUNT}
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_SINGLE}    repetitions=${REPETITIONS_PREFIX_COUNT_SINGLE}    excluded_count=${COUNT_PREFIX_COUNT_SINGLE}
 
 Check_For_Empty_Ipv4_Topology_After_Talking
     [Documentation]    Example-ipv4-topology should be empty now.
@@ -115,21 +126,21 @@ Check_For_Empty_Ipv4_Topology_After_Talking
 
 Start_Listening_BGP_Speaker
     [Documentation]    Start Python speaker in listening mode.
-    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_PREFIX_COUNT} --listen --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
+    BGPSpeaker.Start_BGP_Speaker    --amount ${COUNT_PREFIX_COUNT_SINGLE} --listen --myip=${TOOLS_SYSTEM_IP} --myport=${BGP_TOOL_PORT} --peerip=${ODL_SYSTEM_IP} --insert=${INSERT} --withdraw=${WITHDRAW} --prefill ${PREFILL} --update ${UPDATE} --${BGP_TOOL_LOG_LEVEL} --results ${RESULTS_FILE_NAME}
 
 Reconfigure_ODL_To_Initiate_Connection
     [Documentation]    Replace BGP peer config module, now with initiate-connection set to true.
-    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'true'}
+    ${template_as_string} =    BuiltIn.Set_Variable    {'NAME': 'example-bgp-peer', 'IP': '${TOOLS_SYSTEM_IP}', 'HOLDTIME': '${HOLDTIME_PREFIX_COUNT_SINGLE}', 'PEER_PORT': '${BGP_TOOL_PORT}', 'INITIATE': 'true'}
     ConfigViaRestconf.Put_Xml_Template_Folder_Config_Via_Restconf    ${BGP_VARIABLES_FOLDER}${/}bgp_peer    ${template_as_string}
 
 Wait_For_Stable_Listening_Ipv4_Topology
     [Documentation]    Wait until example-ipv4-topology becomes stable.
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=0
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_SINGLE}    repetitions=${REPETITIONS_PREFIX_COUNT_SINGLE}    excluded_count=0
 
 Check_Listening_Ipv4_Topology_Count
     [Documentation]    Count the routes in example-ipv4-topology and fail if the count is not correct.
     [Tags]    critical
-    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT}
+    PrefixCounting.Check_Ipv4_Topology_Count    ${COUNT_PREFIX_COUNT_SINGLE}
 
 Kill_Listening_BGP_Speaker
     [Documentation]    Abort the Python speaker. Also, attempt to stop failing fast.
@@ -151,13 +162,17 @@ Store_Results_For_Listening_BGP_Speaker
 Wait_For_Stable_Ipv4_Topology_After_Listening
     [Documentation]    Wait until example-ipv4-topology becomes stable again.
     [Tags]    critical
-    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT}    repetitions=${REPETITIONS_PREFIX_COUNT}    excluded_count=${COUNT_PREFIX_COUNT}
+    PrefixCounting.Wait_For_Ipv4_Topology_Prefixes_To_Become_Stable    timeout=${bgp_filling_timeout}    period=${CHECK_PERIOD_PREFIX_COUNT_SINGLE}    repetitions=${REPETITIONS_PREFIX_COUNT_SINGLE}    excluded_count=${COUNT_PREFIX_COUNT_SINGLE}
 
 Check_For_Empty_Ipv4_Topology_After_Listening
     [Documentation]    Example-ipv4-topology should be empty now.
     [Tags]    critical
     PrefixCounting.Check_Ipv4_Topology_Is_Empty
 
+Restore_Karaf_Logging_Levels
+    [Documentation]    Set logging on bgpcep and protocol to the global value.
+    KarafKeywords.Set_Bgpcep_Log_Levels    bgpcep_level=${KARAF_LOG_LEVEL}    protocol_level=${KARAF_LOG_LEVEL}
+
 Delete_Bgp_Peer_Configuration
     [Documentation]    Revert the BGP configuration to the original state: without any configured peers.
     [Setup]    SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
@@ -181,16 +196,15 @@ Setup_Everything
     SSHLibrary.Put_File    ${CURDIR}/../../../../tools/fastbgp/play.py
     # Calculate the timeout value based on how many routes are going to be pushed
     # TODO: Replace 20 with some formula from period and repetitions.
-    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER} * ${COUNT_PREFIX_COUNT} * 3.0 / 10000 + 20
+    ${timeout} =    BuiltIn.Evaluate    ${TEST_DURATION_MULTIPLIER_PREFIX_COUNT_SINGLE} * (${COUNT_PREFIX_COUNT_SINGLE} * 3.0 / 10000 + 20)
     Builtin.Set_Suite_Variable    ${bgp_filling_timeout}    ${timeout}
     Builtin.Set_Suite_Variable    ${bgp_emptying_timeout}    ${bgp_filling_timeout*3.0/4}
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_LOG_LEVEL}
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_BGP_LOG_LEVEL} org.opendaylight.bgpcep
-    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${CONTROLLER_BGP_LOG_LEVEL} org.opendaylight.protocol
+    KarafKeywords.Execute_Controller_Karaf_Command_On_Background    log:set ${KARAF_LOG_LEVEL}
 
 Teardown_Everything
     [Documentation]    Make sure Python tool was killed and tear down imported Resources.
-    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
+    # Environment issue may have dropped the SSH connection, but we do not want Teardown to fail.
+    BuiltIn.Run_Keyword_And_Ignore_Error    KillPythonTool.Search_And_Kill_Remote_Python    'play\.py'
     ConfigViaRestconf.Teardown_Config_Via_Restconf
     RequestsLibrary.Delete_All_Sessions
     SSHLibrary.Close_All_Connections
index 4ba19e548363016a369dc01ade24601f9b56fe35..6de34c9c51922528cdcbfee80e2d080728a217cf 100644 (file)
@@ -6,7 +6,7 @@
 
 # Place the suites in run order:
 integration/test/csit/suites/netconf/ready/netconfready.robot
-integration/test/csit/suites/bgpcep/bgpingest/singlepeer_changecount.robot
 integration/test/csit/suites/bgpcep/bgpingest/singlepeer_prefixcount.robot
-integration/test/csit/suites/bgpcep/bgpingest/manypeers_changecount.robot
+integration/test/csit/suites/bgpcep/bgpingest/singlepeer_changecount.robot
 integration/test/csit/suites/bgpcep/bgpingest/manypeers_prefixcount.robot
+integration/test/csit/suites/bgpcep/bgpingest/manypeers_changecount.robot