Fixed confusing ordering of comparison keyword arguments
[integration/test.git] / csit / suites / netconf / MDSAL / northbound.robot
1 *** Settings ***
2 Documentation     Metconf MDSAL Northbound test suite.
3 ...
4 ...               Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.
5 ...
6 ...               This program and the accompanying materials are made available under the
7 ...               terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ...               and is available at http://www.eclipse.org/legal/epl-v10.html
9 ...
10 ...
11 ...               The request produced by test cases "Get Config Running", "Get Config Running
12 ...               To Confirm No_Edit Before Commit", "Get Config Running To Confirm Delete
13 ...               After Commit" and "Get Config Candidate To Confirm Discard" all use the same
14 ...               message id ("empty") for their requests. This is possible because the
15 ...               requests produced by this suite are strictly serialized. The RFC 6241 does
16 ...               not state that these IDs are unique, it only requires that each ID used is
17 ...               "XML attribute normalized" if the client wants it to be returned unmodified.
18 ...               The RFC specifically says that "the content of this attribute is not
19 ...               interpreted in any way, it only is stored to be returned with the reply to
20 ...               the request. The reuse of the "empty" string for the 4 test cases was chosen
21 ...               for simplicity.
22 ...
23 ...               TODO: Change the 4 testcases to use unique message IDs.
24 Suite Setup       Setup_Everything
25 Suite Teardown    Teardown_Everything
26 Test Setup        SetupUtils.Setup_Test_With_Logging_And_Fast_Failing
27 Test Teardown     FailFast.Start_Failing_Fast_If_This_Failed
28 Library           RequestsLibrary
29 Library           SSHLibrary
30 Resource          ${CURDIR}/../../../libraries/FailFast.robot
31 Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
32 Resource          ${CURDIR}/../../../libraries/SSHKeywords.robot
33 Resource          ${CURDIR}/../../../libraries/Utils.robot
34 Variables         ${CURDIR}/../../../variables/Variables.py
35
36 *** Variables ***
37 ${datadir}        ${CURDIR}/../../../variables/netconf/MDSAL
38 ${dataext}        msg
39 ${ssh_netconf_pid}    -1
40
41 *** Test Cases ***
42 Connect_To_ODL_Netconf
43     [Documentation]    Connect to ODL Netconf and fail if that is not possible.
44     Create_ODL_Netconf_Connection
45     Open_ODL_Netconf_Connection
46
47 Get_Config_Running
48     [Documentation]    Make sure the configuration has only the default elements in it.
49     Check_Test_Objects_Not_Present_In_Config    get-config
50
51 Missing_Message_ID_Attribute
52     [Documentation]    Check that messages with missing "message-ID" attribute are rejected with the correct error (RFC 6241, section 4.1).
53     Perform_Test    missing-attr
54
55 Additional_Attributes_In_Message
56     [Documentation]    Check that additional attributes in messages are returned properly (RFC 6241, sections 4.1 and 4.2).
57     ${reply}=    Load_And_Send_Message    additional-attr
58     BuiltIn.Should_Contain    ${reply}    xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"
59     BuiltIn.Should_Contain    ${reply}    message-id="1"
60     BuiltIn.Should_Contain    ${reply}    attribute="something"
61     BuiltIn.Should_Contain    ${reply}    additional="otherthing"
62     BuiltIn.Should_Contain    ${reply}    xmlns:prefix="http://www.example.com/my-schema-example.html"
63
64 Edit_Config_Modules_Merge
65     [Documentation]    Request a "merge" operation adding an element in candidate configuration and check the reply.
66     Perform_Test    config-modules-merge-1
67
68 Get_Config_Running_To_Confirm_No_Edit_Before_Commit
69     [Documentation]    Make sure the running configuration is still unchanged as the change was not commited yet.
70     Check_Test_Objects_Not_Present_In_Config    get-config-no-edit-before-commit
71
72 Commit_Edit
73     [Documentation]    Commit the change and check the reply.
74     Perform_Test    commit-edit
75
76 name0_In_Config_Running_To_Confirm_Edit_After_Commit
77     [Documentation]    Check that the change is now in the configuration.
78     ${reply}=    Load_And_Send_Message    get-config-edit-after-commit
79     BuiltIn.Should_Contain    ${reply}    <name>name0</name>
80
81 name0_In_Config_Modules_Via_Restconf
82     [Documentation]    Check that the change is also visible through Restconf.
83     ${data}=    Utils.Get_Data_From_URI    config    config:modules    headers=${ACCEPT_XML}
84     BuiltIn.Should_Contain    ${data}    <name>name0</name>
85
86 Terminate_Connection_Gracefully
87     [Documentation]    Close the session and disconnect.
88     Close_ODL_Netconf_Connection_Gracefully
89
90 name0_In_Config_Modules_Via_Restconf_After_Disconnect
91     [Documentation]    Check that the change is still visible through Restconf after Netconf disconnect.
92     ${data}=    Utils.Get_Data_From_URI    config    config:modules    headers=${ACCEPT_XML}
93     BuiltIn.Should_Contain    ${data}    <name>name0</name>
94
95 Reconnect_To_ODL_Netconf_After_Graceful_Session_End
96     [Documentation]    Reconnect to ODL Netconf and fail if that is not possible.
97     Open_ODL_Netconf_Connection
98
99 name0_In_Config_Running_After_Reconnect
100     [Documentation]    Check that the change is now in the configuration.
101     ${reply}=    Load_And_Send_Message    get-config-edit-after-commit
102     BuiltIn.Should_Contain    ${reply}    <name>name0</name>
103
104 name0_In_Config_Modules_Via_Restconf_After_Reconnect
105     [Documentation]    Check that the change is still visible through Restconf after Netconf reconnect.
106     ${data}=    Utils.Get_Data_From_URI    config    config:modules    headers=${ACCEPT_XML}
107     BuiltIn.Should_Contain    ${data}    <name>name0</name>
108
109 Edit_Config_Modules_Create_Shall_Fail_Now
110     [Documentation]    Request a "create" operation of an element that already exists and check that it fails with the correct error (RFC 6241, section 7.2, operation "create").
111     Perform_Test    config-modules-create
112
113 Delete_Modules
114     [Documentation]    Delete the created element from the candidate configuration and check the reply.
115     Perform_Test    config-modules-delete
116
117 Get_Config_Running_To_Confirm_No_Delete_Before_Commit
118     [Documentation]    Make sure the element is still present in the running configuration as the delete command was not committed yet.
119     ${reply}=    Load_And_Send_Message    get-config-no-delete-before-commit
120     BuiltIn.Should_Contain    ${reply}    <name>name0</name>
121
122 Commit_Delete
123     [Documentation]    Commit the deletion of the element and check the reply.
124     Perform_Test    commit-delete
125
126 Get_Config_Running_To_Confirm_Delete_After_Commit
127     [Documentation]    Check that the element is gone.
128     Check_Test_Objects_Not_Present_In_Config    get-config-delete-after-commit
129
130 Restconf_Get_Modules_Shall_Return_404
131     [Documentation]    Check that "Not Found" is returned when Restconf is asked for the deleted element.
132     ${response}=    RequestsLibrary.Get    config    config:modules    ${ACCEPT_XML}
133     BuiltIn.Should_Be_Equal_As_Strings    404    ${response.status_code}
134
135 Commit_No_Transaction
136     [Documentation]    Attempt to perform "commit" when there are no changes in the candidate configuration and check that it fails with the correct error.
137     Test_Commit_With_No_Transactions
138
139 Edit_Config_Another_Modules_Merge_For_Discard
140     [Documentation]    Create an element to be discarded and check the reply.
141     Perform_Test    config-modules-merge-2
142
143 Get_And_Check_Config_Candidate_For_Discard
144     [Documentation]    Check that the element to be discarded is present in the candidate configuration.
145     ${reply}=    Load_And_Send_Message    get-config-candidate
146     BuiltIn.Should_Contain    ${reply}    <name>name1</name>
147     BuiltIn.Should_Not_Contain    ${reply}    name0
148
149 Discard_Changes_Using_Discard_Request
150     [Documentation]    Ask the server to discard the candidate and check the reply.
151     Perform_Test    commit-discard
152
153 Get_And_Check_Config_Candidate_To_Confirm_Discard
154     [Documentation]    Check that the element was really discarded.
155     Check_Test_Objects_Not_Present_In_Config    get-config-candidate-discard
156
157 Edit_Config_Modules_Multiple_Modules_Merge_1
158     [Documentation]    Create the element with "name2" subelement again and check the reply.
159     Perform_Test    merge-multiple-1
160
161 Edit_Config_Modules_Multiple_Modules_Merge_2
162     [Documentation]    Add a "name3" subelement to the element and check the reply.
163     Perform_Test    merge-multiple-2
164
165 Edit_Config_Modules_Multiple_Modules_Merge_3
166     [Documentation]    Add a "name4" subelement to the element and check the reply.
167     Perform_Test    merge-multiple-3
168
169 Commit_Multiple_Modules_Merge
170     [Documentation]    Commit the changes and check the reply.
171     Perform_Test    merge-multiple-commit
172
173 name2_name3_name4_In_Running_Config
174     [Documentation]    Check that the 3 subelements are now present in the running configuration.
175     ${reply}=    Load_And_Send_Message    merge-multiple-check
176     BuiltIn.Should_Contain    ${reply}    <name>name2</name>
177     BuiltIn.Should_Contain    ${reply}    <name>name3</name>
178     BuiltIn.Should_Contain    ${reply}    <name>name4</name>
179
180 name2_name3_name4_In_Config_Modules_Via_Restconf
181     [Documentation]    Check that the 3 subelements are visible via Restconf.
182     ${data}=    Utils.Get_Data_From_URI    config    config:modules    headers=${ACCEPT_XML}
183     BuiltIn.Should_Contain    ${data}    <name>name2</name>
184     BuiltIn.Should_Contain    ${data}    <name>name3</name>
185     BuiltIn.Should_Contain    ${data}    <name>name4</name>
186
187 Abort_Connection_To_Simulate_Session_Failure
188     [Documentation]    Simulate session failure by disconnecting without terminating the session.
189     Abort_ODL_Netconf_Connection
190
191 name2_name3_name4_In_Config_Modules_Via_Restconf_After_Session_Fail
192     [Documentation]    Check that the 3 subelements are visible via Restconf.
193     ${data}=    Utils.Get_Data_From_URI    config    config:modules    headers=${ACCEPT_XML}
194     BuiltIn.Should_Contain    ${data}    <name>name2</name>
195     BuiltIn.Should_Contain    ${data}    <name>name3</name>
196     BuiltIn.Should_Contain    ${data}    <name>name4</name>
197
198 Reconnect_To_ODL_Netconf_After_Session_Failure
199     [Documentation]    Reconnect to ODL Netconf and fail if that is not possible.
200     Open_ODL_Netconf_Connection
201
202 name2_name3_name4_In_Running_Config_After_Session_Failure
203     [Documentation]    Check that the 3 subelements are now present in the running configuration.
204     ${reply}=    Load_And_Send_Message    merge-multiple-check
205     BuiltIn.Should_Contain    ${reply}    <name>name2</name>
206     BuiltIn.Should_Contain    ${reply}    <name>name3</name>
207     BuiltIn.Should_Contain    ${reply}    <name>name4</name>
208
209 Edit_Multiple_Modules_Merge
210     [Documentation]    Add another subelement named "test" to the element and check the reply.
211     Perform_Test    merge-multiple-edit
212
213 Commit_Multiple_Modules_Merge_Edit
214     [Documentation]    Commit the addition of the "test" subelement and check the reply.
215     Perform_Test    merge-multiple-edit-commit
216
217 Check_Multiple_Modules_Merge_Edit
218     [Documentation]    Check that the "test" subelement exists and has correct value for "port" subelement.
219     ${reply}=    Load_And_Send_Message    merge-multiple-edit-check
220     BuiltIn.Should_Contain    ${reply}    <port xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">2022</port>
221
222 Update_Multiple_Modules_Merge
223     [Documentation]    Update the value of the "port" subelement of the "test" subelement and check the reply.
224     Perform_Test    merge-multiple-update
225
226 Commit_Multiple_Modules_Merge_Update
227     [Documentation]    Commit the update and check the reply.
228     Perform_Test    merge-multiple-update-commit
229
230 Check_Multiple_Modules_Merge_Update
231     [Documentation]    Check that the value of the "port" was really updated.
232     ${reply}=    Load_And_Send_Message    merge-multiple-update-check
233     BuiltIn.Should_Contain    ${reply}    <port xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">3000</port>
234
235 Replace_Multiple_Modules_Merge
236     [Documentation]    Replace the content of the "test" with another completely different and check the reply.
237     Perform_Test    merge-multiple-replace
238
239 Commit_Multiple_Modules_Merge_Replace
240     [Documentation]    Commit the replace and check the reply.
241     Perform_Test    merge-multiple-replace-commit
242
243 Check_Multiple_Modules_Merge_Replace
244     [Documentation]    Check that the new content is there and the old content is gone.
245     ${reply}=    Load_And_Send_Message    merge-multiple-replace-check
246     BuiltIn.Should_Contain    ${reply}    <name>test</name>
247     BuiltIn.Should_Contain    ${reply}    <address xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">10.194.132.42</address>
248     BuiltIn.Should_Contain    ${reply}    <tcp-only xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">false</tcp-only>
249     BuiltIn.Should_Contain    ${reply}    <port xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">4000</port>
250     BuiltIn.Should_Contain    ${reply}    <password xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">admin</password>
251     BuiltIn.Should_Contain    ${reply}    <username xmlns="urn:opendaylight:params:xml:ns:yang:controller:md:sal:connector:netconf">admin</username>
252     BuiltIn.Should_Not_Contain    ${reply}    <name>global-event-executor</name>
253     BuiltIn.Should_Not_Contain    ${reply}    <name>binding-osgi-broker</name>
254     BuiltIn.Should_Not_Contain    ${reply}    <name>dom-broker</name>
255     BuiltIn.Should_Not_Contain    ${reply}    <name>global-netconf-dispatcher</name>
256     BuiltIn.Should_Not_Contain    ${reply}    <name>global-netconf-processing-executor</name>
257
258 Remove_Multiple_Modules
259     [Documentation]    Remove the testing "module" element and all its subelements and check the reply.
260     Perform_Test    merge-multiple-remove
261
262 Commit_Multiple_Modules_Removal
263     [Documentation]    Commit the removal and check the reply.
264     Perform_Test    merge-multiple-remove-commit
265
266 Delete_Not_Existing_Module
267     [Documentation]    Attempt to delete the "module" element again and check that it fails with the correct error.
268     Perform_Test    delete-not-existing
269
270 Commit_Delete_Not_Existing_Module
271     [Documentation]    Attempt to commit and check the reply.
272     Test_Commit_With_No_Transactions
273     [Teardown]    Utils.Report_Failure_Due_To_Bug    4455
274
275 Remove_Not_Existing_Module
276     [Documentation]    Attempt to remove the "module" element again and check that the operation is "silently ignored".
277     Perform_Test    remove-not-existing
278
279 Commit_Remove_Not_Existing_Module
280     [Documentation]    Attempt to commit and check the reply.
281     Perform_Test    remove-not-existing-commit
282
283 Close_Session
284     [Documentation]    Close the session and check that it was closed properly.
285     Perform_Test    close-session
286
287 *** Keywords ***
288 Get_Data
289     [Arguments]    ${name}
290     [Documentation]    Load the specified data from the data directory and return it.
291     ${data}=    OperatingSystem.Get_File    ${datadir}${/}${name}.${dataext}
292     [Return]    ${data}
293
294 Create_ODL_Netconf_Connection
295     [Arguments]    ${host}=${CONTROLLER}    ${port}=${ODL_NETCONF_PORT}    ${user}=${ODL_NETCONF_USER}    ${password}=${ODL_NETCONF_PASSWORD}
296     [Documentation]    Open a netconf connecion to the given machine.
297     # The "-s netconf" flag (see the "SSHLibrary.Write" line below)    is not
298     # supported by SSHLibrary, therefore we need to use this elaborate and
299     # pretty tricky way to connect to the ODL Netconf port.
300     # TODO: Extract into NetconfKeywords as there are other tests that are
301     #    going to need to use this operation (Netconf Performance and Scaling
302     #    comes to my mind now as one of the things tested is the performance
303     #    of the direct netconf connection.
304     # TODO: Make this keyword return a dictionary object with all the
305     #    data about the prepared connection neatly packaged. Make all
306     #    the other keywords handling the connection accept such an
307     #    object and pull the data out of it rather than relying on
308     #    global variables. This will allow for tracking more Netconf
309     #    connections, increasing utility.
310     ${control}=    SSHLibrary.Open_Connection    ${host}    prompt=${ODL_SYSTEM_PROMPT}    timeout=10s
311     Utils.Flexible_Controller_Login
312     BuiltIn.Set_Suite_Variable    ${ssh_control}    ${control}
313     ${netconf}=    SSHLibrary.Open_Connection    ${host}    prompt=${ODL_SYSTEM_PROMPT}    timeout=10s
314     Utils.Flexible_Controller_Login
315     BuiltIn.Set_Suite_Variable    ${ssh_netconf}    ${netconf}
316     BuiltIn.Set_Suite_Variable    ${ssh_port}    ${port}
317     BuiltIn.Set_Suite_Variable    ${ssh_user}    ${user}
318     BuiltIn.Set_Suite_Variable    ${ssh_password}    ${password}
319
320 Reopen_ODL_Netconf_Connection
321     [Documentation]    Reopen a closed netconf connection.
322     SSHLibrary.Write    sshpass -p ${ssh_password} ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no ${ssh_user}\@127.0.0.1 -p ${ssh_port} -s netconf
323     ${hello}=    SSHLibrary.Read_Until    ${ODL_NETCONF_PROMPT}
324     SSHLibrary.Switch_Connection    ${ssh_control}
325     ${pid}=    SSHLibrary.Execute_Command    ps -A | grep sshpass | cut -b 1-6
326     BuiltIn.Set_Suite_Variable    ${ssh_netconf_pid}    ${pid}
327     SSHLibrary.Switch_Connection    ${ssh_netconf}
328     [Return]    ${hello}
329
330 Open_ODL_Netconf_Connection
331     [Documentation]    Open a prepared netconf connecion.
332     ${hello}=    Reopen_ODL_Netconf_Connection
333     ${hello_message}=    Get_Data    hello
334     Transmit_Message    ${hello_message}
335     [Return]    ${hello}
336
337 Transmit_Message
338     [Arguments]    ${message}
339     [Documentation]    Transmit message to Netconf connection and discard the echo of the message.
340     SSHLibrary.Write    ${message}
341     SSHLibrary.Write    ${ODL_NETCONF_PROMPT}
342     SSHLibrary.Read_Until    ${ODL_NETCONF_PROMPT}
343
344 Send_Message
345     [Arguments]    ${message}
346     [Documentation]    Send message to Netconf connection and get the reply.
347     Transmit_Message    ${message}
348     ${reply}=    SSHLibrary.Read_Until    ${ODL_NETCONF_PROMPT}
349     [Return]    ${reply}
350
351 Prepare_For_Search
352     [Arguments]    ${searched_string}
353     [Documentation]    Prepare the specified string for searching in Netconf connection replies.
354     ...    The string passed to this keyword is coming from a data
355     ...    file which has different end of line conventions than
356     ...    the actual Netconf reply. This keyword patches the string
357     ...    to match what Netconf actually returns.
358     ${result}=    BuiltIn.Evaluate    "\\r\\n".join("""${searched_string}""".split("\\n"))
359     [Return]    ${result}
360
361 Load_And_Send_Message
362     [Arguments]    ${name}
363     [Documentation]    Load a message from the data file set, send it to Netconf and return the reply.
364     ${request}=    Get_Data    ${name}-request
365     ${reply}=    Send_Message    ${request}
366     [Return]    ${reply}
367
368 Load_Expected_Reply
369     [Arguments]    ${name}
370     [Documentation]    Load the expected reply from the data file set and return it.
371     ${expected_reply}=    Get_Data    ${name}-reply
372     ${expected}=    Prepare_For_Search    ${expected_reply}
373     [Return]    ${expected}
374
375 Abort_ODL_Netconf_Connection
376     [Documentation]    Correctly close the Netconf connection and make sure it is really dead.
377     BuiltIn.Return_From_Keyword_If    ${ssh_netconf_pid} == -1
378     ${kill_command}=    BuiltIn.Set_Variable    kill ${ssh_netconf_pid}
379     BuiltIn.Set_Suite_Variable    ${ssh_netconf_pid}    -1
380     SSHLibrary.Switch_Connection    ${ssh_control}
381     SSHLibrary.Write    ${kill_command}
382     SSHLibrary.Read_Until_Prompt
383     SSHLibrary.Switch_Connection    ${ssh_netconf}
384     SSHLibrary.Read_Until_Prompt
385
386 Close_ODL_Netconf_Connection_Gracefully
387     Perform_Test    close-session
388     Abort_ODL_Netconf_Connection
389
390 Setup_Everything
391     [Documentation]    Setup resources and create session for Restconf checking.
392     SetupUtils.Setup_Utils_For_Setup_And_Teardown
393     RequestsLibrary.Create_Session    config    http://${CONTROLLER}:${RESTCONFPORT}${CONFIG_API}    auth=${AUTH}
394
395 Teardown_Everything
396     [Documentation]    Close the Netconf connection and destroy all sessions in the requests library.
397     Abort_ODL_Netconf_Connection
398     RequestsLibrary.Delete_All_Sessions
399
400 Check_Test_Objects_Not_Present_In_Config
401     [Arguments]    ${name}
402     [Documentation]    Use dataset with the specified name to get the configuration and check that none of our test objects are there.
403     ${reply}=    Load_And_Send_Message    ${name}
404     BuiltIn.Should_Not_Contain    ${reply}    <name>name0</name>
405     BuiltIn.Should_Not_Contain    ${reply}    <name>name1</name>
406     BuiltIn.Should_Not_Contain    ${reply}    <name>name2</name>
407     BuiltIn.Should_Not_Contain    ${reply}    <name>name3</name>
408     BuiltIn.Should_Not_Contain    ${reply}    <name>name4</name>
409
410 Perform_Test
411     [Arguments]    ${name}
412     [Documentation]    Load and send the request from the dataset and compare the returned reply to the one stored in the dataset.
413     ${actual}=    Load_And_Send_Message    ${name}
414     ${expected}=    Load_Expected_Reply    ${name}
415     ${newline}=    BuiltIn.Evaluate    "\\r\\n"
416     BuiltIn.Should_Be_Equal    ${newline}${expected}${ODL_NETCONF_PROMPT}    ${actual}
417     [Return]    ${actual}
418
419 Send_And_Check
420     [Arguments]    ${name}    ${expected}
421     ${actual}=    Load_And_Send_Message    ${name}
422     BuiltIn.Should_Be_Equal    ${expected}    ${actual}
423
424 Test_Commit_With_No_Transactions
425     [Documentation]    Issue a "commit" RPC request and check that it fails with "No current transactions" error.
426     ${reply}=    Load_And_Send_Message    commit-no-transaction
427     ${expected}=    Load_Expected_Reply    commit-no-transaction
428     BuiltIn.Should_Contain    ${reply}    ${expected}