Added doc for fields filtering 87/95687/4
authorJaroslav Tóth <jtoth@frinx.io>
Wed, 7 Apr 2021 06:53:43 +0000 (08:53 +0200)
committerRobert Varga <nite@hq.sk>
Wed, 12 May 2021 15:55:53 +0000 (15:55 +0000)
- User guide - using fields query parameter to leverage NETCONF
  subtree filtering with selected multiple subtrees.
- Developer guide - tutorial for using:
  a. NetconfDOMDataBrokerFieldsExtension
  b. NetconfDataTreeService with added methods

JIRA: NETCONF-735
Change-Id: I8fed93dc45b626ddc99447f0656e671890ad8f17
Signed-off-by: Jaroslav Tóth <jtoth@frinx.io>
docs/developer-guide.rst
docs/user-guide.rst

index 64167d1684567a6bbd03e4dcb15b2c9d8358bb78..68154f1782c69520de910651eed70478878ccea2 100644 (file)
@@ -236,3 +236,183 @@ returns it from ``showNode``.
     More information can be found in the source code of ncmount sample
     app + on wiki:
     https://wiki-archive.opendaylight.org/view/Controller_Core_Functionality_Tutorials:Tutorials:Netconf_Mount
+
+Reading selected fields from device
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Using NETCONF DOM API it is also possible to read only selected fields
+from NETCONF device. NETCONF mountpoint exposes 2 services that provides
+this functionality:
+
+a. **NetconfDataTreeService** - It provides overloaded methods 'get'
+   (operational data) and 'getConfig' (configuration data)
+   with 'List<YangInstanceIdentifier> fields' parameters. This service
+   should be used if transaction properties are not required.
+b. **NetconfDOMDataBrokerFieldsExtension** - It implements DOMDataBroker
+   interface - provides the same transaction functionality plus method
+   for reading of data with selected fields: read(LogicalDatastoreType,
+   YangInstanceIdentifier, List<YangInstanceIdentifier>).
+   Instance of NetconfDOMDataBrokerFieldsExtension can be obtained
+   using 'getExtensions()' method on the DOMDataBroker instance.
+
+'List<YangInstanceIdentifier> fields' parameter specifies list of paths
+that are read from device using subtree filtering. These paths must be
+relative to parent path - returned data follows schema of the last path
+argument of parent path.
+
+**Mechanics**
+
+* parent path: /a/b/c
+* fields: [/x/y; /w/z]
+
+NETCONF server will read data on following paths:
+* /a/b/c/x/y
+* /a/b/c/w/z
+
+And place read data under /a/b/c path - thus returned data will have
+following structure:
+
+.. code-block::
+
+    c: {
+        x: {
+            y: {
+                // data ...
+            }
+        }
+        w: {
+            z: {
+                // data ...
+            }
+        }
+    }
+
+From the view of DOM API, YangInstanceIdentifier (first parameter) represents
+the parent path, and List<YangInstanceIdentifier> represents list of selected
+fields.
+
+**Example: using NetconfDataTreeService**
+
+The following method demonstrates reading of 2 fields under parent list 'l1':
+
+* parent path: /c1/l1
+* fields (leaves 'x1' and 'x2'): [/c2/x1, /c2/l2/x2]
+
+Result will contain whole MapNode with possibly multiple MapEntryNode-s that
+contain only keys of list elements and selected fields.
+
+.. code-block:: java
+
+    public void readData(final DOMMountPoint mountPoint) {
+        final NetconfDataTreeService dataTreeService = mountPoint.getService(NetconfDataTreeService.class).get();
+
+        final YangInstanceIdentifier parentPath = YangInstanceIdentifier.builder()
+                .node(CONTAINER_C1_NI)   // container 'c1' (root element)
+                .node(LIST_L1_NI)        // list 'l1' placed under container 'c1'
+                .build();
+        final YangInstanceIdentifier leafX1Field = YangInstanceIdentifier.builder()
+                .node(CONTAINER_C2_NI)   // container 'c2' placed under list 'l1'
+                .node(LEAF_X1_NI)        // leaf 'x1' placed under container 'c2'
+                .build();
+        final YangInstanceIdentifier leafX2Field = YangInstanceIdentifier.builder()
+                .node(CONTAINER_C2_NI)   // container 'c2' placed under list 'l1'
+                .node(NESTED_LIST_L2_NI) // list 'l2' placed under container 'c2'
+                .node(LEAF_X2_NI)        // leaf 'x2' placed under list 'l2'
+                .build();
+
+        final ListenableFuture<Optional<NormalizedNode<?, ?>>> config = dataTreeService.getConfig(
+                parentPath, Lists.newArrayList(leafX1Field, leafX2Field));
+
+        Futures.addCallback(config, new FutureCallback<Optional<NormalizedNode<?, ?>>>() {
+            @Override
+            public void onSuccess(@Nullable final Optional<NormalizedNode<?, ?>> normalizedNode) {
+                normalizedNode.ifPresent(node -> LOG.info("Read data: {}", NormalizedNodes.toStringTree(node)));
+                /*
+                We expect data with following structure:
+                l1: [
+                    {
+                        key-l1: "k1",
+                        c2: {
+                            x1: 10,
+                            l2: [
+                                {
+                                    key-l2: 1,
+                                    x2: "foo"
+                                },
+                                ...
+                            ]
+                        }
+                    },
+                    ...
+                ]
+                */
+            }
+
+            @Override
+            public void onFailure(final Throwable throwable) {
+                LOG.error("Failed to read data: {}", parentPath, throwable);
+            }
+        });
+    }
+
+**Example: using NetconfDOMDataBrokerFieldsExtension**
+
+The following method demonstrates reading of 2 fields under parent
+container 'c1':
+
+* parent path: /c1
+* fields (leaf-list 'll1 'and container 'c2'): [/ll1, /l1=k1/c2]
+
+Result will contain ContainerNode with identifier 'c1' and children
+nodes (if they are present) of type LeafSetNode and MapNode.
+
+.. code-block:: java
+
+    public void readData(final DOMMountPoint mountPoint) {
+        final DOMDataBroker domDataBroker = mountPoint.getService(DOMDataBroker.class).get();
+        final NetconfDOMDataBrokerFieldsExtension domFieldsDataBroker = domDataBroker.getExtensions().getInstance(
+                NetconfDOMDataBrokerFieldsExtension.class);
+
+        final YangInstanceIdentifier parentPath = YangInstanceIdentifier.builder()
+                .node(CONTAINER_C1_NI)   // container 'c1' (root element)
+                .build();
+        final YangInstanceIdentifier ll1Field = YangInstanceIdentifier.builder()
+                .node(LEAF_LIST_LL1_NI)  // leaf-list 'll1' placed under container 'c1'
+                .build();
+        final YangInstanceIdentifier c2Field = YangInstanceIdentifier.builder()
+                .node(LIST_L1_NI)        // list 'l1' placed under container 'c1'
+                .nodeWithKey(LIST_L1_QN, LIST_L1_KEY_QN, "k1") // specific list entry with key value 'k1'
+                .node(CONTAINER_C2_NI)   // container 'c2' placed under container 'c1'
+                .build();
+
+        final FluentFuture<Optional<NormalizedNode<?, ?>>> data;
+        try (NetconfDOMFieldsReadTransaction roTx = domFieldsDataBroker.newReadOnlyTransaction()) {
+           data = roTx.read(LogicalDatastoreType.CONFIGURATION, parentPath, Lists.newArrayList(ll1Field, c2Field));
+        }
+
+        data.addCallback(new FutureCallback<>() {
+            @Override
+            public void onSuccess(@Nullable final Optional<NormalizedNode<?, ?>> normalizedNode) {
+                normalizedNode.ifPresent(node -> LOG.info("Read data: {}", NormalizedNodes.toStringTree(node)));
+                /*
+                We expect data with following structure:
+                c1: {
+                    ll1: [...],
+                    l1: [
+                        {
+                            l1-key: "k1",
+                            c2: {
+                                // data ...
+                            }
+                        }
+                    ]
+                }
+                */
+            }
+
+            @Override
+            public void onFailure(final Throwable throwable) {
+                LOG.error("Failed to read data: {}", parentPath, throwable);
+            }
+        }, MoreExecutors.directExecutor());
+    }
index 9502f3bcf083c3adf6f5ce697fc4264a72fe3d5d..d0b8b62a4685de602f6849483e3b8b39428cd988 100644 (file)
@@ -1191,3 +1191,378 @@ blueprint configuration file.
 The device **must** initiate the connection and the server will not try to re-establish the
 connection in case of a drop. By requirement, the server cannot assume it has connectivity
 to the device due to NAT or firewalls among others.
+
+Reading data with selected fields
+---------------------------------
+
+Overview
+~~~~~~~~
+
+If user would like to read only selected fields from NETCONF device, it is possible to use
+fields query parameter that is described by RFC-8040. RESTCONF parses content of query
+parameter into format that is accepted by NETCONF subtree filtering - filtering of data is done
+on NETCONF server, not on NETCONF client side. This approach optimizes network traffic load,
+because data in which user doesn't have interest, is not transferred over network.
+
+Next advantages:
+
+* using single RESTCONF request and single NETCONF RPC for reading multiple subtrees
+* possibility to read only selected fields under list node across multiple hierarchies
+  (it cannot be done without proper selection API)
+
+.. note::
+
+  More information about fields query parameter: `RFC 8071 <https://tools.ietf.org/html/rfc8040#section-4.8.3>`__
+
+Preparation of data
+~~~~~~~~~~~~~~~~~~~
+
+Mounting NETCONF device that runs on NETCONF testtool:
+
+.. code-block:: bash
+
+  curl --location --request PUT 'http://127.0.0.1:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=testtool' \
+  --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+  --header 'Content-Type: application/json' \
+  --data-raw '{
+      "node": [
+          {
+              "node-id": "testtool",
+              "netconf-node-topology:host": "127.0.0.1",
+              "netconf-node-topology:port": 36000,
+              "netconf-node-topology:keepalive-delay": 100,
+              "netconf-node-topology:tcp-only": false,
+              "netconf-node-topology:username": "admin",
+              "netconf-node-topology:password": "admin"
+          }
+      ]
+  }'
+
+Setting initial configuration on NETCONF device:
+
+.. code-block:: bash
+
+  curl --location --request PUT 'http://127.0.0.1:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=testtool/yang-ext:mount/test-module:root' \
+  --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+  --header 'Content-Type: application/json' \
+  --data-raw '{
+      "root": {
+          "simple-root": {
+              "leaf-a": "asddhg",
+              "leaf-b": "ffffff",
+              "ll": [
+                  "str1",
+                  "str2",
+                  "str3"
+              ],
+              "nested": {
+                  "sample-x": true,
+                  "sample-y": false
+              }
+          },
+          "list-root": {
+              "branch-ab": 5,
+              "top-list": [
+                  {
+                      "key-1": "ka",
+                      "key-2": "kb",
+                      "next-data": {
+                          "switch-1": [
+                              null
+                          ],
+                          "switch-2": [
+                              null
+                          ]
+                      },
+                      "nested-list": [
+                          {
+                              "identifier": "f1",
+                              "foo": 1
+                          },
+                          {
+                              "identifier": "f2",
+                              "foo": 10
+                          },
+                          {
+                              "identifier": "f3",
+                              "foo": 20
+                          }
+                      ]
+                  },
+                  {
+                      "key-1": "kb",
+                      "key-2": "ka",
+                      "next-data": {
+                          "switch-1": [
+                              null
+                          ]
+                      },
+                      "nested-list": [
+                          {
+                              "identifier": "e1",
+                              "foo": 1
+                          },
+                          {
+                              "identifier": "e2",
+                              "foo": 2
+                          },
+                          {
+                              "identifier": "e3",
+                              "foo": 3
+                          }
+                      ]
+                  },
+                  {
+                      "key-1": "kc",
+                      "key-2": "ke",
+                      "next-data": {
+                          "switch-2": [
+                              null
+                          ]
+                      },
+                      "nested-list": [
+                          {
+                              "identifier": "q1",
+                              "foo": 13
+                          },
+                          {
+                              "identifier": "q2",
+                              "foo": 14
+                          },
+                          {
+                              "identifier": "q3",
+                              "foo": 15
+                          }
+                      ]
+                  }
+              ]
+          }
+      }
+  }'
+
+Examples
+--------
+
+1. Reading whole leaf-list 'll' and leaf 'nested/sample-x' under 'simple-root' container.
+
+RESTCONF request:
+
+.. code-block:: bash
+
+    curl --location --request GET 'http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=testtool/yang-ext:mount/test-module:root/simple-root?content=config&fields=ll;nested/sample-x' \
+    --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+    --header 'Cookie: JSESSIONID=node01h4w82eorc1k61866b71qjgj503.node0'
+
+Generated NETCONF RPC request:
+
+.. code-block:: xml
+
+    <rpc message-id="m-18" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
+        <get-config>
+            <source>
+                <running/>
+            </source>
+            <filter xmlns:ns0="urn:ietf:params:xml:ns:netconf:base:1.0" ns0:type="subtree">
+                <root xmlns="urn:ietf:params:xml:ns:yang:test-model">
+                    <simple-root>
+                        <ll/>
+                        <nested>
+                            <sample-x/>
+                        </nested>
+                    </simple-root>
+                </root>
+            </filter>
+        </get-config>
+    </rpc>
+
+.. note::
+
+    Using fields query parameter it is also possible to read whole leaf-list or list without
+    necessity to specify value / key predicate (without reading parent entity). Such scenario
+    is not permitted in RFC-8040 paths alone - fields query parameter can be used as
+    workaround for this case.
+
+RESTCONF response:
+
+.. code-block:: json
+
+    {
+        "test-module:simple-root": {
+            "ll": [
+                "str3",
+                "str1",
+                "str2"
+            ],
+            "nested": {
+                "sample-x": true
+            }
+        }
+    }
+
+2. Reading all identifiers of 'nested-list' under all elements of 'top-list'.
+
+RESTCONF request:
+
+.. code-block:: bash
+
+    curl --location --request GET 'http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=testtool/yang-ext:mount/test-module:root/list-root?content=config&fields=top-list(nested-list/identifier)' \
+    --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+    --header 'Cookie: JSESSIONID=node01h4w82eorc1k61866b71qjgj503.node0'
+
+Generated NETCONF RPC request:
+
+.. code-block:: xml
+
+    <rpc message-id="m-27" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
+        <get-config>
+            <source>
+                <running/>
+            </source>
+            <filter xmlns:ns0="urn:ietf:params:xml:ns:netconf:base:1.0" ns0:type="subtree">
+                <root xmlns="urn:ietf:params:xml:ns:yang:test-model">
+                    <list-root>
+                        <top-list>
+                            <nested-list>
+                                <identifier/>
+                            </nested-list>
+                            <key-1/>
+                            <key-2/>
+                        </top-list>
+                    </list-root>
+                </root>
+            </filter>
+        </get-config>
+    </rpc>
+
+.. note::
+
+    NETCONF client automatically fetches values of list keys since they are required for correct
+    deserialization of NETCONF response and at the end serialization of response to RESTCONF
+    response (JSON/XML).
+
+RESTCONF response:
+
+.. code-block:: json
+
+    {
+        "test-module:list-root": {
+            "top-list": [
+                {
+                    "key-1": "ka",
+                    "key-2": "kb",
+                    "nested-list": [
+                        {
+                            "identifier": "f3"
+                        },
+                        {
+                            "identifier": "f2"
+                        },
+                        {
+                            "identifier": "f1"
+                        }
+                    ]
+                },
+                {
+                    "key-1": "kb",
+                    "key-2": "ka",
+                    "nested-list": [
+                        {
+                            "identifier": "e3"
+                        },
+                        {
+                            "identifier": "e2"
+                        },
+                        {
+                            "identifier": "e1"
+                        }
+                    ]
+                },
+                {
+                    "key-1": "kc",
+                    "key-2": "ke",
+                    "nested-list": [
+                        {
+                            "identifier": "q3"
+                        },
+                        {
+                            "identifier": "q2"
+                        },
+                        {
+                            "identifier": "q1"
+                        }
+                    ]
+                }
+            ]
+        }
+    }
+
+3. Reading value of leaf 'branch-ab' and all values of leaves 'switch-1' that are placed
+   under 'top-list' list elements.
+
+RESTCONF request:
+
+.. code-block:: bash
+
+    curl --location --request GET 'http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=testtool/yang-ext:mount/test-module:root/list-root?content=config&fields=branch-ab;top-list/next-data/switch-1' \
+    --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+    --header 'Cookie: JSESSIONID=node01jx6o5thwae9t1ft7c2zau5zbz4.node0'
+
+Generated NETCONF RPC request:
+
+.. code-block:: xml
+
+    <rpc message-id="m-42" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
+        <get-config>
+            <source>
+                <running/>
+            </source>
+            <filter xmlns:ns0="urn:ietf:params:xml:ns:netconf:base:1.0" ns0:type="subtree">
+                <root xmlns="urn:ietf:params:xml:ns:yang:test-model">
+                    <list-root>
+                        <branch-ab/>
+                        <top-list>
+                            <next-data>
+                                <switch-1/>
+                            </next-data>
+                            <key-1/>
+                            <key-2/>
+                        </top-list>
+                    </list-root>
+                </root>
+            </filter>
+        </get-config>
+    </rpc>
+
+RESTCONF response:
+
+.. code-block:: json
+
+    {
+        "test-module:list-root": {
+            "branch-ab": 5,
+            "top-list": [
+                {
+                    "key-1": "ka",
+                    "key-2": "kb",
+                    "next-data": {
+                        "switch-1": [
+                            null
+                        ]
+                    }
+                },
+                {
+                    "key-1": "kb",
+                    "key-2": "ka",
+                    "next-data": {
+                        "switch-1": [
+                            null
+                        ]
+                    }
+                },
+                {
+                    "key-1": "kc",
+                    "key-2": "ke"
+                }
+            ]
+        }
+    }