From e954c42ed4def1ff1326986d088f1a533311cb95 Mon Sep 17 00:00:00 2001 From: Thanh Ha Date: Fri, 17 Aug 2018 13:39:36 -0400 Subject: [PATCH] Migrate conceptual-data-tree to RST Change-Id: Ic3c417a1f985055b0a80e3162940480c674d4712 Signed-off-by: Thanh Ha --- docs/conceptual-data-tree.rst | 294 ++++++++++++++++++++++++++++++++++ docs/conf.py | 5 + docs/index.rst | 9 +- 3 files changed, 305 insertions(+), 3 deletions(-) create mode 100644 docs/conceptual-data-tree.rst diff --git a/docs/conceptual-data-tree.rst b/docs/conceptual-data-tree.rst new file mode 100644 index 0000000000..23051ef475 --- /dev/null +++ b/docs/conceptual-data-tree.rst @@ -0,0 +1,294 @@ +#################### +Conceptual Data Tree +#################### + +.. _terminology: + +Terminology +=========== + +:Data Tree: + An instantiated logical tree that represents configuration or operational + state data of a modeled problem domain (for example, a controller or a + network) + +:Data Tree Consumer: + A component acting on data, after this data are introduced into one or more + particular subtrees of a Data Tree. + +:Data Tree Identifier: + A unique identifier for a particular subtree of a Data Tree. It is composed + of the logical data store type and the instance identifier of the subtree's + root node. It is represented by a ``DOMDataTreeIdentifier``. + +:Data Tree Producer: + A component responsible for providing data for one or more particular + subtrees of a Data Tree. + +:Data Tree Shard: + A component responsible for providing storage or access to a particular + subtree of a Data Tree. + +:Shard Layout: + A longest-prefix mapping between Data Tree Identifiers and Data Tree Shards + responsible for providing access to a data subtree. + +Basic Concepts +============== + +Data Tree is a Namespace +------------------------ + +The concept of a data tree comes from `RFC6020 +`_. It is is vaguely +split into two instances, configuration and operational. The implicit +assumption is that **config implies oper**, i.e. any configuration data is +also a valid operational data. Further interactions between the two are left +undefined and the YANG language is not strictly extensible in the number and +semantics of these instances, leaving a lot to implementation details. An +outline of data tree use, which is consistent with the current MD-SAL design, +is described in `draft-kwatsen-netmod-opstate +`_. + +The OpenDaylight MD-SAL design makes no inherent assumptions about the +relationship between the configuration and operational data tree instances. +They are treated as separate entities and they are both fully addressable via +the ``DOMDataTreeIdentifier`` objects. It is up to MD-SAL plugins (e.g. +protocol plugins or applications) to maintain this relationship. This reflects +the asynchronous nature of applying configuration and also the fact that the +intended configuration data may be subject to translation (such as template +configuration instantiation). + +Both the configuration and operational namespaces (data trees) are instances +of the Conceptual Data Tree. Any data item in the conceptual data tree is +addressed via a ``YangInstanceIdentifier`` object, which is a unique, +hierarchical, content-based identifier. All applications use the identifier +objects to identify data to MD-SAL services, which in turn are expected to +perform proper namespace management such that logical operation connectivity is +maintained. + +.. todo:: + + Can you reword '...are expected to perform proper namespace management such + that logical operation connectivity is maintained...' - not clear what you + mean + +.. _identifiers-vs-locators: + +Identifiers versus Locators +--------------------------- + +It is important to note that when we talk about Identifiers and Locators, +we **do not** mean `URIs and URLs +`_, +but rather URNs and URLs as strictly separate entities. MD-SAL plugins do not +have access to locators and it is the job of MD-SAL services to provide +location independence. + +The details of how a particular MD-SAL service achieves location independence +is currently left up to the service's implementation, which leads to the +problem of having MD-SAL services cooperate, such as storing data in different +backends (in-memory, SQL, NoSQL, etc.) and providing unified access to all +available data. Note that data availability is subject to capabilities of a +particular storage engine and its operational state, which leads to the design +decision that a ``YangInstanceIdentifier`` lookup needs to be performed in two +steps: + +#. A longest-prefix match is performed to locate the storage backend instance + for that identifier +#. Masked path elements are resolved by the storage engine + +.. _data-tree-shard: + +Data Tree Shard +--------------- + +A process similar to the first step above is performed today by the Distributed +Data Store implementation to split data into Shards. The concept of a Shard as +currently implemented is limited to specifying namespaces, and it does not +allow for pluggable storage engines. + +In context of the Conceptual Data Tree, the concept of a Shard is generalized +as the shorthand equivalent of a storage backend instance. A Shard can be +attached at any (even wildcard) ``YangInstanceIdentifier``. This contract is +exposed via the ``DOMShardedDataTree``, which is an MD-SAL SPI class that +implements an ``YangInstanceIdentifier`` -> ``Shard`` registry service. This is +an omnipresent MD-SAL service, Shard Registry, whose visibility scope is a +single OpenDaylight instance (i.e. a cluster member). **Shard Layout** refers +to the mapping information contained in this service. + +Federation, Replication and High Availability +--------------------------------------------- + +Support for various multi-node scenarios is a concern outside of core MD-SAL. +If a particular scenario requires the shard layout to be replicated (either +fully or partially), it is up to Shard providers to maintain an omnipresent +service on each node, which in turn is responsible for dynamically registering +``DOMDataTreeShard`` instances with the Shard Registry service. + +Since the Shard Layout is strictly local to a particular OpenDaylight instance, +an OpenDaylight cluster is not strictly consistent in its mapping of +``YangInstanceIdentifier`` to data. When a query for the entire data tree is +executed, the returned result will vary between member instances based on the +differences of their Shard Layouts. This allows each node to project its local +operational details, as well as the partitioning of the data set being worked +on based on workload and node availability. + +Partial symmetry of the conceptual data tree can still be maintained to +the extent that a particular deployment requires. For example the Shard +containing the OpenFlow topology can be configured to be registered on all +cluster members, leading to queries into that topology returning consistent +results. + +.. _design: + +Design +====== + +.. _design-listener: + +Data Tree Listener +------------------ + +A Data Tree Listener is a data consumer, for example a process that wants +to act on data after it has been introduced to the Conceptual Data Tree. + +A Data Tree Listener implements the :mdsal-apidoc:`DOMDataTreeListener +` interface and registers itself using +:mdsal-apidoc:`DOMDataTreeService `. + +A Data Tree Listener may register for multiple subtrees. Each time it is +invoked it will be provided with the current state of all subtrees that it +is registered for. + +.. todo:: Consider linking / inlining interface + +.DOMDataTreeListener interface signature + +.. code-block:: java + + public interface DOMDataTreeListener extends EventListener { + + void onDataTreeChanged(Collection changes, // (1) + Map> subtrees); + + void onDataTreeFailed(Collection causes); // (2) + } + +#. Invoked when the data tree to which the Data Tree Listener is subscribed + to changed. `changes` contains the collection of changes, `subtrees` + contains the current state of all subtrees to which the listener is + registered. +#. Invoked when a subtree listening failure occurs. For example, a failure + can be triggered when a connection to an external subtree source is + broken. + + +.. _design-producer: + +Data Tree Producer +------------------ + +A Data Tree Producer represents source of data in system. Data TreeProducer +implementations are not required to implement a specific interface, but +use a :mdsal-apidoc:`DOMDataTreeProducer ` instance +to publish data (i.e. to modify the Conceptual Data Tree). + +A Data Tree Producer is exclusively bound to one or more subtrees of the +Conceptual Data Tree, i.e. binding a Data Tree Producer to a subtree prevents +other Data Tree Producers from modifying the subtree. + +* A failed Data Tree Producer still holds a claim to the namespace to which + it is bound (i.e. the exclusive lock of the subtree) until it is closed. + +:mdsal-apidoc:`DOMDataTreeProducer ` represents a +Data Tree Producer context + +* allows transactions to be submitted to subtrees specified at creation + time +* at any given time there may be a single transaction open. +* once a transaction is submitted, it will proceed to be committed + asynchronously. + +.. todo:: Consider linking / inlining interface + +.DOMDataTreeProducer interface signature + +.. code-block:: java + + public interface DOMDataTreeProducer extends DOMDataTreeProducerFactory, AutoCloseable { + DOMDataWriteTransaction createTransaction(boolean isolated); // (1) + DOMDataTreeProducer createProducer(Collection subtrees); // (2) + } + +#. Allocates a new transaction. All previously allocated transactions must + have been either submitted or canceled. Setting `isolated` to `true` + disables state compression for this transaction. +#. Creates a sub-producer for the provided `subtrees`. The parent producer + loses the ability to access the specified paths until the resulting child + producer is closed. + +.. _design-shard: + +Data Tree Shard +--------------- + +- **A Data Tree Shard** is always bound to either the ``OPERATIONAL``, or the + ``CONFIG`` space, never to both at the same time. + +- **Data Tree Shards** may be nested, the parent shard must be aware of + sub-shards and execute every request in context of a self-consistent view of + sub-shards liveness. Data requests passing through it must be multiplexed + with sub-shard creations/deletions. In other words, if an application creates + a transaction rooted at the parent Shard and attempts to access data residing + in a sub-shard, the parent Shard implementation must coordinate with the + sub-shard implementation to provide the illusion that the data resides in the + parent shard. In the case of a transaction running concurrently with + sub-shard creation or deletion, these operations need to execute atomically + with respect to each other, which is to say that the transactions must + completely execute as if the sub-shard creation/deletion occurred before the + transaction started or as if the transaction completed before the sub-shard + creation/deletion request was executed. This requirement can also be + satisfied by the Shard implementation preventing transactions from + completing. A Shard implementation may choose to abort any open transactions + prior to executing a sub-shard operation. + +- **Shard Layout** is local to an OpenDaylight instance. + +- **Shard Layout** is modified by agents (registering / unregistering Data Tree + Shards) in order to make the Data Tree Shard and the underlaying data + available to plugins and applications executing on that particular + OpenDaylight instance. + +Registering a Shard with the Conceptual Data Tree +------------------------------------------------- + +.. note:: Namespace in this context means a Data Tree Identifier prefix. + +#. **Claim a namespace** - An agent that is registering a shard must prove that + it has sufficient rights to modify the subtree where the shard is going to be + attached. A namespace for the shard is claimed by binding a Data Tree + Producer instance to same subtree where the shard will be bound. The Data + Tree Producer must not have any open child producers, and it should not have + any outstanding transactions. + +#. **Create a shard instance** - Once a namespace is claimed, the agent creates + a shard instance. + +#. **Attach shard** - The agent registers the created shard instance and + provides in the reigstration the Data Tree Producer instance to verify the + namespace claim. The newly created Shard is checked for its ability to + cooperate with its parent shard. If the check is successful, the newly + created Shard is attached to its parent shard and recorded in the Shard + layout. + +#. **Remove namespace claim** (optional) - If the Shard is providing storage for + applications, the agent should close the Data Tree Producer instance to make + the subtree available to applications. + +.. important:: + + Steps 1, 2 and 3 may fail, and the recovery strategy depends + on which step failed and on the failure reason. + +.. todo:: Describe possible failures and recovery scenarios diff --git a/docs/conf.py b/docs/conf.py index 537834136c..0136f14820 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -12,4 +12,9 @@ from docs_conf.conf import * +extensions.append('sphinx.ext.extlinks') extensions.append('sphinxcontrib.plantuml') + +extlinks = { + 'mdsal-apidoc': ('https://github.com/sphinx-doc/sphinx/issues/%s', 'api '), +} diff --git a/docs/index.rst b/docs/index.rst index dc4a0e97cc..5c111367be 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -116,8 +116,11 @@ of API-centric integration. transactions with isolation from other running transactions. * **Transaction chaining** +Table of Contents +================= - .. toctree:: - :maxdepth: 1 +.. toctree:: + :maxdepth: 1 - architecture + architecture + conceptual-data-tree -- 2.36.6