Merge "TSDR H2 user and install documentation"

[docs.git] / manuals / developer-guide / src / main / asciidoc / controller / md-sal-data-tx.adoc

=== MD-SAL Data Transactions

MD-SAL *Data Broker* provides transactional access to conceptual *data trees*
representing configuration and operational state.

NOTE: *Data tree* usually represents state of the modeled data, usually this
      is state of controller, applications and also external systems (network
      devices).

*Transactions* provide *<<_transaction_isolation, stable and isolated view>>*
from other currently running transactions. The state of running transaction and
underlying data tree is not affected by other concurrently running transactions.

.Transaction Types
Write-Only::
    Transaction provides only modification capabilities, but does not provide
    read capabilities. Write-only transaction is allocated using
    `newWriteOnlyTransaction()`.
+
NOTE: This allows less state tracking for
      write-only transactions and allows MD-SAL Clustering to optimize
      internal representation of transaction in cluster.
Read-Write::
    Transaction provides both read and write capabilities. It is allocated using
    `newReadWriteTransaction()`.
Read-Only::
    Transaction provides stable read-only view based on current data tree.
    Read-only view is not affected by any subsequent write transactions.
    Read-only transaction is allocated using `newReadOnlyTransaction()`.
+
NOTE: If an application needs to observe changes itself in data tree, it should use
*data tree listeners* instead of read-only transactions and polling data tree.

Transactions may be allocated using the *data broker* itself or using
*transaction chain*. In the case of *transaction chain*, the new allocated transaction
is not based on current state of data tree, but rather on state introduced by
previous transaction from the same chain, even if the commit for previous transaction
has not yet occurred (but transaction was submitted).


==== Write-Only & Read-Write Transaction

Write-Only and Read-Write transactions provide modification capabilities for
the conceptual data trees.

.Usual workflow for data tree modifications
1. application allocates new transactions using `newWriteOnlyTransaction()`
   or `newReadWriteTransaction()`.
2. application <<_modification_of_data_tree,modifies data tree>> using `put`,
   `merge` and/or `delete`.
3. application finishes transaction using <<_submitting_transaction,`submit()`>>,
   which seals transaction and submits it to be processed.
4. application observes the result of the transaction commit using either blocking
   or asynchronous calls.

The *initial state* of the write transaction is a *stable snapshot* of the current
data tree state captured when transaction was created and it's state and
underlying data tree are not affected by other concurrently running transactions.

Write transactions are *isolated* from other concurrent write transactions. All
*<<_transaction_local_state,writes are local>>* to the transaction and
represents only a *proposal of state change* for data tree and *are not visible*
to any other concurrently running transactions (including read-only transactions).

The transaction *<<_commit_failure_scenarios,commit may fail>>* due to failing
verification of data or concurrent transaction modifying and affected data
in an incompatible way.

===== Modification of Data Tree

Write-only and read-write transaction provides following methods to modify
data tree:

put::
+
[source, java]
----
<T> void put(LogicalDatastoreType store, InstanceIdentifier<T> path, T data);
----
+
Stores a piece of data at a specified path. This acts as an *add / replace*
operation, which is to say that whole subtree will be replaced by the
specified data.


merge::
+
[source, java]
----
<T> void merge(LogicalDatastoreType store, InstanceIdentifier<T> path, T data);
----
+
Merges a piece of data with the existing data at a specified path.
Any *pre-existing data* which are not explicitly overwritten *will be preserved*.
This means that if you store a container, its child subtrees will be merged.

delete::
+
[source, java]
----
void delete(LogicalDatastoreType store, InstanceIdentifier<?> path);
----
+
Removes a whole subtree from a specified path.

===== Submitting transaction

Transaction is submitted to be processed and committed using following method:

[source, java]
----
CheckedFuture<Void,TransactionCommitFailedException> submit();
----

Applications publish the changes proposed in the transaction by calling `submit()`
on the transaction.
This *seals the transaction* (preventing any further writes using this transaction)
and submits it to be processed and applied to global conceptual data tree.
The `submit()` method does not block, but rather returns `ListenableFuture`, which
will complete successfully once processing of transaction is finished and changes
are applied to data tree. If *commit* of data failed, the future will fail with
`TransactionFailedException`.

Application may listen on commit state asynchronously using `ListenableFuture`.

[source, java]
----
Futures.addCallback( writeTx.submit(), new FutureCallback<Void>() { // <1>
        public void onSuccess( Void result ) { // <2>
            LOG.debug("Transaction committed successfully.");
        }

        public void onFailure( Throwable t ) { // <3>
            LOG.error("Commit failed.",e);
        }
    });
----

<1> Submits `writeTx` and registers application provided `FutureCallback`
    on returned future.
<2> Invoked when future completed successfully - transaction `writeTx` was
    successfully committed to data tree.
<3> Invoked when future failed - commit of transaction `writeTx` failed.
    Supplied exception provides additional details and cause of failure.

If application need to block till commit is finished it may use `checkedGet()`
to wait till commit is finished.

[source, java]
----
try {
    writeTx.submit().checkedGet(); // <1>
} catch (TransactionCommitFailedException e) { // <2>
    LOG.error("Commit failed.",e);
}
----

<1> Submits `writeTx` and blocks till commit of `writeTx` is finished. If
    commit fails `TransactionCommitFailedException` will be thrown.
<2> Catches `TransactionCommitFailedException` and logs it.

===== Transaction local state

Read-Write transactions maintain transaction-local state, which renders all
modifications as if they happened, but this is only local to transaction.

Reads from the transaction returns data as if the previous modifications in
transaction already happened.

Let assume initial state of data tree for `PATH` is `A`.
[source, java]
----
ReadWriteTransaction rwTx = broker.newReadWriteTransaction(); // <1>

rwRx.read(OPERATIONAL,PATH).get(); // <2>
rwRx.put(OPERATIONAL,PATH,B); // <3>
rwRx.read(OPERATIONAL,PATH).get(); // <4>
rwRx.put(OPERATIONAL,PATH,C); // <5>
rwRx.read(OPERATIONAL,PATH).get(); // <6>
----

<1> Allocates new `ReadWriteTransaction`.
<2> Read from `rwTx` will return value `A` for `PATH`.
<3> Writes value `B` to `PATH` using `rwTx`.
<4> Read will return value `B` for `PATH`, since previous write occurred in same
    transaction.
<5> Writes value `C` to `PATH` using `rwTx`.
<6> Read will return value `C` for `PATH`, since previous write occurred in same
    transaction.

==== Transaction isolation

Running (not submitted) transactions are isolated from each other and changes
done in one transaction are not observable in other currently running
transaction.

Lets assume initial state of data tree for `PATH` is `A`.

[source, java]
----
ReadOnlyTransaction txRead = broker.newReadOnlyTransaction(); // <1>
ReadWriteTransaction txWrite = broker.newReadWriteTransaction(); // <2>

txRead.read(OPERATIONAL,PATH).get(); // <3>
txWrite.put(OPERATIONAL,PATH,B); // <4>
txWrite.read(OPERATIONAL,PATH).get(); // <5>
txWrite.submit().get(); // <6>
txRead.read(OPERATIONAL,PATH).get(); // <7>
txAfterCommit = broker.newReadOnlyTransaction(); // <8>
txAfterCommit.read(OPERATIONAL,PATH).get(); // <9>
----

<1> Allocates read only transaction, which is based on data tree which
    contains value  `A` for `PATH`.
<2> Allocates read write transaction, which is based on data tree which
    contains value `A` for `PATH`.
<3> Read from read-only transaction returns value `A` for `PATH`.
<4> Data tree is updated using read-write transaction, `PATH` contains `B`.
    Change is not public and only local to transaction.
<5> Read from read-write transaction returns value `B` for `PATH`.
<6> Submits changes in read-write transaction to be committed to data tree.
    Once commit will finish, changes will be published and `PATH` will be
    updated for value `B`. Previously allocated transactions are not affected by
    this change.
<7> Read from previously allocated read-only transaction still returns value `A`
    for `PATH`, since it provides stable and isolated view.
<8> Allocates new read-only transaction, which is based on data tree,
    which contains value `B` for `PATH`.
<9> Read from new read-only transaction return value `B` for `PATH` since
    read-write transaction was committed.

NOTE: Examples contain blocking calls on future only to illustrate
that action happened after other asynchronous action. The use of the blocking call
`ListenableFuture#get()` is discouraged for most use-cases and you should use
`Futures#addCallback(ListenableFuture, FutureCallback)` to listen asynchronously
for result.


==== Commit failure scenarios

A transaction commit may fail because of following reasons:

Optimistic Lock Failure::
Another transaction finished earlier and *modified the same node in a
non-compatible way*. The commit (and the returned future) will fail
with an `OptimisticLockFailedException`.
+
It is the responsibility of the
caller to create a new transaction and submit the same modification again in
order to update data tree.
+
[WARNING]
====
`OptimisticLockFailedException` usually exposes *multiple writers* to
the same data subtree, which may conflict on same resources.

In most cases, retrying may result in a probability of success.

There are scenarios, albeit unusual, where any number of retries will
not succeed. Therefore it is strongly recommended to limit the number of
retries (2 or 3) to avoid an endless loop.
====

Data Validation::
The data change introduced by this transaction *did not pass validation* by
commit handlers or data was incorrectly structured. The returned future will
fail with a `DataValidationFailedException`. User *should not retry* to
create new transaction with same data, since it probably will fail again.

====== Example conflict of two transactions

This example illustrates two concurrent transactions, which derived from
same initial state of data tree and proposes conflicting modifications.

[source, java]
----
WriteTransaction txA = broker.newWriteTransaction();
WriteTransaction txB = broker.newWriteTransaction();

txA.put(CONFIGURATION, PATH, A);    // <1>
txB.put(CONFIGURATION, PATH, B);     // <2>

CheckedFuture<?,?> futureA = txA.submit(); // <3>
CheckedFuture<?,?> futureB = txB.submit(); // <4>
----

<1> Updates `PATH` to value `A` using `txA`
<2> Updates `PATH` to value `B` using `txB`
<3> Seals & submits `txA`. The commit will be processed asynchronously and
    data tree will be updated to contain value `A` for `PATH`.
    The returned `ListenableFuture' will complete successfully once
    state is applied to data tree.
<4> Seals & submits `txB`. Commit of `txB` will fail, because previous transaction
    also modified path in a concurrent way. The state introduced by `txB` will
    not be applied. The returned `ListenableFuture` will fail
    with `OptimisticLockFailedException` exception, which indicates
    that concurrent transaction prevented the submitted transaction from being
    applied.

===== Example asynchronous retry-loop

[source, java]
----
private void doWrite( final int tries ) {
    WriteTransaction writeTx = dataBroker.newWriteOnlyTransaction();

    MyDataObject data = ...;
    InstanceIdentifier<MyDataObject> path = ...;
    writeTx.put( LogicalDatastoreType.OPERATIONAL, path, data );

    Futures.addCallback( writeTx.submit(), new FutureCallback<Void>() {
        public void onSuccess( Void result ) {
            // succeeded
        }

        public void onFailure( Throwable t ) {
            if( t instanceof OptimisticLockFailedException && (( tries - 1 ) > 0)) {
                doWrite( tries - 1 );
            }
        }
      });
}
...
doWrite( 2 );
----

==== Concurrent change compatibility

There are several sets of changes which could be considered incompatible
between two transactions which are derived from same initial state.
Rules for conflict detection applies recursively for each subtree
level.

Following table shows  state changes and failures between two concurrent
transactions, which are based on same initial state, `tx1` is submitted before
`tx2`.

// FIXME: Providing model and concrete data structures will be probably better.

INFO: Following tables stores numeric values and shows data using `toString()`
to simplify examples.

.Concurrent change resolution for leaves and leaf-list items
[options="header"]
|===========================================================
|Initial state | tx1  | tx2 | Observable Result
|Empty |`put(A,1)` |`put(A,2)` |`tx2` will fail, value of `A` is `1`
|Empty |`put(A,1)` |`merge(A,2)` |value of `A` is `2`
|Empty |`merge(A,1)` |`put(A,2)` |`tx2` will fail, value of `A` is `1`
|Empty |`merge(A,1)` |`merge(A,2)` |`A` is `2`
|A=0 |`put(A,1)` |`put(A,2)` |`tx2` will fail, `A` is `1`
|A=0 |`put(A,1)` |`merge(A,2)` |`A` is `2`
|A=0 |`merge(A,1)` |`put(A,2)` |`tx2` will fail, value of `A` is `1`
|A=0 |`merge(A,1)` |`merge(A,2)` |`A` is `2`
|A=0 |`delete(A)` |`put(A,2)` |`tx2` will fail, `A` does not exists
|A=0 |`delete(A)` |`merge(A,2)` |`A` is `2`
|===========================================================

.Concurrent change resolution for containers, lists, list items
[options="header"]
|=======================================================================
|Initial state |`tx1` |`tx2` |Result
|Empty |put(TOP,[]) |put(TOP,[]) |`tx2` will fail, state is TOP=[]

|Empty |put(TOP,[]) |merge(TOP,[]) |TOP=[]

|Empty |put(TOP,[FOO=1]) |put(TOP,[BAR=1]) |`tx2` will fail, state is
TOP=[FOO=1]

|Empty |put(TOP,[FOO=1]) |merge(TOP,[BAR=1]) |TOP=[FOO=1,BAR=1]

|Empty |merge(TOP,[FOO=1]) |put(TOP,[BAR=1]) |`tx2` will fail, state is
TOP=[FOO=1]

|Empty |merge(TOP,[FOO=1]) |merge(TOP,[BAR=1]) |TOP=[FOO=1,BAR=1]

|TOP=[] |put(TOP,[FOO=1]) |put(TOP,[BAR=1]) |`tx2` will fail, state is
TOP=[FOO=1]

|TOP=[] |put(TOP,[FOO=1]) |merge(TOP,[BAR=1]) |state is
TOP=[FOO=1,BAR=1]

|TOP=[] |merge(TOP,[FOO=1]) |put(TOP,[BAR=1]) |`tx2` will fail, state is
TOP=[FOO=1]

|TOP=[] |merge(TOP,[FOO=1]) |merge(TOP,[BAR=1]) |state is
TOP=[FOO=1,BAR=1]

|TOP=[] |delete(TOP) |put(TOP,[BAR=1]) |`tx2` will fail, state is empty
store

|TOP=[] |delete(TOP) |merge(TOP,[BAR=1]) |state is TOP=[BAR=1]

|TOP=[] |put(TOP/FOO,1) |put(TOP/BAR,1]) |state is TOP=[FOO=1,BAR=1]

|TOP=[] |put(TOP/FOO,1) |merge(TOP/BAR,1) |state is TOP=[FOO=1,BAR=1]

|TOP=[] |merge(TOP/FOO,1) |put(TOP/BAR,1) |state is TOP=[FOO=1,BAR=1]

|TOP=[] |merge(TOP/FOO,1) |merge(TOP/BAR,1) |state is TOP=[FOO=1,BAR=1]

|TOP=[] |delete(TOP) |put(TOP/BAR,1) |`tx2` will fail, state is empty
store

|TOP=[] |delete(TOP) |merge(TOP/BAR,1] |`tx2` will fail, state is empty
store

|TOP=[FOO=1] |put(TOP/FOO,2) |put(TOP/BAR,1) |state is TOP=[FOO=2,BAR=1]

|TOP=[FOO=1] |put(TOP/FOO,2) |merge(TOP/BAR,1) |state is
TOP=[FOO=2,BAR=1]

|TOP=[FOO=1] |merge(TOP/FOO,2) |put(TOP/BAR,1) |state is
TOP=[FOO=2,BAR=1]

|TOP=[FOO=1] |merge(TOP/FOO,2) |merge(TOP/BAR,1) |state is
TOP=[FOO=2,BAR=1]

|TOP=[FOO=1] |delete(TOP/FOO) |put(TOP/BAR,1) |state is TOP=[BAR=1]

|TOP=[FOO=1] |delete(TOP/FOO) |merge(TOP/BAR,1] |state is TOP=[BAR=1]
|=======================================================================