controller.git
8 years agoAdd LeaderTransitioning message to RaftActor 07/31507/3
Tom Pantelis [Tue, 15 Dec 2015 22:23:01 +0000 (17:23 -0500)]
Add LeaderTransitioning message to RaftActor

Adds a LeaderTransitioning message that sends a LeaderStateChanged
message to the RoleChangeNotifier actor with a null leaderId. This will
be used in subsequent patches for leadership transfer. This message will
be sent from the leader to its followers. The resulting
LeaderStateChanged message causes the ShardManager to clear its cached
leader info for the shard so subsequent transactions will wait for the new
leader to come on line.

Change-Id: I2a91374992687a478af1c76b74128e7b0f813c45
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoTransaction retry when leader is isolated 30/31230/4
Tom Pantelis [Fri, 11 Dec 2015 23:02:07 +0000 (18:02 -0500)]
Transaction retry when leader is isolated

Implemented retry of transaction ready messages,
ForwardedReadyTransaction, ReadyLocalTransaction, and
BatchedModifications, when the leader is isolated.

Change-Id: I4524dd99f867173f8666de4aed637124ff1efa50
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoTransaction message retry when no shard leader present 83/31183/8
Tom Pantelis [Fri, 11 Dec 2015 08:27:28 +0000 (03:27 -0500)]
Transaction message retry when no shard leader present

Implemented retry of transaction ready messages,
ForwardedReadyTransaction, ReadyLocalTransaction, and
BatchedModifications, when there's no current shard leader.

A new class, ShardTransactionMessageRetrySupport, maintains a
list of messages to retry and handles the retry logic. If there
is no leader, the message is added to the list and a timer
(2 * election time out) is started. If a leader is elected, on leader
changed, the messages are retried. If no leader is elected in time,
the messages are removed and NoShardLeaderException is returned.

Change-Id: Iade3fd245982d75ee97acf0534e9224551d9e45d
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoFixed incorrect model. 38/31638/1
Tony Tkacik [Sat, 19 Dec 2015 17:55:17 +0000 (18:55 +0100)]
Fixed incorrect model.

Change-Id: I6a3e533e04c76a9d9f69815fcf1cbc91b52834c8
Signed-off-by: Tony Tkacik <ttkacik@cisco.com>
8 years agoAdd unit test for ReadyLocalTransaction NPE 63/31463/2
Tom Pantelis [Tue, 15 Dec 2015 17:46:48 +0000 (12:46 -0500)]
Add unit test for ReadyLocalTransaction NPE

This is a follow-up to https://git.opendaylight.org/gerrit/#/c/31448/
to add a unit test for the scenario that caused a null modification
instance passed to ReadyLocalTransaction from
LocalThreePhaseCommitCohort. Basically the LocalThreePhaseCommitCohort
was created with a non-null operationError which was later erroneously
set to null.

Change-Id: I2891b5653b5d425079dbb99ce65e52d3ebbc6a27
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 4774: Wait for prior RO tx creates on tx chain 18/31318/5
Tom Pantelis [Mon, 14 Dec 2015 23:27:10 +0000 (18:27 -0500)]
Bug 4774: Wait for prior RO tx creates on tx chain

Added a priorReadOnlyTxPromises map to TransactionChainProxy that holds
Promise instances for each read-only tx. When the parent class completes
the primary shard lookup and creates the TransactionContext (either success or
failure), onTransactionContextCreated is called which completes the Promise. A
write tx that is created prior to completion will wait on the Promise's Future via
findPrimaryShard.

Change-Id: Ib1a620cfd5be3e38f633b3faf9ef7a31abaaf345
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoAdd missing relatedPath for aggregator's parent 94/31194/2
adetalhouet [Fri, 11 Dec 2015 13:15:32 +0000 (08:15 -0500)]
Add missing relatedPath for aggregator's parent

Change-Id: I93d6f9f47d4da3ae7f113651b4903bc6d9de7f86
Signed-off-by: adetalhouet <adetalhouet@inocybe.com>
8 years agoBUG-1014: adjust compatibility tests 79/31479/2
Robert Varga [Wed, 16 Dec 2015 21:28:13 +0000 (22:28 +0100)]
BUG-1014: adjust compatibility tests

Hydrogen compatibility tests used operational data with configuration
datastore, preventing merge job from completing. Switch to using
operational datastore.

Change-Id: Ifb0dac52523f87d0b6482e4741036ef8395ff3a8
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoMake a method static 27/31427/2
Robert Varga [Wed, 16 Dec 2015 14:28:32 +0000 (15:28 +0100)]
Make a method static

This test method can be made static, make it so to fix an eclipse
warning (with strict warnings).

Change-Id: I492f31782ff2be712d66ce9823bf32db2bc394e1
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoDo not overwrite operation failure 48/31448/2
Robert Varga [Wed, 16 Dec 2015 15:44:40 +0000 (16:44 +0100)]
Do not overwrite operation failure

When a ready-time failure occurs, do not overwrite it with a null
operation error (if no operations failed).

Change-Id: If2ac379e25af7bc93602b91ea5b068974a196771
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoGuard against null transaction IDs 25/31425/1
Robert Varga [Wed, 16 Dec 2015 13:55:09 +0000 (14:55 +0100)]
Guard against null transaction IDs

This pushes down the null guards so that we can track down the source of
nulls.

Change-Id: Id94db7ba1810b2331f625f335ee610b3a22b3c44
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG 2187 Implement Remove Shard Replica RPC 99/30999/3
Moiz Raja [Tue, 8 Dec 2015 17:23:24 +0000 (09:23 -0800)]
BUG 2187 Implement Remove Shard Replica RPC

Change-Id: I8ccdf81903c505ecf77ed9f91b6021aae9fe8d0d
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoBUG 2187 Handle RemoveShardReplica in ShardManager 30/30330/6
Moiz Raja [Fri, 20 Nov 2015 01:35:42 +0000 (17:35 -0800)]
BUG 2187 Handle RemoveShardReplica in ShardManager

Change-Id: Iaaabe6ab351d8f0964d32fa9e7e71416a2a10209
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoRemove DatastoreContext#getDataStoreType() 43/30443/4
Robert Varga [Tue, 1 Dec 2015 16:55:22 +0000 (17:55 +0100)]
Remove DatastoreContext#getDataStoreType()

This method should not be used anywhere, remove it. Also fix all callers
to use getDataStoreName().

Change-Id: I3c9761951df335a1b3925f3756ff0fb90616a614
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG-1014: convert IMDS 37/31137/2
Robert Varga [Thu, 10 Dec 2015 13:07:12 +0000 (14:07 +0100)]
BUG-1014: convert IMDS

This patch fixes the IMDS to properly pass down the TreeType.

Change-Id: I724fac68e62c58ed71eefbf03d50c82fb8d4a4e7
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG-1014: pass proper TreeType to DataTreeFactory 69/29669/12
Robert Varga [Fri, 13 Nov 2015 17:28:54 +0000 (18:28 +0100)]
BUG-1014: pass proper TreeType to DataTreeFactory

Convert the LogicalStoreType into a TreeType and pass it down to the
DataTreeFactory, thus ensuring the resulting DataTree is aware of which
YANG store (CONFIG or OPER) it should enforce.

Change-Id: Ibe9d16161db11f564823df2bf1e73b78f21c4f98
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBug 4774: Add Tx ID to logging on Tx chain failures 17/31317/2
Tom Pantelis [Mon, 14 Dec 2015 19:17:12 +0000 (14:17 -0500)]
Bug 4774: Add Tx ID to logging on Tx chain failures

To help with debugging, it's useful to see the tx ID when a create fails
due to previous tx not ready.

Change-Id: I0547048ea62340a0297affed3512271908eba65a
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoHandle leader change on ForwardedReadyTransaction in Shard 71/31171/3
Tom Pantelis [Fri, 11 Dec 2015 00:41:00 +0000 (19:41 -0500)]
Handle leader change on ForwardedReadyTransaction in Shard

When the ShardWriteTransaction actor sends the ForwardedReadyTransaction
to the Shard, if the local shard is no longer the leader it still tries
to commit the transaction. Replication fails and it ends up timing out
but a side effect is that it persista a new log entry which isn't good.

Therefore I added logic similar as was done with ReadyLocalTransaction
to forward to the leader. For ReadyLocalTransaction, a special
serializer was added to marshal it over the wire as a BatchedModifications
message. Rather then creating another serializer, I converted the
ForwardedReadyTransaction instance to a ReadyLocalTransaction instance.

Change-Id: Ia44e86606846f2af9599ad222b7dcae1bd0cf804
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoMove Tx ready call from ShardWriteTransaction to Shard 70/31170/2
Tom Pantelis [Thu, 10 Dec 2015 23:31:21 +0000 (18:31 -0500)]
Move Tx ready call from ShardWriteTransaction to Shard

On ready, the ShardWriteTransaction actor calls ready on the
ReadWriteShardDataTreeTransaction to obtain the ShardDataTreeCohort
and passes it with the ForwardedReadyTransaction message sent to
the Shard. Internally the ReadWriteShardDataTreeTransaction accesses
its parent, which is either the ShardDataTree or a ShardDataTreeTransactionChain,
to obtain the ShardDataTreeCohort. The parent holds unsynchronized
internal Shard state and is potentially unsafe to indirectly leak to the
transaction actor. This isn't an issue with the ShardDataTree b/c
finishTransaction doesn't access internal state but
ShardDataTreeTransactionChain does.

To be safe, we need to move the ready call to the Shard. Therefore, the
ReadWriteShardDataTreeTransaction instance is now passed with the
ForwardedReadyTransaction and the ShardCommitCoordinator now calls
ready.

Change-Id: I59295936944326b68506380519f8ade3b11164d9
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBUG-4514: clear destroyed modules 98/31298/2
Robert Varga [Mon, 14 Dec 2015 20:03:11 +0000 (21:03 +0100)]
BUG-4514: clear destroyed modules

When the transaction concludes we do not need to know the destroyed
modules anymore. Clear them out as part of the close sequence.

Change-Id: Ia2a43ed424d96a99cc27df7b685e6abf7393f8a3
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoAdd RaftActorServerConfigurationSupport.raftActor 11/30511/2
Gary Wu [Wed, 2 Dec 2015 21:34:47 +0000 (13:34 -0800)]
Add RaftActorServerConfigurationSupport.raftActor

Add raftActor as a field to
RaftActorServerConfigurationSupport to avoid passing
raftActor around through all the method calls.

Change-Id: I19eb16877af98e9e05ec698c321081e211a7e572
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBug 2187: Implement add-replicas-for-all-shards RPC 22/31022/4
Tom Pantelis [Tue, 8 Dec 2015 13:38:47 +0000 (08:38 -0500)]
Bug 2187: Implement add-replicas-for-all-shards RPC

Implemented the add-replicas-for-all-shards RPC. The yang RPC definition
was changed to return a list of results for each shard.

In the unit test I added a shard config dynamically to the new replica
member that doesn't exist in the leader to test a replica failure case.
This revealed an issue for FindPrimary where both member nodes forwarded the
RemoteFindPrimary message back and forth to each other as neither had
a local replica. To fix this, I added a visitedAddresses set to the
RemoteFindPrimary message to prevent the endless mutual recursion.

Change-Id: Icb0329db2c935f9825b81f593b83bdab13fa6b52
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: Prevent non-voting member from initiating elections 47/30947/2
Tom Pantelis [Tue, 8 Dec 2015 01:14:11 +0000 (20:14 -0500)]
Bug 2187: Prevent non-voting member from initiating elections

Change-Id: Id6359e6af5ff7389e1707e683310d103b167b402
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: Timeout the Persist state 93/30493/6
Tom Pantelis [Tue, 1 Dec 2015 12:59:13 +0000 (07:59 -0500)]
Bug 2187: Timeout the Persist state

On add/remove server, if replication consensus isn't reached in the
Persist state, any pending or future operations will remain queued
until they eventually timeout on the caller side with no response. If
consensus is eventually reached, the pending operations would get
processed even though the caller is gone.

To alleviate this, I added a timer to the Persist state (2 * election
timeout). If it times out, pending operations are failed with a
PRIOR_REQUEST_CONSENSUS_TIMEOUT response. Also future operations are
failed if the timeout occurred.

Change-Id: I83ae528d6bec3fb8f8e3da7c5fd4ca75cfeeb4d5
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoCorrect .gitreview (paoject -> project) 69/31269/2
Stephen Kitt [Mon, 14 Dec 2015 12:23:22 +0000 (13:23 +0100)]
Correct .gitreview (paoject -> project)

Change-Id: Ie2894e2c4cf46ccb0c72d4c7521e0fb32a98df0b
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoAdded support for data store type (OPERATIONAL or CONFIG or BOTH) 40/31240/3
Jan Medved [Mon, 14 Dec 2015 01:38:22 +0000 (17:38 -0800)]
Added support for data store type (OPERATIONAL or CONFIG or BOTH)

Change-Id: I33e59fd4fdab2058ce863643f84fede50e79271e
Signed-off-by: Jan Medved <jmedved@cisco.com>
8 years agoBUG-4514: do not retain old internal info 75/31075/2
Robert Varga [Wed, 9 Dec 2015 14:13:35 +0000 (15:13 +0100)]
BUG-4514: do not retain old internal info

Retaining this information past second phase commit leads to it being
retained via dependencyResolver. Thus a series of reconfigurations will
invariably retain the complete history, which is completely wasteful and
constitutes a memory leak (with GC chains of ~1800 hops observed).

Change-Id: Id67b8813a1d55b36f0b55a1c96099b906bf313ad
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG-1014: clean up DatastoreContext.dataStoreType() 68/29668/12
Robert Varga [Fri, 13 Nov 2015 17:15:08 +0000 (18:15 +0100)]
BUG-1014: clean up DatastoreContext.dataStoreType()

This is not really a type, but rather a name. Introduce
logicalStoreType() and dataStoreName() to disambiguate the two uses.

Change-Id: Ibd454ce1dd4428908ada92cb504c478d10ace779
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoAlign commons-net and jolokia with odlparent 48/28848/5
Stephen Kitt [Tue, 27 Oct 2015 14:52:43 +0000 (15:52 +0100)]
Align commons-net and jolokia with odlparent

Remove controller's overrides of commons-net and jolokia; this
upgrades:
* commons-net 3.0.1 -> 3.3
* joloka 1.1.4 -> 1.3.1

Change-Id: I76b787934740b0024bdc4020527f6a8b5dd01c3c
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoRemove CompositeModification field in ShardWriteTransaction 69/31169/1
Tom Pantelis [Wed, 9 Dec 2015 17:12:05 +0000 (12:12 -0500)]
Remove CompositeModification field in ShardWriteTransaction

The CompositeModification field no longer needs to be passed with the
ForwardedReadyTransaction message so remove it.

Change-Id: Ieae3f3898d33f0f1e2bee720e2d3ff77ef80b73e
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoRemoved unused Modification field in ForwardedReadyTransaction 68/31168/1
Tom Pantelis [Wed, 9 Dec 2015 17:00:30 +0000 (12:00 -0500)]
Removed unused Modification field in ForwardedReadyTransaction

The Modification field is no longer used.

Change-Id: I1b0ace3cf7a663fa3c14da1b5785fc05b77203bc
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoConvert all tests that use ForwardReadyTransaction to use BatchedModifications 15/29715/6
Moiz Raja [Sun, 15 Nov 2015 20:23:13 +0000 (12:23 -0800)]
Convert all tests that use ForwardReadyTransaction to use BatchedModifications

The ShardTests that cover various scenarios were all using ForwardedReadyTransaction
which I have now migrated to use BatchedModifications. This removes most direct
usages of this message.

Change-Id: I7b3eae972dfa3daf30d24795fc3f34a699f98eab
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoMake karaf-empty a POM package 28/31128/2
Stephen Kitt [Thu, 10 Dec 2015 09:55:56 +0000 (10:55 +0100)]
Make karaf-empty a POM package

opendaylight-karaf-empty is currently a (default) JAR package, which
causes maven-install-plugin to try to install a JAR at all costs. It
ends up copying pom.xml to opendaylight-karaf-empty-....jar which
breaks anything then processing the JARs and expecting them to be
valid (e.g. for signing). Declaring that opendaylight-karaf-empty is a
POM package still produces the Karaf distributions along with the POM,
but avoids generating an invalid JAR.

Change-Id: I766acd4fd12002a05968b286a5290461c409c92e
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoBUG-4626: create AbstractNormalizedNodeDataOutput 08/31008/5
Robert Varga [Tue, 8 Dec 2015 13:02:50 +0000 (14:02 +0100)]
BUG-4626: create AbstractNormalizedNodeDataOutput

Create an internal NormalizedNodeDataOutput implementation for reuse of
code between stream versions.

Change-Id: I60e68fe150ba49d37a6821b057b9ef93a8e600e5
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG-4626: Introduce NormalizedNodeData{Input,Output} 72/30872/10
Robert Varga [Sun, 6 Dec 2015 22:07:36 +0000 (23:07 +0100)]
BUG-4626: Introduce NormalizedNodeData{Input,Output}

Introduce the interfaces that are actually consumed, so that the
external interface is accurately captured.

Change-Id: I8ba93c279cfb4de063afb9c1078e97080e735e44
Signed-off-by: Robert Varga <robert.varga@pantheon.sk>
8 years agoBUG-4626: Split out stream token types 83/30283/12
Robert Varga [Thu, 26 Nov 2015 11:46:59 +0000 (12:46 +0100)]
BUG-4626: Split out stream token types

Move token types and version into a dedicated internal class, adding
some documentation and introducing planned token types.

Also adds validation of stream version on reception, so that parsing
fails predictably in case of sender doing the worng thing and sending us
an unrecognized version.

Change-Id: I85fd9615b897e0fe4addb7f1807416d7bfe1b903
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoSet config pusher thread to daemon 08/30908/3
Tom Pantelis [Sun, 6 Dec 2015 17:23:04 +0000 (12:23 -0500)]
Set config pusher thread to daemon

Change-Id: Ida1992884e3b6be1e8188e9f6c8393b0baa08db0
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: Add datastoreType to add-shard-replica RPC 59/30559/2
Tom Pantelis [Thu, 3 Dec 2015 14:47:27 +0000 (09:47 -0500)]
Bug 2187: Add datastoreType to add-shard-replica RPC

Change-Id: I4d24063d9fac5f23d207f621879c4369819c0bd5
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 3874: Support of yang modeled AnyXML 50/29550/6
Peter Kajsa [Wed, 11 Nov 2015 13:53:34 +0000 (14:53 +0100)]
Bug 3874: Support of yang modeled AnyXML

Controller changes due to anyXml support in Yangtools.

Notice: Yangtools patch and Controller patch should be merged at once.

Change-Id: I4ff9ef4a0e03f6ad9b30c2a35b97b3a6fa1793d9
Signed-off-by: Peter Kajsa <pkajsa@cisco.com>
8 years agoBUG-4514: clear oldModule/oldInstance when no longer needed 87/30487/3
Robert Varga [Wed, 2 Dec 2015 14:20:22 +0000 (15:20 +0100)]
BUG-4514: clear oldModule/oldInstance when no longer needed

When we acquire a new instance we no longer need the old module nor
instance. Failing to clear these references leads to slow memory leak,
as we retain the complete history of modules as transactions are being
made.

Change-Id: I42899176d335e5f6ac66ecb1dfe080c4dd14ab2a
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoClean NormalizedNodeSerializer up 10/30410/3
Robert Varga [Mon, 30 Nov 2015 23:27:22 +0000 (00:27 +0100)]
Clean NormalizedNodeSerializer up

The map holding deserializer functions should be an immutable constant.
Create a temporary EnumMap and then wrap it via an immutable facade.
Also make the function defitions denser.

Change-Id: I9adc4151c222ca32902ccd59902cbea1c9100b74
Signed-off-by: Robert Varga <robert.varga@pantheon.sk>
8 years agoRevert "Revert "Remove obsolete artifacts from commons.opendaylight"" 94/27494/3
Ed Warnicke [Sat, 26 Sep 2015 19:01:12 +0000 (12:01 -0700)]
Revert "Revert "Remove obsolete artifacts from commons.opendaylight""

This reverts commit 4030381d1997fd5d4764c826632e5f283c2338fd.

This reverts gerrit https://git.opendaylight.org/gerrit/#/c/27467/

Change-Id: I54b0b975956a95d215096397ad881e0919774836
Signed-off-by: Ed Warnicke <hagbard@gmail.com>
8 years agoDeprecate NormalizedNodeInputStreamReader(InputStream) 00/30300/5
Robert Varga [Fri, 27 Nov 2015 16:48:23 +0000 (17:48 +0100)]
Deprecate NormalizedNodeInputStreamReader(InputStream)

Using ByteStreams.newDataInput() allows us to deprecated ineffcient
constructor, keeping the API cleaner.

Change-Id: Ia8fa4114ce82ccb440f859584069a8c6d6a397ea
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoUse ${karaf.version} in AbstractConfigTestBase 97/30497/3
Stephen Kitt [Wed, 2 Dec 2015 17:51:07 +0000 (18:51 +0100)]
Use ${karaf.version} in AbstractConfigTestBase

Instead of hard-coding the default version of Karaf in
AbstractConfigTestBase, pull in the version stored in ${karaf.version}
via a filtered properties file.

Change-Id: Ia739dd9d2ab34520b611122b296664c82687b3a2
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoExplicitly export package 61/30561/3
Jakub Morvay [Thu, 3 Dec 2015 15:41:05 +0000 (16:41 +0100)]
Explicitly export package

Explicitly export
org.opendaylight.controller.cluster.schema.provider.impl package since
maven bundle plugin ignores impl packages by default.

Change-Id: Ic33e5a610611b703c4365b11a8a0cb587fa70030
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
8 years agoLazily create schema context in GlobalBundleScanning* 57/28057/4
Tom Pantelis [Tue, 6 Oct 2015 21:44:10 +0000 (17:44 -0400)]
Lazily create schema context in GlobalBundleScanning*

On GlobalBundleScanningSchemaServiceImpl startup, it calls
tryToUpdateSchemaContext when bundleTracker.open() completes.
This parses and creates a schema context and notifies listeners
of the updated schema context. However we don't have to call
tryToUpdateSchemaContext in start() unless there actually are
listeners registered as the purpose of tryToUpdateSchemaContext
is to update listeners. So made this change. It's not likely there
would be any listeners at that point anway.

I also changed the synchronization so as not to synchronize on 'this'
or the class instance as both are unsafe (users could sync on the
instance causing unwanted contention, either innocently or maliciously).

Change-Id: I435358b0851671b7fbfdc9784577c91ff20556df
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBUG 2187 : Add more parameters to RemoveShardReplica input 08/29908/4
Moiz Raja [Thu, 19 Nov 2015 01:51:29 +0000 (17:51 -0800)]
BUG 2187 : Add more parameters to RemoveShardReplica input

Change-Id: I6e47fa31a04cb901d7495784e4c3590dbdddb520
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoRename packages 40/30540/3
Jakub Morvay [Thu, 3 Dec 2015 09:47:47 +0000 (10:47 +0100)]
Rename packages

Rename package org.opendaylight.controller.cluster.schema.repository ->
org.opendaylight.controller.cluster.schema.provider and also package
org.opendaylight.controller.cluster.schema.repository.impl ->
org.opendaylight.controller.cluster.schema.provider.impl.

Also rename RemoteYangTextSourceImpl -> RemoteYangTextSourceProviderImpl.

Change-Id: Ibb3eb34a7669f7568996fcd4109ae672816a2235
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
8 years agoUpgrade to Karaf 3.0.4 80/30480/2
Stephen Kitt [Wed, 2 Dec 2015 12:19:24 +0000 (13:19 +0100)]
Upgrade to Karaf 3.0.4

With Karaf 3.0.4 we need everything to be in sync (${karaf.version},
SingleFeatureTest and AbstractConfigTestBase). This goes with
Ia6e719ab165586be25d664e226815c0df9478cd2.

Change-Id: I9da7721aa29e3f480e8bc35a57acbe77f89a70c4
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoRemote yang text source provider implementation 07/29107/7
Jakub Morvay [Mon, 2 Nov 2015 08:00:20 +0000 (09:00 +0100)]
Remote yang text source provider implementation

Change-Id: Ie2495b1561ff95508b9a31ffa0aad79ddb59a8c5
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
8 years agoBUG 2187 - Removal of old snapshots in ShardManager 50/29850/5
kalaiselvik [Mon, 23 Nov 2015 08:49:30 +0000 (14:19 +0530)]
BUG 2187 - Removal of old snapshots in ShardManager

In the ShardManager, when persisting the shard information as
snapshots, old snapshots are still retained. These are removed
when the latest snapshot persistence is successful.

Change-Id: I6f138ae364bcaa577fa2e5e3758b3ca75486a469
Signed-off-by: kalaiselvik <Kalaiselvi_K@Dell.com>
8 years agoBug 4564: Implement backup/restore of ShardManagerSnapshot 38/30038/5
Tom Pantelis [Thu, 19 Nov 2015 21:18:31 +0000 (16:18 -0500)]
Bug 4564: Implement backup/restore of ShardManagerSnapshot

Change-Id: I939318ac0414dd43c182399bb6dd6f72180f50b9
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoCDS: Modify ShardManagerTest to use TestActorFactory 37/30037/4
Tom Pantelis [Thu, 19 Nov 2015 19:33:19 +0000 (14:33 -0500)]
CDS: Modify ShardManagerTest to use TestActorFactory

Change-Id: I457df705ce5923eeca15e5633700b77d7e4a339b
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: EOS shard recovery after AddShardReplica 04/29904/6
Tom Pantelis [Wed, 18 Nov 2015 05:50:09 +0000 (00:50 -0500)]
Bug 2187: EOS shard recovery after AddShardReplica

On restart after an EOS shard replica is added and persisted, the
ShardManager recovers its snapshot and attempts to add the local member
to the shard replicas in the configuration. However, since there's no
static module conguration for the EOS shard, the ShardManager can't
create the shard on recovery complete. The shard does get created on
the subsequent CreateShard message however, if there's no local shards
in the static configuration, it creates the shard as inactive, ie with
the DisableElectionsRaftPolicy which we don't want.

To alleviate this, the ShardManager now stores its recovered snapshot
and, on CreateShard, if the shard was in the recovered shard list then
it was pre-existing so is not initialized with the
DisableElectionsRaftPolicy.

I extended
DistributedEntityOwnershipIntegrationTest::testEntityOwnershipShardBootstrapping
to restart the newly created replica and verify it's re-instated
properly. I added the customRaftPolicyClassName to the OnDemandRaftState
so the test can verify.

Testing revealed some timing issues in the EntityOwnershipShard on
re-instatement where pending modifications weren't sent to the leader.
The EntityOwnershipShard does respond to raft behavior state changes to send
pending modifications but, on startup, if the shard stays in the
follower state then no behavior change occurs. In that case the leaderId
changes and onLeaderChanged occurs so I changed it to also notify the
commit coordinator to commit the next batched transaction, if any. I
also did the same for onPeerUp since, in some test runs, the MemberUp
event hadn't occured yet.

Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
Change-Id: Id6bf966e0aa9a0f12f30327c617cb84f10e6b10f

8 years agoCDS: ShardManagerTest cleanup 36/30036/5
Tom Pantelis [Thu, 19 Nov 2015 18:55:05 +0000 (13:55 -0500)]
CDS: ShardManagerTest cleanup

Added a Builder to TestShardManager and modified tests to use the
Builder to gain more consistency between the tests. As a result, I made the
ShardManager builder class abstract for derivation same as was done for
the Shard builder.

I removed the ForwardingShardManager class and merged the functionality
into TestShardManager.

Change-Id: I55471b388a40b9da68bdb249f4cc597d2a0e7f90
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 4355 - Data Tree: Enforce case uniqueness and 'mandatory true' leaf presence 97/30197/4
Peter Kajsa [Wed, 25 Nov 2015 12:56:22 +0000 (13:56 +0100)]
Bug 4355 - Data Tree: Enforce case uniqueness and 'mandatory true' leaf presence

- quickfix of controller unit tests after yangtools changes

Change-Id: Id05ef60abf88008c56ee2d2e00879e3eb680c3d7
Signed-off-by: Peter Kajsa <pkajsa@cisco.com>
8 years agoChange InMemoryJournal to ser/der the data 22/29922/4
Tom Pantelis [Thu, 19 Nov 2015 04:36:36 +0000 (23:36 -0500)]
Change InMemoryJournal to ser/der the data

Performing derialization/de-serialization of data payload in InMemoryJournal
mirrors production and can possibly uncover bugs earlier.

Also changed doAsyncReplayMessages to honor the 'max' param.

Change-Id: I786962a49b4417a3b8b1731528accdd8f33f5dad
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: Refactor MemberNode from ClusterAdminRpcServiceTest 03/29903/6
Tom Pantelis [Wed, 18 Nov 2015 05:13:03 +0000 (00:13 -0500)]
Bug 2187: Refactor MemberNode from ClusterAdminRpcServiceTest

Moved the MemberNode class in ClusterAdminRpcServiceTest so it can be
reused with the DistributedEntityOwnershipIntegrationTest.

Also fixed log statements in a couple other classes.

Change-Id: I66aaa44a30aeabef2e7e3d74f9566f464565995d
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoRemove <repositories> and <pluginRepositories> 74/26574/4
Thanh Ha [Sun, 6 Sep 2015 12:59:36 +0000 (08:59 -0400)]
Remove <repositories> and <pluginRepositories>

Over a year ago we made the decision to remove these sections from poms
project wide to be compliant with Maven Central policies that these
sections should never appear in local project poms.

https://lists.opendaylight.org/pipermail/discuss/2015-January/004482.html

Change-Id: I4ff57905525756ddb246f91083011f8dfe77f4e2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
8 years agoExplicitly set git-review branch to master 15/24415/4
Thanh Ha [Wed, 22 Jul 2015 19:11:58 +0000 (15:11 -0400)]
Explicitly set git-review branch to master

Change-Id: I26dfd2c088fa2ae66597cb99c1def270a2feaeaa
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
8 years agoDeprecate TestHelper 86/30286/3
Stephen Kitt [Fri, 27 Nov 2015 09:49:58 +0000 (10:49 +0100)]
Deprecate TestHelper

This shouldn't be used any more (certainly not for new IT suites).

Change-Id: Iaf4a9c71f978303b8daa1f104175618bc2529b8e
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoSwitch to enunciate 2 00/30100/3
Robert Varga [Mon, 23 Nov 2015 23:51:45 +0000 (00:51 +0100)]
Switch to enunciate 2

Enunciate 2.1.1 supports JDK8.

Change-Id: I448664beea2ccb4be3a286e840f7673968191ca0
Signed-off-by: Robert Varga <robert.varga@pantheon.sk>
8 years agoUse odlparent netty feature 87/30287/4
Robert Varga [Fri, 27 Nov 2015 10:56:55 +0000 (11:56 +0100)]
Use odlparent netty feature

Instead of enumerating the jars, use odlparent-prorovided feature.

Change-Id: I03b20316d9bbb22bcde05414510678afc323fdef
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoUse odlparent-lite as artifacts parent 31/30331/1
Thanh Ha [Sun, 29 Nov 2015 21:42:59 +0000 (16:42 -0500)]
Use odlparent-lite as artifacts parent

We created odlparent-lite to provide a minimal pom which allows projects
to publish to the correct server. Without this ${project}-artifacts may
fail to deploy to Nexus.

Change-Id: Id39b219c48cb7804377724294695f5d17577453e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
8 years agoAutomatic addition of final keywords 82/30282/2
Robert Varga [Thu, 26 Nov 2015 11:44:11 +0000 (12:44 +0100)]
Automatic addition of final keywords

Mostly method arguments, prior to making any other change.

Change-Id: Ie25e0284fea0acf004fe43e4f0ac6f67843be25c
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoRemove antlr4-runtime-osgi-nohead 78/30278/2
Thanh Ha [Fri, 27 Nov 2015 06:45:29 +0000 (01:45 -0500)]
Remove antlr4-runtime-osgi-nohead

This bundle was removed from yangtools so no longer exists. See:
https://git.opendaylight.org/gerrit/28272/

Change-Id: I5871ce5a21f01f8b414df14fbf160708f2b805e8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoProvide the relativePath for benchmark modules 80/30080/2
Stephen Kitt [Mon, 23 Nov 2015 15:57:56 +0000 (16:57 +0100)]
Provide the relativePath for benchmark modules

Change-Id: I7f5a0c74ba8698762155c7e1d48945fafcc2e0e1
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoBUG-1014: expose a proper ShardDataTree constructor 45/29645/6
Robert Varga [Fri, 13 Nov 2015 12:02:33 +0000 (13:02 +0100)]
BUG-1014: expose a proper ShardDataTree constructor

This patch exposes the proper constructor, deprecating the previous one
(which defaults to TreeType.OPERATIONAL). Furthermore convert all tests
to explicitly use OPERATIONAL tree.

The final bit which remains to be figured out is instantiation inside a
Shard instance, which is marked with a FIXME.

Change-Id: Ic8941c8fa5782b162e6faed7bc2d34920debc46e
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoMake ModuleConfig immutable 61/29761/5
Tom Pantelis [Mon, 16 Nov 2015 10:49:05 +0000 (05:49 -0500)]
Make ModuleConfig immutable

The ModuleConfig class has mutator methods for
ModuleShardConfigProviders to initially construct instances. However
once supplied to the CondifuratonImpl they are intended to be immutable
yet the mutator methods expose loopholes around it. Therefore I added a
Builder to ModuleConfig and made ModuleConfig truly immutable.

Change-Id: I0b8070ff3db1563427a6a70ff174053b2a66feca
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoRefactor MessageCollectorActor and DoNothingActor 47/29947/5
Gary Wu [Fri, 20 Nov 2015 01:16:13 +0000 (17:16 -0800)]
Refactor MessageCollectorActor and DoNothingActor

Refactor/consolidate the duplicate copies of
MessageCollectorActor and DoNothingActor used in
org/opendaylight/controller/cluster/datastore and
org/opendaylight/controller/cluster/raft/utils to
use just the one in raft.

Also moved the EchoActor into raft.

Change-Id: I72784a6799ae4331ab52d497d421b9a8bb98f34a
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBUG-4638: fix typedef types 89/30089/2
Robert Varga [Mon, 23 Nov 2015 19:19:28 +0000 (20:19 +0100)]
BUG-4638: fix typedef types

This adds the fix for model.util.type type structure, which differs for
leaf types with default value.

Change-Id: Ibefebda88d5a6876a72b1ff1ebdf0cb639135a07
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoBUG-3516: make PingPongTransactionChain.close() asynchronous 79/30079/1
Robert Varga [Mon, 8 Jun 2015 18:18:31 +0000 (20:18 +0200)]
BUG-3516: make PingPongTransactionChain.close() asynchronous

When the system is critical loads Thread.yield can bring a long-term
blocking effect for hijacked threads like Netty. We should not be
blocking for prolonged time.

Rework the shutdown logic to be asynchronous, and scheduling the
potential outstanding transaction to complete as appropriate. Also fixes
the case where we would end up not reporting a transaction failure if
the transaction is readied, but was not submitted to the backend.

Change-Id: Ic7796a980d9e87242f70b7f7b9cdb30caeab9dd9
Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoChanged the artifact id from 'benchmark-features' to 'features-benchmark' 46/30046/4
Jan Medved [Sun, 22 Nov 2015 03:24:24 +0000 (19:24 -0800)]
Changed the artifact id from 'benchmark-features' to 'features-benchmark'
Added 'odl-mdsal-features' feature definition that contains all benchmark artifacts (api, ds, ntf and rpc)
Added rpcbenchmark
Fixed tabs and white space at the end-of-lines
Fixed more white spaces

Change-Id: I1789ae09c3f316facef38e484310f6c3a4098dd7
Signed-off-by: Jan Medved <jmedved@cisco.com>
8 years agoAdded notification benchmark (ntfbenchmark) and rpc benchmark models 35/29635/4
Jan Medved [Fri, 13 Nov 2015 05:47:38 +0000 (21:47 -0800)]
Added notification benchmark (ntfbenchmark) and rpc benchmark models
Rplaced tabls with spaces

Change-Id: Ic81947d69ddc6286a9ed3be3600f77d46088b6b0
Signed-off-by: Jan Medved <jmedved@cisco.com>
8 years agoFix warnings on unparameterized generic types 44/29944/2
Gary Wu [Thu, 19 Nov 2015 23:38:55 +0000 (15:38 -0800)]
Fix warnings on unparameterized generic types

Fix warnings on unparameterized generic types in
and around BucketStore and Messages.

Change-Id: I867e5f030f88b56c837780e2bae2e1de54266b26
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBug 4651: Implement handling of ClusteredDOMDataTreeChangeListener in CDS 43/29943/2
Tom Pantelis [Thu, 19 Nov 2015 11:26:10 +0000 (06:26 -0500)]
Bug 4651: Implement handling of ClusteredDOMDataTreeChangeListener in CDS

Implemented handling of ClusteredDOMDataTreeChangeListener similar as to
what was done previously for ClusteredDOMDataChangeListener.

I also refactored the listener support classes used by Shard and
extracted generic base classes for the common functionality.

Change-Id: I694a6a4ce41284f7ecd3bf73bc6201e9d5555998
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 4651: Add ClusteredDataTreeChangeListener interface and binding adapter 42/29942/2
Tom Pantelis [Thu, 19 Nov 2015 06:33:58 +0000 (01:33 -0500)]
Bug 4651: Add ClusteredDataTreeChangeListener interface and binding adapter

Change-Id: I1254a73570ded65925374021341f6900b9a7bdf9
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoCDS: Fix deleteSnapshots criteria in SnapshotManager 00/29900/4
Tom Pantelis [Wed, 18 Nov 2015 22:35:50 +0000 (17:35 -0500)]
CDS: Fix deleteSnapshots criteria in SnapshotManager

The SnapshotManager specifies a magic number 43200000 as the timestamp
for the criteria passed to deleteSnapshots. It's unclear where this
number came from but it prevents prior snapshots from getting deleted
as stored snapshot timestamps will be greater than this value (unless
one was created back in the 70's or 80's :)). Since the SnapshotManager
passes a valid upper bounds for the criteria's maxSequenceNr, I changed
it to pass Long.MAX_VALUE for the timestamp.

The ReplicationAndSnapshotsIntegrationTest actually verifies prior
snapshots were deleted by checking for size 1 when querying the
InMemorySnapshotStore. However this only passes b/c the
InMemorySnapshotStore::doDelete is incorrect in that it doesn't compare
the stored snapshot timestamp against the criteria timestamp. So I
changed the InMemorySnapshotStore to correctly compare the timestamps as
well. I found the source for an InMemorySnapshotStore on line and that's
what it does.

Change-Id: Ie7d5eec14f684a469f4b6ff84732c9a9c6042360
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoFix bug in DatastoreContext copy constructor 45/29945/1
Gary Wu [Thu, 19 Nov 2015 23:42:00 +0000 (15:42 -0800)]
Fix bug in DatastoreContext copy constructor

Change-Id: I0ea1f79a8ab3f092a76b690f5f2089c3a2e7d6cb
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBUG 2817 - Create reusable classes for doing an action on finding primary 37/29837/5
Moiz Raja [Wed, 18 Nov 2015 02:55:23 +0000 (18:55 -0800)]
BUG 2817 - Create reusable classes for doing an action on finding primary

For AddShardReplica we are finding the primary and when it is found we send
a ForwardedAddServer* message to self. I need to do something similar for
RemoveShardReplica - so I've extracted the design pattern that was used for
AddShardReplica into a set of reusable classes. To start with this I can
use this for RemoveShardReplica but I think it can be used for other such
messages in future.

Change-Id: Ib625403f6eab5b07bc126af9db3d4e6e566e2038
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoBUG 2817 : Handle ServerRemoved message in Shard/ShardManager 04/29804/5
Moiz Raja [Tue, 17 Nov 2015 03:03:40 +0000 (19:03 -0800)]
BUG 2817 : Handle ServerRemoved message in Shard/ShardManager

When a server is removed and the new ServerConfiguration is
replicated and consensus has been reached on it the RaftActor
sends a ServerReoved message to the Replica which has just been
removed.

This ServerRemoved messsage is received by the Shard and it
forwards the message to the ShardManager. The ShardManager
then removes the replica from it's persistent list.

Change-Id: I9252ab9d9768b549915d8cccf46f102127d97945
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoRefactor MockConfiguration to extend ConfigurationImpl 60/29760/4
Tom Pantelis [Mon, 16 Nov 2015 07:47:21 +0000 (02:47 -0500)]
Refactor MockConfiguration to extend ConfigurationImpl

MockConfiguration is now essentially a wrapper for
ModuleShardConfigProvider whose source is a shard name -> members map.
This will make it easier when adding new methods to Configuration
plus unit tests will now use the producton ConfigurationImpl as this
class is simple enough where we don't really need the functionality mocked.

Change-Id: I88e520b275a658a6d718442ad31c1f1e3603c70c
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 2187: Remove ShardManager mbean replica operations 19/29719/5
Tom Pantelis [Sat, 14 Nov 2015 15:29:12 +0000 (10:29 -0500)]
Bug 2187: Remove ShardManager mbean replica operations

Remove the add/remove shard relica mbean operations as it was decided to
use RPCs instead.

Change-Id: I419a1ec57dfaa9b1d8d55aae5a995d8050b43d70
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoPrevent partial init in DatastoreSnapshotRestore 81/29881/1
Gary Wu [Wed, 18 Nov 2015 19:23:06 +0000 (11:23 -0800)]
Prevent partial init in DatastoreSnapshotRestore

The config subsystem should only push one config
at a time, but in case it doesn't, synchronize
DatastoreSnapshotRestore.initialize() to prevent
partial initialization in the event of concurrent
calls to getAndRemove().

Change-Id: Ie614e8b2045d86ea46b55609bf5cde9e6597b086
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBug 2187: Implement add-shard-replica RPC 05/29705/8
Tom Pantelis [Sat, 14 Nov 2015 05:08:25 +0000 (00:08 -0500)]
Bug 2187: Implement add-shard-replica RPC

The unit test creates 3 actor systems each with their own datastores.
Now that the ShardManager persists shard info and due to the static
nature of the InMemorySnapshotStore, each ShardManager needs to have a
unique persistenceId otherwise the equivalent ShardManager's persistence
Ids will clash. Therefore I added a shardManagerPersistenceId field to
the DatastoreContext so the unit test can provide a uniique Id based on
member name.

Change-Id: I907cd568d64f43586ffc1ec8581e4208f46db327
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoClean up plugin management 31/26531/6
Stephen Kitt [Fri, 25 Sep 2015 12:26:24 +0000 (14:26 +0200)]
Clean up plugin management

A number of plugins are managed by odlparent, so remove unnecessary
entries (i.e. specified identically in odlparent).

Remove all references to ${exam.version} (the dependencies are
inherited from odlparent).

Change-Id: I43ac4a692b7911321b448e788536d58f916657d1
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoBUG 2817 - Basic implementation of RemoveServer in the Raft code 03/29803/3
Moiz Raja [Mon, 16 Nov 2015 23:56:46 +0000 (15:56 -0800)]
BUG 2817 - Basic implementation of RemoveServer in the Raft code

When a RemoveServer is received it may ask for the removal of
a the current leader or one of the followers. As a first pass
we do not support removal of the current leader. To correctly
implement removal of the leader we would have to implement
leader transition which I intend to build in a future patch.

When a follower is removed the server configuration is changed
immediately on the leader and the new configuration persisted
to the journal. When other followers receive the removed
journal entry they would also remove the server from their
configuration, this is the same as what was done for the
AddServer implementation.

As soon as then new configuration is persisted we respond with
success to the caller. This is the same as for AddServer.

When the ServerConfiguration is complete we send a ServerRemoved
message to the follower which has been removed.

Change-Id: I2b85d82cbeef13cca830e3cc212aebbbcd95c818
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoRemove unused ShardCommitCoordinator#CohortEntry constructor 13/29713/4
Moiz Raja [Sun, 15 Nov 2015 02:01:42 +0000 (18:01 -0800)]
Remove unused ShardCommitCoordinator#CohortEntry constructor

Change-Id: I43b478bd6b5467cc46a65c97a5888ce0ec5ded5c
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoFix failure of testCloseCandidateRegistrationInQuickSuccession 11/29811/2
Moiz Raja [Fri, 13 Nov 2015 21:32:13 +0000 (13:32 -0800)]
Fix failure of testCloseCandidateRegistrationInQuickSuccession

Moved checking of whether the ownershipchange event occurred with
hasOwner=false to the loop so that we pass the test only when all
listeners receive that event with hasOwner=false

Change-Id: I463272822e6a39f310fef5996b541e1d06c79548
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoBug 2187: Don't close over internal state in ShardManager 18/29718/4
Tom Pantelis [Sat, 14 Nov 2015 12:41:00 +0000 (07:41 -0500)]
Bug 2187: Don't close over internal state in ShardManager

For AddShardReplica, we use the ask pattern for the FindPrimary and
AddServer messages. However in the OnComplete callbacks we're closing
over internal state which isn't safe since the callback will be notified
outside of the actor's execution context which may result in concurrent
mutation of internal state. Therefore I added internal messages that are
sent to self in the callbacks.

Change-Id: I1f6662a4e473749925046f127cad868e54b761a2
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug 3231 jolokia access should be controlled by aaa 76/28976/9
Ryan Goulding [Thu, 29 Oct 2015 18:48:59 +0000 (14:48 -0400)]
Bug 3231 jolokia access should be controlled by aaa

Due to unfortunate lack of support, we are going to have to just use
basic authentication from config file for now.  I have committed this
patch to upstream jolokia:
https://github.com/rhuss/jolokia/pull/225
which will unlock power for us to use AAA.  However, this won't be
available until a new release is cut on Jolokia's end.

The only options for jolokia-osgi bundle for authentication are basic
file authn (which is implemented in this proposed changeset) and JAAS.
ODL's JAAS is unencrypted and generally disregarded, so basic file
authN was chosen.  By default, the credentials are admin/admin.

Change-Id: I35770bcf13b3cb32e59685e9bbf0ef47d73d132f
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
8 years agoBug 2187: Bootstrap EOS shard when no local shards configured 76/29576/12
Tom Pantelis [Thu, 12 Nov 2015 08:47:27 +0000 (03:47 -0500)]
Bug 2187: Bootstrap EOS shard when no local shards configured

The intended workflow to initially form a cluster dynamically is to
change the role for a second node to say member-2. Since the initial
static shard config is bootstrapped to member-1, no local shards will be
created. However, the entity-ownership shard is special in that it is
intended to exist on every node.

The EOS will be boostrapped as follows:

For the EOS CreateShard message, all unique members for all shards are
obtained from the static shard config. It assumes the local member is
present in the config however in the above workflow it won't be. So on EOS
CreateShard, if the local member isn’t in the initial member list then
it will create the local shard with an empty peer list and
DisableElectionsRaftPolicy so it stays as follower. Also the shard will
be flagged as inactive in the ShardManager. A subsequent
AddShardReplica will be needed to make it active.

The other option is to not create EOS shard but there may be initial
candidate registrations which would be missed unless we add retry logic
in the service class. But the EOS shard already has retry logic so it
would be ideal to leverage it.

I also made changes to the AddShardReplica logic to handle an existing
local shard as will occur for the EOS shard:

 - remove the failure reply if local shard already exists
 - if the local shard exists and the primary shard is the local shard,
   do nothing and return AlreadyExistsException failure reply
 - otherwise send AddServer to the primary
 - on FindPrimary, if the local shard exists but is not active, do a
   remote find as if the local shard doesn't exist
 - on AddServer, if the new server is already in the peer list, the
   ALREADY_EXISTS status is returned. Return AlreadyExistsException
   failure reply
 - on AddServer failure, if the local shard was pre-existing don't
   remove it.

We still want to prevent an AddShardReplica request which one is already
in progress so I added a Set to track this.

I added an integration test for bootstrapping the EOS shard. It starts
with an inactive shard and registers a candidate, which gets queued
since there's no leader. It then issues AddShardReplica and verifies the
candidate gets registered with the leader.

To get this to work required some teaks in the RaftActor and Follower.
When the ShardManager clears the DisableElectionsRaftPolicy, the
RaftActor creates a new Follower instance however it loses the previous
leader Id. If the new server config hasn't been replicated yet then it
has no peers and immediately tries to start an election. Since it has no
peers it goes to Leader wih no followers creating a 2 leader situation.
To alleviate this I transferred the previous leader Id to the new
Follower instance to prevent the immediate election.

Eeven after that the test still didn't work b/c the leader was still not
in the EOS shard peer list so lookup of the leader address returned
null. So I changed getPeerAddress in the RaftActorContext to lookup in
the resolver if no peer info exists.

I also added more units for AddShardReplica to increase code coverage.

Change-Id: Id2a12ae226af69611d5ca5155f5f018cef82dff4
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBug-4636: NotificationSubscriber's exception prevents notifications to other listeners 81/29681/3
Kamal Rameshan [Fri, 13 Nov 2015 19:26:21 +0000 (11:26 -0800)]
Bug-4636: NotificationSubscriber's exception prevents notifications to other listeners

Catch the exception and log it with enough context.

Change-Id: I23c248c59753008e6d09155513b2dba108fbccbf
Signed-off-by: Kamal Rameshan <kramesha@cisco.com>
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoSpecify dsbenchmark's parent POM relativePath 45/29745/1
Stephen Kitt [Mon, 16 Nov 2015 12:53:57 +0000 (13:53 +0100)]
Specify dsbenchmark's parent POM relativePath

This is required to build controller with no pre-existing controller
artifacts.

Change-Id: I7fe9f6ae015a75ddaa5d53dcdd770a214b4322bb
Signed-off-by: Stephen Kitt <skitt@redhat.com>
8 years agoBUG 2187 - Persisting shard list in ShardManager 70/29370/7
kalaiselvik [Sat, 14 Nov 2015 19:00:42 +0000 (00:30 +0530)]
BUG 2187 - Persisting shard list in ShardManager

In ShardManager, the local shard list is persisted as a snapshot.
On recovery, persisted shard list is used to create the shards.
During recovery, obtained persisted information is updated to the
configuration so that it is uniformly available to the DatastoreContext.

Incorporated the comments

Also, as localShards are now created after RecoveryCompletion, the
shardManager mbean is associated with the shardManager immediately
after creation. On creating the localShards, the shards addition
is notified to the mbean object.
In the shardManagerTests involving verification of the syncStatus
and CountDownLatch objects, the testcases are made to wait for
localShard creation by waiting for recoveryCompletion message.

Change-Id: I523ed9b14af4b1b6e272f05faac1cf37abfef336
Signed-off-by: kalaiselvik <Kalaiselvi_K@Dell.com>
8 years agoRemove unused ShardCommitCoordinator constructor parameter 10/29710/3
Moiz Raja [Sat, 14 Nov 2015 22:50:45 +0000 (14:50 -0800)]
Remove unused ShardCommitCoordinator constructor parameter

Change-Id: I1c25a18e6f4ed700547f7cc9931d5a44d31c7b93
Signed-off-by: Moiz Raja <moraja@cisco.com>
8 years agoBug 4564: Implement datastore restore from backup file 43/29243/12
Tom Pantelis [Wed, 4 Nov 2015 08:59:39 +0000 (03:59 -0500)]
Bug 4564: Implement datastore restore from backup file

Added a singleton DatastoreSnapshotRestore class that looks for and
reads a restore file in a specific directory and deserializes the datastore
snapshots. The restore file is then deleted.

The DatastoreSnapshotRestore instance needs to be injected into both
DistributedDatastore instances which are created via separate config
system Module instances. However the only way to inject the
DatastoreSnapshotRestore instance would be to define a yang module
and service. I didn't want to go thru the overhead of all that and I
didn't want the DatastoreSnapshotRestore advertised as a service. So I made
it a static singleton that is created via a new bundle Activator class.

The DatastoreSnapshot instance is passed to the ShardManager which
passes each ShardSnapshot to the corresponding Shard actor. On
recovery complete, the RaftActor takes care of applying the restored
snapshot.

Change-Id: Ied3db4e49b98320abb34e2acf73b27b29232f8d6
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
8 years agoBUG-865: specify DataTreeType explicitly 44/29644/4
Robert Varga [Fri, 13 Nov 2015 11:22:42 +0000 (12:22 +0100)]
BUG-865: specify DataTreeType explicitly

This removes the use of the compatibility create() method and specifies
the requested OPERATIONAL data tree explicitly.

Change-Id: Ib0f84202357cd413b43035450af1ecef0898a0ad
Signed-off-by: Robert Varga <rovarga@cisco.com>
8 years agoFix resource leaks in test cases 63/28763/6
Gary Wu [Fri, 23 Oct 2015 22:13:03 +0000 (15:13 -0700)]
Fix resource leaks in test cases

Close AutoCloseable objects created in
test cases that were not being closed.
Add mock calls for close() methods that
now need to be stubbed.

Change-Id: Iab057a3a1850d024f02656eb1ae82c6fb1486030
Signed-off-by: Gary Wu <gary.wu1@huawei.com>
8 years agoBug 2187: Persisting Actor peerIds' in snapshot 42/29542/3
kalaiselvik [Thu, 12 Nov 2015 08:48:39 +0000 (14:18 +0530)]
Bug 2187: Persisting Actor peerIds' in snapshot

Persisting Raft Actor's peer information in a snapshot and recovering the same
from the snapshot.
Incorporated the comments.

Change-Id: I12831f129b2bdeb1c64f473e94be617f8d6ee487
Signed-off-by: kalaiselvik <Kalaiselvi_K@Dell.com>