controller.git
7 years agoRemove deprecated RaftActor inner classes 25/36125/4
Tom Pantelis [Mon, 19 Sep 2016 19:19:35 +0000 (15:19 -0400)]
Remove deprecated RaftActor inner classes

These classes have been split out and deprecated in Lithium timeframe,
remove them.

Change-Id: If79245a39202e3bbfc6797d1a570ce8fe5af4363
Signed-off-by: Robert Varga <rovarga@cisco.com>
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix a few javadoc errors 52/45152/2
Robert Varga [Mon, 5 Sep 2016 10:34:23 +0000 (12:34 +0200)]
Fix a few javadoc errors

Fixes a reference and a parameter name.

Change-Id: Ibd625211d2a6873766bb1a0cb666445d39a323ed
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 6659: Fix intermittent PartitionedCandidateOnStartupElectionScenarioTest failure 53/45353/2
Tom Pantelis [Wed, 7 Sep 2016 20:42:51 +0000 (16:42 -0400)]
Bug 6659: Fix intermittent PartitionedCandidateOnStartupElectionScenarioTest failure

The didn't setup the SimpleReplicatedLog, commitIndex, lastAppliedIndex correctly.

Change-Id: I76fbf98f1a227245ca3a61e399258bd3bd4e743a
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
(cherry picked from commit 369664974cc0ed545b86aa78c400bff743b7b178)

7 years agoBug 6540: EOS - Rework behavior of onPeerDown 29/45129/5
Tom Pantelis [Sun, 4 Sep 2016 01:08:41 +0000 (21:08 -0400)]
Bug 6540: EOS - Rework behavior of onPeerDown

https://git.opendaylight.org/gerrit/#/c/26808/ modified the behavior of
onPeerDown to remove all the down node's candidates. However this behavior is
problematic in the case when the shard leader is isolated. The majority partition
will elect a new leader which temporarily results in split-brain and 2 leaders
which independently attempt to remove the other side's candidates. When the partition
is healed, all hell breaks loose trying to reconcile their differences. This is
compounded with the singleton service because it uses 2 entities that are related
to one another.

To alleviate this, I reverted back to the behavior of selecting a new owner for
the entities owned by the down node and leaving the down node as a candidate.
In the case where the down node is the only candidate, it leaves it as the owner.
This doesn't hurt anything and avoids complications with having to re-instate the
down node as owner when it re-joins if it was actually isolated. The idea here is
to keep its candidacy to minimize disruption until proven otherwise since we don't
know if the downed node's process is actually still alive. If another node registers
a candidate it will replace the down node as the owner.

To handle the case where the down node actually restarted, after startup when it
first hears from the leader, it sends a RemoveAllCandidates message to the leader to
remove it from all entities. This cleans out stale candidates should no local client
register a candidate in the new incarnation.

The unit tests revealed an orthogonal issue with the PreLeader state. The PreLeader
switches to Leader when the commit index is up to date but before applying the entries
to the state. However the EOS may commit modifications immediately before the
ApplyState message for prior entries is received. This can result in the "Store tree X  and candidate base Y differ" exception. So I modified the PreLeader behavior to
switch to Leader when the last applied index is up to date. This makes sense b/c
the PreLeader bevavior is intended to protect the state from inconsistencies.

I also fixed a couple bugs where the downPeerMemberNames was accessed with a String
rather than a MemberName instance. This was a remnant of changing downPeerMemberNames
to store MemberName.

Change-Id: I326660c172353539146a2216cc8a70a4b842affe
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoDeprecated in-memory datastore 82/44882/3
Alexis de Talhouët [Tue, 30 Aug 2016 18:31:19 +0000 (14:31 -0400)]
Deprecated in-memory datastore

Change-Id: Ib960e71d0e73b0ebc72be3279345425853ca50cb
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
7 years agoNotify listeners on applySnapshot 28/45028/1
Tom Pantelis [Thu, 1 Sep 2016 16:14:07 +0000 (12:14 -0400)]
Notify listeners on applySnapshot

When a snapshot is applied from the leader, we weren't notifying data change listeners.
We should.

Change-Id: If721c2ce7e6f27aa01f7babc0a0ad3c4468840c1
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 6587: Retain state when transitioning between Leader and IsolatedLeader 99/44899/4
Tom Pantelis [Wed, 31 Aug 2016 04:55:57 +0000 (00:55 -0400)]
Bug 6587: Retain state when transitioning between Leader and IsolatedLeader

If there's a transaction in the COMMIT_PENDING state, ie it has been persisted and
is in the process of being replicated, and the Leader switches to IsolatedLeader, the
ClientRequestTracker state is lost. As a result when the follower(s) come back and
replication consensus is achieved and the tx is applied to state, the tx ID isn't
available and the ShardDataTree applies it as a foreign candidate, leaving the
tx in the pending queue. This prevents subsequent transactions from making progress.

To fix this, we need to retain/copy the internal leader state when transitioning
between Leader and IsolatedLeader.

Change-Id: If06996dccf083fd5d37757fd91fde2eb0eb82ea1
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix intermittent failure in testRemoveShardReplica 53/44953/2
Tom Pantelis [Wed, 31 Aug 2016 16:48:38 +0000 (12:48 -0400)]
Fix intermittent failure in testRemoveShardReplica

This test has failed on jenkins a few times:

java.lang.AssertionError: Shard cars is present
at org.junit.Assert.fail(Assert.java:88)
at org.opendaylight.controller.cluster.datastore.MemberNode.verifyNoShardPresent(MemberNode.java:174)
at org.opendaylight.controller.cluster.datastore.admin.ClusterAdminRpcServiceTest.testRemoveShardReplica(ClusterAdminRpcServiceTest.java:373)

The log output indicates member-2 hadn't re-joined with member-1 yet after it was
restarted. So when RemoveServer was sent to member-1 to remove member-2, it tried to
send the ServerRemoved message to the member-2 shard but it wasn't delivered and thus
the shard wasn't shut down and removed. To alleviate this I added a waitTillReady call
on member-2's config data store to ensure it has synced with the shard leader on
member-1.

Change-Id: I8de9e585998d9f7b2ab8e4fd3f23c1ab222886cc
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix incorrect remove call in ShardManager 49/44949/1
Tom Pantelis [Wed, 31 Aug 2016 16:21:38 +0000 (12:21 -0400)]
Fix incorrect remove call in ShardManager

THe method onRemoveServerReply calls shardReplicaOperationsInProgress.remove
passing the ShardIdentifier however the shardReplicaOperationsInProgress
Set contains shard name Strings. We need to pass shardId.getShardName().

Change-Id: I3441fe8bbe88d65d20f8d9e8d572865f3cd381d8
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 6535: Fix feature config pusher warning 69/44669/6
Tom Pantelis [Thu, 25 Aug 2016 13:52:35 +0000 (09:52 -0400)]
Bug 6535: Fix feature config pusher warning

The feature config pusher assumes config files installed by features
that have XML extension are CSS files and tries to parse them. On failure
it prints a warning but ignores it and moves on. Up till now we've only
had CSS XML files but we now have XML files related to the
clustered-app-config. To avoid the warning I added a check to see if the
root element is "snapshot" if JAXB parsing fails.

Change-Id: I877921afc13564f131f61a1eb8327db71d3638fe
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: expose backing client actor reference 52/44852/1
Robert Varga [Thu, 25 Aug 2016 13:37:53 +0000 (15:37 +0200)]
BUG-5280: expose backing client actor reference

Exposing the actor directly allows us to retain only the context
without having to attach a specific behavior to implementation
utility classes.

Change-Id: I6fd26a23b08f4c7cb9d70d817aaf8deb44d55d88
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: make EmptyQueue public 51/44851/1
Robert Varga [Thu, 25 Aug 2016 12:13:43 +0000 (14:13 +0200)]
BUG-5280: make EmptyQueue public

Allow EmptyQueue to be used externally and correct its javadoc.

Change-Id: I4ae03095844ea235e735ffbdc48f974343a8e9a1
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoFix issue when AE leader differs from prior install snapshot leader 13/44813/3
Tom Pantelis [Mon, 29 Aug 2016 21:11:35 +0000 (17:11 -0400)]
Fix issue when AE leader differs from prior install snapshot leader

When a leader snapshot install is in progress, a follower doesn't process
AppendEntries and merely returns a reply with its last term/index. However,
if an install snapshot is initiated by a leader and there is more than one
chunk to send, it's possible for a leader change to occur prior to completing
sending all the chunks. When this happens, the new leader will begin sending
AppendEntries but the follower won't process them and make progress. We need
to clear the follower's snapshot state if the AppendEntries leaderId doesn't
match the prior install snapshot leaderId.

Change-Id: I4051bd064b6a20f4bcfe38b50656488fcb09274e
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: fix a few warnings 55/44655/1
Robert Varga [Thu, 25 Aug 2016 08:44:00 +0000 (10:44 +0200)]
BUG-5280: fix a few warnings

This fixes javadoc warnings, addsa tiny bit of documentation
and corrects logging during recovery.

Change-Id: I13076ec052febb801a08c05adb7d8affca1fea82
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: add ExplicitAsk utility class 54/44654/1
Robert Varga [Thu, 25 Aug 2016 09:31:24 +0000 (11:31 +0200)]
BUG-5280: add ExplicitAsk utility class

Akka's support for Explicit Ask with Java functions
is broken and requires a workaround. This patch moves
the previous implementation site to a reusable place
and fixes a caller which was broken.

Change-Id: Idc0fc961b1808c23e01a4ae8c4eafdc93b7974f2
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoImplement toString in DataBroker impls 73/44573/2
Tom Pantelis [Tue, 23 Aug 2016 21:45:57 +0000 (17:45 -0400)]
Implement toString in DataBroker impls

PingPongDataBroker extends from ForwardingObject which forwards toString to the
delegate which is the ConcurrentDOMDataBroker. Unfortunately this hides the fact
that it's actually the PingPongDataBroker and has lead to confusion. Similar with the
BindingDOMDataBrokerAdapter. So I implemented toString to appropriately to refect the
true identity and forwarding nature of the classes.

Change-Id: I58b931590ca41bdb90bad31f6fedfe71466787ea
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: Create AbstractProxyHistory class 77/44277/4
Robert Varga [Thu, 18 Aug 2016 14:05:32 +0000 (16:05 +0200)]
BUG-5280: Create AbstractProxyHistory class

Given the connection-oriented nature of SequencedQueue, we
really need to properly encapsulate various aspects of the client,
so we can perform proper state propagation, both during message
transmission and on reconnection.

This is a first step in that direction, which encapsulates client's
sendRequest() and self() methods at proper levels. It furthermore
makes state tracking in proxies consistent with state tracking in
their aggregate counterparts, hence each ProxyTransaction is guaranteed
to have an associated ProxyHistory.

Change-Id: I8c15b234ec813ac427e63a6e077ae17cde443be3
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 6521: Fix ClassNotFoundException in sal-remoterpc-connector 09/44609/2
Tom Pantelis [Wed, 24 Aug 2016 14:17:38 +0000 (10:17 -0400)]
Bug 6521: Fix ClassNotFoundException in sal-remoterpc-connector

Introduced by https://git.opendaylight.org/gerrit/#/c/44553/. The generated yang
files were importing org.opendaylight.controller.cluster. Now that the yang has been
removed we need to explicitly import the package into the bundle.

Change-Id: I049751f08f26bdceed1a6e8c2af7be940d92c591
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 5700 - Backwards compatibility of sharding api's with old api's 13/43613/11
Jakub Morvay [Wed, 10 Aug 2016 12:08:36 +0000 (14:08 +0200)]
Bug 5700 - Backwards compatibility of sharding api's with old api's

Implementation of controller DOMDataBroker interface which delegates
calls to shard aware implementation of md-sal DOMDataBroker

Change-Id: I5694da6d660453ed6a0382006df808cc321d4130
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
7 years agoBUG-5280: move proxy instantiation to AbstractClientHistory 76/44276/4
Robert Varga [Thu, 18 Aug 2016 13:15:12 +0000 (15:15 +0200)]
BUG-5280: move proxy instantiation to AbstractClientHistory

Histories should be the synchronization point for accessing
per-cookie proxies. Move instantiation code, making cookie/history
mapping internal to AbstractClientHistory.

Change-Id: I512e93d72b682668790a5dd213112d772143f045
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: separate request sequence and transmit sequence 33/43333/28
Robert Varga [Mon, 8 Aug 2016 15:04:54 +0000 (17:04 +0200)]
BUG-5280: separate request sequence and transmit sequence

Clean up the confusion in sequence numbers. There are actually
two sequences:
- logical request sequence, which indicates the order of requests
  in which they should be applied to the target entity. It is
  assigned by logic emitting the request.
- transmit sequence within a connection to the backend, as
  initiated by ConnectClientRequest. It is assigned by SequencedQueue
  as it is transmitting requests and reset when a new connection
  to the backend is made.

This requires establishing the concept of a session, which is
a single conversation between frontend and backend. It is severed
whenever the frontend times out and re-established when the leader
is found and it responds with ConnectClientSuccess.

The sending of ConnectClientRequest is not done via the queue,
as it is part of backend resolution process. Since this is not
a performance-critical path, we use simple Patterns.ask() to
send the request and get completion notification -- which we then
translate to ShardBackendInfo.

ConnectClientSuccess gives us backend-preferred version and
backend-specified cap on the number of outstanding messages then
it can handle concurrently. This maximum is used to limit the
transmit path of SequencedQueue, so that it does not attempt
to send more requests at any given time.

Internal queue for unsent requests is kept unbounded for now,
subject to a Semaphore-driven throttle in a follow-up patch.

Change-Id: I61663073bf6632c1ed8c036dee37f1ac39cf7794
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoRemove deprecated CSS files - sal-remoterpc-connector 53/44553/3
Alexis de Talhouët [Tue, 23 Aug 2016 13:44:25 +0000 (09:44 -0400)]
Remove deprecated CSS files - sal-remoterpc-connector

Change-Id: I2d163486e15ca7eaededb45194645f24f00bfca2
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
7 years agoFix issues when persistence enabled 21/44521/3
Tom Pantelis [Mon, 22 Aug 2016 21:32:59 +0000 (17:32 -0400)]
Fix issues when persistence enabled

When persistence is dynamically enabled, we start persisting subsequent log
entries which causes issues on restart due to a gap in journal indexes. We need to
persist a snapshot with the current state to avoid this.

Also if persistence is disabled, we still persist ReplicatedLogEntry instances that
have a PersistentPayload, of which ServerConfigurationPayload is currently the only one.
This also can cause gaps in the persisted journal indexes which cause issues if
persistence is later enabled. To avoid this, we really shouldn't persist
ReplicatedLogEntry instances at all if data persistence is disabled since we don't
add them to the in-memory journal on recovery anyway - we just recover and apply the
ServerConfigurationPayload. Instead we should persist just the
ServerConfigurationPayload.

Change-Id: Ief78d68423b33aac1649220a36d32ff50f493eb7
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoTurn off visibility of GlobalBundleScanningSchemaServiceImpl#start() 49/44549/1
Alexis de Talhouët [Tue, 23 Aug 2016 12:26:32 +0000 (08:26 -0400)]
Turn off visibility of GlobalBundleScanningSchemaServiceImpl#start()

Since the start() method is only used in the createInstance(), it
should be private and not exposed.

Change-Id: I0264d0a66bbfb2536bc4d6c57f27f15584ddfabb
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
7 years agoRevert "Fix feature config pusher warning" 16/44516/1
Tom Pantelis [Mon, 22 Aug 2016 20:05:42 +0000 (16:05 -0400)]
Revert "Fix feature config pusher warning"

This reverts commit 64e760caf2a5d223cca7598b3ff8941ac37c7a96. Unfortunately some projects don't follow the convention of installing their CSS xml files under etc/opendaylight/karaf.

Change-Id: I6194035399eddc1d66cadaa7f9b0964b9a46d92b
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix feature config pusher warning 85/44185/2
Tom Pantelis [Wed, 17 Aug 2016 17:12:31 +0000 (13:12 -0400)]
Fix feature config pusher warning

The feature config pusher assumes config files installed by features
that have XML extension are CSS files and tries to parse them. On failure
it prints a warning but ignores it and moves on. Up till now we've only
had CSS XML files but we now have XML files related to the
clustered-app-config. To avoid the warning I added an additional check
for "opendaylight/karaf" present in the file path. To be complete we would
open the file and inspect the first element but I didn't think that was worth
the effort since all current CSS files are put under opendaylight/karaf and
ODL is moving away from CSS.

Change-Id: I1f0dcda9efa14db330a72e455fe4756c8fbfcaab
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoMake DataTreeCandidatePayload MigratedSerializable 09/44109/2
Robert Varga [Tue, 16 Aug 2016 22:44:41 +0000 (00:44 +0200)]
Make DataTreeCandidatePayload MigratedSerializable

This class has been superseded and we want to purge it from
everywhere. Mark it as MigratedSerializable, which will trigger
a snapshot to be taken.

Change-Id: I1613d3cc2c359296718a20a4263ba848259f46d3
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoTake snapshot after recovery on migrated messages 47/43747/8
Tom Pantelis [Thu, 11 Aug 2016 01:19:59 +0000 (21:19 -0400)]
Take snapshot after recovery on migrated messages

Modified RaftActorRecoverySupport to capture and persist a snapshot
after recovery when there are migrated messages recovered. It utilizes
the new MigratedSerializable interface.

I also created equivalent classes in the persisted packages for
UpdateElectionTerm, DeleteEntries and ApplyJournalEntries that implement
MigratedSerializable and use the Externalizable proxy pattern. The
existing classes were deprecated and readResolve to the new classes.

Change-Id: Ia2e664de9ffd59991c49160424b13bc8ca0bfcbf
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix FinalModifierException due to BindingToNormalizedNodeCodec final 74/44274/2
Tom Pantelis [Thu, 18 Aug 2016 14:31:00 +0000 (10:31 -0400)]
Fix FinalModifierException due to BindingToNormalizedNodeCodec final

The BindingToNormalizedNodeCodec class is final which prevents blueprint from
being able to create a proxy instance when advertising as a service. This is
is OK as it will use the actual instance but it logs an exception:

2016-08-11 23:03:18,215 | INFO  | rint Extender: 2 | ServiceRecipe
| 15 - org.apache.aries.blueprint.core - 1.6.1 | Unable to create a
proxy object for the service .component-2 defined in bundle
org.opendaylight.controller.sal-binding-broker-impl/1.5.0.SNAPSHOT with
id. Returning the original object instead.
org.apache.aries.proxy.FinalModifierException: The class
org.opendaylight.controller.md.sal.binding.impl.BindingToNormalizedNodeCodec
is final.
at
org.apache.aries.proxy.impl.gen.ProxySubclassGenerator.scanForFinalModifiers(ProxySubclassGenerator.java:330)[12:org.apache.aries.proxy.impl:1.0.5]

Although it's logged at INFO level we should avoid it.

Changing BindingToNormalizedNodeCodec to non-final fixes the exception however it causes
an error when it tries to create the proxy b/c some methods are final. To avoid this, weneed to avoid the proxy creation altogether so I added a method to
BindingToNormalizedNodeCodecFactory to advertise the OSGi service instead of using
the blueprint <service> element.

Note: it's not good practice to advertise a service with the actual class but it's
needed with BindingToNormalizedNodeCodec for backwards compatibility with CSS modules
that inject the BindingToNormalizedNodeCodec instance and not its interfaces. The
BindingToNormalizedNodeCodec CSS service identity is deprecated and will/should go away in the future.

Change-Id: I96b6cb8b030de39808de17142d79f8bbd09bf735
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 6278: Move opendaylight-karaf-empty to odlparent 77/44077/4
Ryan Goulding [Tue, 9 Aug 2016 21:00:26 +0000 (17:00 -0400)]
Bug 6278: Move opendaylight-karaf-empty to odlparent

Move opendaylight-karaf-empty from controller to odlparent. As with
moves of other artifacts, the following process will happen:

1) Copy opendaylight-karaf-empty into odlparent and adjust the groupId.
   The patch that handled the copy work is:

   https://git.opendaylight.org/gerrit/#/c/43988/

2) Change opendaylight-karaf-empty in controller to derive from odlparent's
   opendaylight-karaf-empty.  This is handled in this patch.

3) Change references to use the artifact added in #1.

4) Deprecate and remove controller's opendaylight-karaf-empty.

This patch just handles #2, re-parenting the existing controller
opendaylight-karaf-empty such that it derives from odlparent's
opendaylight-karaf-empty (added in #1).

Change-Id: Ifbfedd8a06f5f03900277d005906af83220707cc
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
7 years agoDataBrokerTestModule: use AbstractDataBrokerTest without inheritance 45/43645/3
Michael Vorburger [Wed, 10 Aug 2016 15:22:30 +0000 (17:22 +0200)]
DataBrokerTestModule: use AbstractDataBrokerTest without inheritance

While starting to write JUnit tests (for functional components) with the
AbstractDataBrokerTest, I found myself not liking its design which
forces one to use a class SomeTest extends AbstractDataBrokerTest...

Modern JUnit tests should be written by composition, not enforced
inheritance.  Thus this new DataBrokerTestModule allows one to obtain a
DataBroker suitable for Tests from anywhere.

The *(Test)Module naming convention is obviously inspired by a DI point
of view (think Spring Framework's @Configuration classes, or Google
Guice or Dagger (yay!) *Module classes - which is exactly what this
really is - a particular way to obtain an instance of an
implementationof some service (here a DataBroker) in a certain
environment (here for tests).

The internal implementation of DataBrokerTestModule could be changed
later; I guess ideally what's in AbstractDataBrokerTest could go into
DataBrokerTestModule and AbstractDataBrokerTest could use it, but for
now I'm too lazy to change that, as this does the trick nicely.  Later
implementation changes would be transparent to users of
DataBrokerTestModule.

Change-Id: I9641f527bbc0cb92732f2e513cdd64cc6a837200
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
7 years agoLimit visibility of ShardDataTree methods 06/43006/12
Robert Varga [Wed, 3 Aug 2016 00:13:36 +0000 (02:13 +0200)]
Limit visibility of ShardDataTree methods

Reduce visibility to package-private and add
@VisibleForTesting where this is not possible.

Some of the methods are not converted, as they are used
by the EOS implementation.

Change-Id: Ie7472fec4439e69479d097520fd81be9ea16a88c
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-865: remove String-based getDataChildByName() 88/43588/3
Martin Ciglan [Wed, 10 Aug 2016 08:58:08 +0000 (10:58 +0200)]
BUG-865: remove String-based getDataChildByName()

This is follow-up patch to reflect Yangtools changes
and needs to be merged after
https://git.opendaylight.org/gerrit/#/c/42898

Change-Id: Icb66e8096cb77589de3ee89a87917a528e8d0dc0
Signed-off-by: Martin Ciglan <mciglan@cisco.com>
7 years agoAddress review comments 75/43775/2
Tom Pantelis [Fri, 12 Aug 2016 06:00:36 +0000 (02:00 -0400)]
Address review comments

For https://git.opendaylight.org/gerrit/#/c/42974/.

Change-Id: If054abf5dce4207fe536beeef15ffaad61b63074
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: fix compilation after unrebased merge 78/43778/1
Robert Varga [Fri, 12 Aug 2016 12:03:54 +0000 (14:03 +0200)]
BUG-5280: fix compilation after unrebased merge

Merging the last batch of patches resulted in a mismatch
between newly-introduced tests and method movement. Fix
that up.

Change-Id: I326ad06ff07b902987e397f13f8b0afdb8c45231
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: add maxMessages field to ConnectClientSuccess 28/43328/9
Robert Varga [Mon, 8 Aug 2016 14:31:44 +0000 (16:31 +0200)]
BUG-5280: add maxMessages field to ConnectClientSuccess

This field will act as a hint on how many messages may
be queued by the frontend towards the backend at any
given time.

Change-Id: Ibb8bbe2af9595bc0ecee090acea35aa78a9250b7
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: add FrontendMetadata 24/43124/15
Robert Varga [Wed, 3 Aug 2016 00:52:20 +0000 (02:52 +0200)]
BUG-5280: add FrontendMetadata

This patch adds the frontend tracking abilities for followers.
It also defines the corresponding ShardDataTreeSnapshotMetadata
for use with persistence.

Change-Id: I7e2c6755c3389dcb5284f17a9c6076fb9e7ac95e
Signed-off-by: Robert Varga <rovarga@cisco.com>
Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
7 years agoBUG-5280: persist metadata in snaphots 74/42974/19
Robert Varga [Mon, 1 Aug 2016 21:20:11 +0000 (23:20 +0200)]
BUG-5280: persist metadata in snaphots

This patch adds the wiring in ShardDataTree to persist
various pieces of metadata in a snapshot. It also includes
metadata recovery from a snapshot.

In order to make this work, this patch centralizes all
actual payload and snapshot handling within the ShardDataTree
by introducing explicit entrypoints for each avenue through
which data can be introduced.

Change-Id: Ibc15bd152bd44dd583d67bb7fc61bc8f3086df30
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: split out cds akka client substrate 21/43321/9
Robert Varga [Mon, 8 Aug 2016 13:54:20 +0000 (15:54 +0200)]
BUG-5280: split out cds akka client substrate

This patch splits out the baseline frontend client
into a separate package.

Change-Id: I2d8ca8b81f29a45dd8c30f3bef467fcda94d4887
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoMove MessageTrackerTest 22/43322/7
Robert Varga [Mon, 8 Aug 2016 14:13:42 +0000 (16:13 +0200)]
Move MessageTrackerTest

This moves the test to sal-clustering-commons, so we end
up testing in its home artifact.

Change-Id: Ie001bee6de92381ab6140a3a54e31c854a46ae1d
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 5450: Query akka cluster state on Follower ElectionTimeout 65/43265/6
Tom Pantelis [Fri, 5 Aug 2016 20:10:58 +0000 (16:10 -0400)]
Bug 5450: Query akka cluster state on Follower ElectionTimeout

Added changes to query the akka ClusterState to see if the leader is
actually unreachable or not Up on ElectionTimeout. If not, Follower
reschedules election timer and stays as Follower.

Change-Id: I3a054a82edbe975ad9e27c4d208083b19b392d2d
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoChange InstallSnapshot and reply to use Externalizable Proxy 38/42638/4
Tom Pantelis [Tue, 26 Jul 2016 22:36:06 +0000 (18:36 -0400)]
Change InstallSnapshot and reply to use Externalizable Proxy

This makes InstallSnapshot cleaner with no public no-arg constructor.

I also removed the InstallSnapshot protobuff message. In addition,
SerializableUtils is no longer needed as there's no more protobuff
messages.

Change-Id: I17aa4f7195cf09b798daee5587bbf50ccbc4bff0
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-6111: fix a thinko 28/43628/1
Robert Varga [Wed, 10 Aug 2016 09:24:31 +0000 (11:24 +0200)]
BUG-6111: fix a thinko

Failure to initialize isOpen leads to the codepath never being
triggered.

Change-Id: I20f1b76c9ada581edc1c92c61447fd97d0d1b2ea
Signed-off-by: Robert Varga <rovarga@cisco.com>
(cherry picked from commit 01ca6a43914fb8dd27a24da7476b835fc2570e40)

7 years agoMove ServerConfigurationPayload to cluster.raft.persisted 66/43266/7
Robert Varga [Fri, 5 Aug 2016 17:44:14 +0000 (19:44 +0200)]
Move ServerConfigurationPayload to cluster.raft.persisted

This introduces its mirror copy and modifies the old class
so that it readResolve()s to the new class. It also adjusts
all users to use the new class.

The new class uses Externalizable proxy pattern to allow the
class itself be evolved without breaking compatibility. Also
NoOpPayload is retrofitted this way, which makes all subclasses
of Payload not have their serialization format tied to Payload
itself.

Change-Id: I26010a9e1438dbc4cb1822e1c4dbb51e2b6e538e
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 6278: Switch to use odlparent's karaf-parent 44/43144/3
Ryan Goulding [Thu, 4 Aug 2016 09:24:19 +0000 (05:24 -0400)]
Bug 6278: Switch to use odlparent's karaf-parent

Switch archetypes so they lay down the correct karaf-parent implementation.

Change-Id: Ib9aa057f45141579a0d2bc895d373bd2882f975c
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
7 years agoBump ietf versions to ...10-SNAPSHOT 40/43540/2
Thanh Ha [Tue, 9 Aug 2016 17:43:50 +0000 (13:43 -0400)]
Bump ietf versions to ...10-SNAPSHOT

Bump versions according to:
https://lists.opendaylight.org/pipermail/release/2016-August/007731.html

Change-Id: I3038eaf68217d131bf867b1fcb2abb3abbf76663
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoBug 6348 : car:stop-stress-test RPC to return success & failure counters 59/43259/5
Sai MarapaReddy [Tue, 19 Jul 2016 20:49:21 +0000 (13:49 -0700)]
Bug 6348 : car:stop-stress-test RPC to return success & failure counters

Current RPC car:stop-stress-test doesn't return how many
cars are created or failed. Adding success and failure counters
will help user to determine the number of cars created or failed
during the the process of creation of car tests using
car:stress-test. This patch enhances car:stop-stress-test RPC.

Change-Id: Iff054c8210ce49f06b4fa96ca5a437d9b82deddb
Signed-off-by: Sai MarapaReddy <sai.marapareddy@gmail.com>
Author: Sai MarapaReddy <sai.marapareddy@gmail.com>

7 years agoFix ietf-yang-types version 68/43468/1
Thanh Ha [Tue, 9 Aug 2016 04:54:26 +0000 (00:54 -0400)]
Fix ietf-yang-types version

The version bump script messed up bumping the ietf version. Fixing it
with this patch.

Change-Id: I9b3975582a3a2fe35ab5f4509b6d3a70bf5ca243
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix relative paths for mdsal-it-parent 64/43464/1
Anil Belur [Tue, 9 Aug 2016 01:39:55 +0000 (11:39 +1000)]
Fix relative paths for mdsal-it-parent

Change-Id: Id321a86500216a8cf0ebe5ce7ebb698717996c7c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoBump versions by 0.1.0 for next dev cycle 09/43409/1
Thanh Ha [Mon, 8 Aug 2016 21:50:13 +0000 (17:50 -0400)]
Bump versions by 0.1.0 for next dev cycle

Change-Id: I9c4a4b1e6d0e101392fb19dc68f814e30de4fa5c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoBUG-6111: implement PingPongTransactionChain cancelation 58/43058/3
Robert Varga [Wed, 3 Aug 2016 15:15:57 +0000 (17:15 +0200)]
BUG-6111: implement PingPongTransactionChain cancelation

This patch implements transaction cancelation in PingPongDataBroker,
which has slightly different semantics -- if a transaction is canceled
while being in a batch, proper isolation of the batch is maintained
and after preceding batch completes, the transaction chain is aborted.

Since there is no transaction isolation within a batch, this is the
only course of action we can take.

Change-Id: I0058503165dbfba8748a17a9ef9272265f4bc1c9
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoAdd PMD exclusion for config-generated files 42/43242/2
Robert Varga [Fri, 5 Aug 2016 15:09:31 +0000 (17:09 +0200)]
Add PMD exclusion for config-generated files

Unfortunately PMD does not support wildcard
root exclusions, hence we have to match odlparent
configuration and extend it.

Change-Id: I4bc7a1b8c25b75cb5b348fb2a16f0e5b2c111359
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 5504: Add PreLeader raft state 28/42728/4
Tom Pantelis [Wed, 27 Jul 2016 19:52:53 +0000 (15:52 -0400)]
Bug 5504: Add PreLeader raft state

The following scenario can result in a "store tree and candidate base
differ" IllegalStateException on commit:

A follower receives a replicate and adds it to the log, say at index 1,
but the leader transfers or dies before committing and applying it to the
state. The follower becomes leader and when the next tx is applied, log
index 2, it has to first apply all log entries from the previous term that
hadn't been committed yet, in this case index 1. Since we got consensus for
index 2 that means index 1 has also been replicated to a majority. Therefore
ApplyState is sent for index 1 and then index 2. However index 1 is applied
as a "foreign" candidate while index 2 is in the pre-commit state. When
index 2 is applied the commit fails.

To prevent this scenario, we introduce a new raft state, PreLeader,
which is transitioned to from Candidate if there are uncommitted
entries, ie commit index < last log index. The PreLeader state performs all
the duties of Leader with the added behavior of attempting to commit all
uncommitted entries from the previous leader's term. Raft does not allow a
leader to commit entries from a previous term by simply counting replicas -
only entries from the leader's current term can be committed (§5.4.2). Rather
then waiting for a client interaction to commit a new entry, the PreLeader
state immediately appends a no-op entry (NoopPayload) to the log with the
leader's current term. Once the no-op entry is committed, all prior entries
are committed indirectly. Once all entries are committed, ie commitIndex matches
the last log index, it switches to the normal Leader state.

The PreLeader state is considered an inactive leader state and thus
client transactions are delayed until it transitions to Leader.

Change-Id: I20a541de0eba9b0075b9952dc6d5808943b7bb8f
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: expand ShardDataTree to cover transaction mechanics 97/42497/27
Robert Varga [Mon, 25 Jul 2016 18:59:37 +0000 (20:59 +0200)]
BUG-5280: expand ShardDataTree to cover transaction mechanics

A chunk of ShardCommitCoordinator should actually be implemented
by ShardDataTree. This includes transaction queueing, commit timers,
interaction with user cohorts and persistence.

This patch implements the relevant operations in an message-agnostic,
callback-driven way.

Fix: ShardDataTreeTest (missing ShardStat MBean)

Change-Id: I353bacce8245df85c5f4d6b4cc0ce5416f2f0337
Signed-off-by: Robert Varga <rovarga@cisco.com>
Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix relativePath declaration 60/43060/2
Robert Varga [Wed, 3 Aug 2016 16:03:01 +0000 (18:03 +0200)]
Fix relativePath declaration

Since karaf-parent's parent is outside of controller,
relativePath has to be empty.

Change-Id: I491c73f2d42b8d5f3e159625f8ba01fdadd32497
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoReturn shortened string from TransactionIdentifier.toString 14/42314/3
Tom Pantelis [Fri, 22 Jul 2016 01:30:36 +0000 (21:30 -0400)]
Return shortened string from TransactionIdentifier.toString

For debug logging we need a shortened string for better readability and
grepping. The standard toString is way too long. I changed toString to a
similar compact form that we had before. adding in the frontend generation id
and type, eg

  member-1-datastore-config-fe-1-txn-3
  member-1-datastore-operational-fe-1-chn-2-txn-3

Change-Id: I942eaaa0e8ceedf42eed964f2a2e3a76d8c09806
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoEnable akka WeaklyUp feature 99/42799/3
Tom Pantelis [Fri, 29 Jul 2016 18:33:27 +0000 (14:33 -0400)]
Enable akka WeaklyUp feature

By enabling allow-weakly-up-members, akka will allow new nodes to join a
cluster if there are unreachable nodes. However, this only pertains to
new nodes that weren't previously in the cluster. Unfortunately it
doesn't pertain to node restarts where a node was in the cluster then
attempts to re-join with a new incarnation, which is what we really want.
Despite that, it will at least work for new nodes so I think it's worth
enabling. Akka might be further enhanced to broaden WeaklyUp to include
new incarnations (there's requests for that).

I also changed the ShardManager to handle MemberWeaklyUp events in
the same manner as MemberUp.

Change-Id: I5cf6c1967162b8a9bc6ffb59d34a50560699e4ca
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 6278: Copy karaf-parent from controller to odlparent 49/42649/5
Ryan Goulding [Fri, 22 Jul 2016 07:52:08 +0000 (03:52 -0400)]
Bug 6278: Copy karaf-parent from controller to odlparent

As discussed in the MD-SAL call, there is an architectural need to move
karaf-parent from the controller project to the odlparent project.  This
is particularly useful for karaf upgrades, since right now a bump in karaf
version within odlparent requires a rebuild of controller to reflect the
change in karaf-parent, and our build jobs are not set up to support such
a process.

The move process will be handled in multiple steps:

1) Copy karaf-parent, karaf-branding and opendaylight-karaf-resources to
odlparent.  All three of these should belong in odlparent.  All three must
be moved since karaf-parent depends on the latter two artifacts.  Since
controller depends on odlparent (and not the other way around), they must
be moved upstream to odlparent.

2) Have controller's karaf-parent derive from odlparent's karaf-parent.
This preserves the ability for downstream consumers to derive from the
controller karaf-parent in the interim, while allowing changes to odlparent's
karaf-parent to be recognized since controller does not need to be rebuilt.
[THIS PATCH]

This also involves removing karaf-branding and opendaylight-karaf-resources
from the controller project, since they are no longer needed.  There are two
consumers that need to be patched:
lispflowmapping: https://git.opendaylight.org/gerrit/42647
vtn: https://git.opendaylight.org/gerrit/42648

3) Change all downstream projects to utilize odlparent's karaf-parent.  This
is future work and will be done in several patches.

4) Remove controller's karaf-parent once we feel all downstream consumers
are using the odlparent's karaf-parent.

Change-Id: Ib42ff5212bbfb93883346a19855544df4fb06d61
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
7 years agoDo not use ShardDataTree in PruningDataTreeModificationTest 75/42975/5
Robert Varga [Tue, 2 Aug 2016 13:33:25 +0000 (15:33 +0200)]
Do not use ShardDataTree in PruningDataTreeModificationTest

This test requires on a DataTree, hence use that.

Change-Id: I37697121f6686cdfe6b1d71ca87ff79281619532
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: add client connect messages 66/42866/7
Robert Varga [Sun, 31 Jul 2016 21:55:28 +0000 (23:55 +0200)]
BUG-5280: add client connect messages

When a frontend is attempting to re-establish communication
with the backend it sends its coordinates and various other
information to a backend.

Sending ConnectClientRequest initiates a handshake, to which
the backend will respond either with a failure, or with an
adjusted ConnectClientSuccess.

Change-Id: I58ba9a2103f80e528654222f82f07416f7d7815e
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoHandle DeleteSnapshots response messages 00/42800/3
Tom Pantelis [Fri, 29 Jul 2016 19:23:23 +0000 (15:23 -0400)]
Handle DeleteSnapshots response messages

This is a follow-up to https://git.opendaylight.org/gerrit/#/c/42272/.
I didn't think deleteSnapshots returned a response but it does - I see
a warning for unhandled DeleteSnapshotsSuccess. I added handling for
DeleteSnapshotsSuccess and DeleteSnapshotsFailure. For the latter I log
a warning but don't fail the actor.

Change-Id: Ibb41e5124eb22530f98a5ef958abffc556dea4cf
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix missing LeaderStateChanged event 97/42597/4
Tom Pantelis [Tue, 26 Jul 2016 15:55:09 +0000 (11:55 -0400)]
Fix missing LeaderStateChanged event

In RaftActor, the logic to detect a leader state change compares the last
valid leader Id with the current behavior leader Id. Consider the
following leader Id change sequence:

  "member-1" -> null (goes leaderless)
  null -> "member-1" (member-1 becomes leader again)

The first state change will send a LeaderStateChanged event to the
ShardManager with null leader Id causing the ShardManager to clean its
primary shard info cache. However for the second state change, no
LeaderStateChanged event is sent b/c the new leader Id is the same as
the last valid/non-null leader Id. Therefore transactions fail due to no
shard leader.

I changed it to use the last leader Id (null or not) for the comparison
so every state change is detected.

Change-Id: I060872d4712e040b60acfc998914b394a40943af
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoImprove leader election convergence 69/42969/3
Tom Pantelis [Tue, 2 Aug 2016 02:23:33 +0000 (22:23 -0400)]
Improve leader election convergence

When 2 nodes startup with the first node's log behind the second node's,
it usually takes several election rounds to converge - I've seen
anywhere from 40 s to 3 min, depending on timing. What happens is that
the first node goes to Candidate first but it's RequestVote is rejected
by the seconds node. Shortly after the seconds node goes to Candidate -
the term is higher than the first which causes the first node to go back
to Follower. However it doesn't respond to the RequestVote. Then the
first node goes to Candidate and the cycle repeats. Eventually, due to
the election variance, the seconds node times out first and the second
node process the RequestVote and grants it. But it can take more than 10
cycles.

We can improve the convergence by allowing a Candidate to process and
respond to RequestVote when the sender's term is greater. It still
transitions to Follower as per the raft rules. The raft paper does not
say whether or not a Candidate can/should process a RequestVote in this
case but it seems to make sense. With this change, the first RequestVote
sent by the second node is granted and it converges quickly.

Change-Id: If9416ddf7bf0dfc1220a169be4174f440626a0dd
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoAlleviate premature elections in followers 64/42564/7
Tom Pantelis [Tue, 26 Jul 2016 04:07:02 +0000 (00:07 -0400)]
Alleviate premature elections in followers

If a follower actor is busy or some non-leader messages take longer to process,
leader messages may get backed up enough to cause the election timer to
expire, thereby resulting in an unwanted election and leader disruption. To
alleviate this scenario, I added a Stopwatch to keep the last time a leader
message was received, ie when a leader message is received it restarts
the Stopwatch. When ElectionTimeout is received, it checks if the
elapsed time of the Stopwatch has exceeded the election timeout
interval. Therefore if leader messages were occurring during the
election timeout interval but were delayed, they will be processed
before the ElectionTimeout message and restart the Stopwatch such that the
elapsed time will/should be less than the election timeout interval by the
time ElectionTimeout is received (unless the last leader message happened to
take longer than the election timeout interval).

There are cases where ElectionTimeout is manually sent to force an
election timeout (eg during leadership transfer). In these cases we
don't want to check the Stopwatch so I added an explicit TimeoutNow
message to distinguish the 2 messages.

Change-Id: I6b745288040da2fdcef1d29cb5ffc482c9e66003
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBUG-5280: centralize ShardSnapshot operations 85/42785/11
Robert Varga [Fri, 29 Jul 2016 13:58:49 +0000 (15:58 +0200)]
BUG-5280: centralize ShardSnapshot operations

Current shard snapshotting mechanism does not allow for evolution
of the snapshot contents and contains only the root node. Also
the serialization and deserialization operations are scattered
in multiple places, making coordinated changes a bit troublesome.

This patch introduces a versioned snapshot abstraction and moves
serdes operations into a single place. A new serialization format
is introduced, which holds the root node and some additional
metadata. No concrete metadata is defined in this patch, but this
will be used to transfer frontend protocol state from shard leader
to shard follower.

It also moves the act of creating a snapshot into ShardDataTree
and creates a dedicated actor to handle the snapshotting task,
which is used for all snapshot requests for a particular Shard.
Also makes the actor message internal to the ShardSnapshotActor,
providing a convenience method to create and dispatch it.

Change-Id: I6d9680b6ef08672c363092a649255013980c0bd6
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoReduce missing feature warning to debug 01/42801/2
Tom Pantelis [Fri, 29 Jul 2016 19:37:35 +0000 (15:37 -0400)]
Reduce missing feature warning to debug

Many warn messages are emmitted for the "startup" feature:

  Feature: startup, 0.0.0 is missing from features service. Skipping

It seems this is an internal karaf feature and can't be retrieved. This
warning has been seen for other system features in the past. Since the
FeatureConfigPusher is only interested in ODL bundles for the purpose of
pushing CSS modules and since this warning has always been benign, I reduced
the log to debug to avoid the many warnings on startup.

Change-Id: Iad00acccfd0eabd55acb02493e689879e29646f0
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoMove generated sources to target/ directory 64/42864/2
Robert Varga [Sun, 31 Jul 2016 20:42:37 +0000 (22:42 +0200)]
Move generated sources to target/ directory

Generating bindings into src/ is a very bad idea, as that:
- does not conform the expectations
- does not get cleaned by maven
- makes the life for bindings hard (checkstyle, cpd, etc.)

Change-Id: I0d594aa849934c4ac88f8ebb0d4dcc8ea5a4e3e6
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoMake deserializeNormalizedNode more obvious 03/42803/2
Robert Varga [Fri, 29 Jul 2016 19:41:17 +0000 (21:41 +0200)]
Make deserializeNormalizedNode more obvious

Performing a direct return makes the code flow
more obvious, notably the fact that a null node
may only be received in case of new serialization.

Change-Id: Ib10c23d5990ca4452914b7f81647fd67b84863d2
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoAcquire SchemaContext from ShardDataTree 84/42784/3
Robert Varga [Fri, 29 Jul 2016 14:02:10 +0000 (16:02 +0200)]
Acquire SchemaContext from ShardDataTree

Instead of passing the SchemaContext explictly, make
ShardRecoveryCoordinator understand that it can obtain
the SchemaContext from ShardDataTree.

Change-Id: Id5ed521a96e8a741ad7da6199ba117b99a8f78e4
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoFix incorrect readResolve signatures 31/42731/2
Tom Pantelis [Thu, 28 Jul 2016 16:46:28 +0000 (12:46 -0400)]
Fix incorrect readResolve signatures

In several Serializable classes the return type of the readResolve
method is the class. However this is incorrect - the return type must
be Object or it is not recognized by the serialization framework.
Eclipse actually flags the incorrect signature with the "unused" warning
but the warning was suppressed in the code. Using the correct Object
return type, Eclipse doesn't issue the warning.

Change-Id: Id53182925fa48879f1f754c3f25361fb846b23ca
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoCDS Frontend client actor should delete prior snapshots 72/42272/4
Tom Pantelis [Thu, 21 Jul 2016 19:57:35 +0000 (15:57 -0400)]
CDS Frontend client actor should delete prior snapshots

The RecoveringClientActorBehavior increments the last generation id and
saves a new snapshot. However the prior snapshots remain in akka
persistence - every time the controller is restarted a new snapshot file
is created. We should delete the prior snaphsots.

The snapshot file names were very long with escape chars b/c
FrontendIdentifier.toString is used for the frontend client actor's
persistence ID. We should use a shorter, more readable ID, so I changed
it to the form:

  member-1-frontend-datastore-config
  member-1-frontend-datastore-operational

Change-Id: I1c77c826729ca1a36497a1236ac99f7cc77efb72
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoUse ActorSystem.terminate() 68/42668/2
Robert Varga [Thu, 28 Jul 2016 01:53:44 +0000 (03:53 +0200)]
Use ActorSystem.terminate()

ActorSystem.shutdown() has been deprecated, move on
to the replacement call.

Change-Id: I21cee3100c84003585afd9c95706c26f686d0eec
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoRemove src/main/yang as source folder 86/42586/2
Michael Vorburger [Tue, 26 Jul 2016 16:33:14 +0000 (18:33 +0200)]
Remove src/main/yang as source folder

The binding-parent now adds it as a resource folder.

Bug: 6252
Change-Id: I4b4d9b15b95b021f78e70dfb33c6f5287a0f44fe
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
7 years agoConvert AppendEntries and reply to Externalizable proxy 79/42479/3
Tom Pantelis [Sun, 24 Jul 2016 20:59:35 +0000 (16:59 -0400)]
Convert AppendEntries and reply to Externalizable proxy

Converted the AppendEntries and AppendEntriesReply messages to use the
Externalizable proxy pattern. The classes remain Serializable but use an
internal Externalizable Proxy class with writeReplace and readResolve.
This reduces the serialized size to less than half.

Change-Id: Ica1a8ce09458b49b2993d3304ee2d80e38d4fc59
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix test failure in ShardTest 70/42570/2
Tom Pantelis [Tue, 26 Jul 2016 05:22:31 +0000 (01:22 -0400)]
Fix test failure in ShardTest

SHardTest#createSnapshotTest failed on jenkins due to using the
CallingThreadDispatcher, which is the default for test actors, instead
of the default dispatcher.

Change-Id: Id0d8186cb8bda356b81a056f4a0fd2bbecd3c7b4
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoRemove getModuleName()/getInstanceName() in IT 02/42502/2
Alexis de Talhouët [Mon, 25 Jul 2016 20:30:14 +0000 (16:30 -0400)]
Remove getModuleName()/getInstanceName() in IT

Since it was depreacted here https://git.opendaylight.org/gerrit/#/c/39891/

Change-Id: Iaaa6f4b89f4220c2fff01d216a97d785ab4ac3b5
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
7 years agoApply SchemaContext to dataTree first 10/42410/3
Robert Varga [Sun, 24 Jul 2016 19:05:30 +0000 (21:05 +0200)]
Apply SchemaContext to dataTree first

DataTree.setSchemaContext() can fail, hence do not update the schema
context before we propagate it to data tree.

Change-Id: I0170133177ac74280da2ccc367b3c447f9d4cdc9
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBUG-5280: implement transaction dispatch 46/39946/63
Robert Varga [Tue, 7 Jun 2016 12:59:14 +0000 (14:59 +0200)]
BUG-5280: implement transaction dispatch

This patch adds the DOMStore interface in DistributedDataStoreClient
and defines the missing messages.

Change-Id: I6b0905fb97e3269c12a5cd8f2c681e4caeb14e3e
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoRestart BP container after dependency wait time out 20/42320/3
Tom Pantelis [Fri, 22 Jul 2016 04:11:44 +0000 (00:11 -0400)]
Restart BP container after dependency wait time out

The blueprint container first waits for all dependencies (ie OSGi
services, clustered app config etc). By default it waits 5 min after
which it fails the container. For a missing OSGi service this is
probably OK but it could take longer for a clustered-app-config if the
data store isn't available. Ideally we would use an infinite timeout but
unfortunately the timeout can't be configured globally - it can only be set
at the bundle level in the manifest and we don't want to have to
configure it in every bundle (although it may be possible with some maven
magic in odlparent). Therefore I added code in BlueprintBundleTracker to
listen for container FAILURE events and restart the container if it's
due to missing dependencies.

Change-Id: Ib8ebb1a02dfd601e48722f9fc3011df7391432cb
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoBug 6027 - Can't start karaf using symbolic link 82/39982/12
Alexis de Talhouët [Tue, 7 Jun 2016 23:04:15 +0000 (19:04 -0400)]
Bug 6027 - Can't start karaf using symbolic link

When executing the karaf script, it gets the DIRNAME based on $0
which is the path used to start the script. This DIRNAME is then
used to set the KARAF_HOME and multiple other KARAF_* evn variables.

Using a symbolic link, you would have, for instance, usr/bin/karaf
redirecting to /opt/opendaylight/bin/karaf.
So $0 would be usr/bin and not /opt/opendaylight/bin so the locateHome
function isn't setting the right path for the KARAF_HOME.

This ends up failing to start ODL with following ERROR:
Error: Could not find or load main class org.apache.karaf.main.Main

see:
https://github.com/opendaylight/controller/blob/master/karaf/opendaylight-karaf-resources/src/main/resources/bin/karaf#l114l126

Change-Id: I36eff657972768de7d7b90f6563addfc3dd96c0f
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
Co-Authored-By: Michael Vorburger <vorburger@redhat.com>
7 years agoRemove useless .gitignore 34/40934/5
Alexis de Talhouët [Tue, 28 Jun 2016 12:51:08 +0000 (08:51 -0400)]
Remove useless .gitignore

src/main/yang doesn't contain any models, nor nothing.
So it is safe to remove this gitignore

Change-Id: Ib97cb5b1443a02399d8ce3c0c8b56c7a44ee614b
Signed-off-by: Alexis de Talhouët <adetalhouet@inocybe.com>
7 years agoAdd again mdsal-singleton but remove prefix DOM 17/42317/4
Vaclav Demcak [Fri, 22 Jul 2016 10:54:16 +0000 (12:54 +0200)]
Add again mdsal-singleton but remove prefix DOM

We have only one implementation of ClusterSingletonServiceProvider
(DOM implementation) and we'd like to present it without DOM prefix.

* remove DOM prefix from ConfigSubsystem yang
* again add the odl-mdsal-singleton-dom feature
* again add CSSProvider in 06-clustered-entity-ownership.xml file

depends on: https://git.opendaylight.org/gerrit/#/c/42294/

Change-Id: Ieae0d462fe9fa523b2b1b18528759e0614b0225f
Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
7 years agoFix delete snapshots criteria 03/42303/2
Tom Pantelis [Thu, 21 Jul 2016 23:09:47 +0000 (19:09 -0400)]
Fix delete snapshots criteria

When a snapshot is saved, we attempt to delete prior snapshots as only
the last one is recovered from persistence. For the maxSequenceNr in the
criteria, we use snapshot sequenceNr - snapshot batch count. However
this assumes every snapshot is based on snapshot batch count. We may
snapshot for other reasons, eg install snapshot on follower. This can
leave multiple prior snapshots behind and use up significant disk space.

We should only retain the last snapshot saved so I changed the criteria
to use: maxSequenceNr = the saved snapshot sequenceNr (it's possible the
sequenceNr hasn't changed from th elast snapshot) and maxTimeStamp =
saved snapshot timestamp -1.

Change-Id: I35b1d71ed433d52ecff79ca07a81616e393a7b7f
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoAdd mdsal-eos-binding-adapter feature 66/42166/2
Tom Pantelis [Wed, 20 Jul 2016 06:54:59 +0000 (02:54 -0400)]
Add mdsal-eos-binding-adapter feature

Added the feature that contains the binding adapter implementation of
the EntityOwnershipService.

Change-Id: Iccdd99833938b827e69603412385392b98b62abf
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoComment out mdsal-signleton-service in controller 11/42311/1
Vaclav Demcak [Fri, 22 Jul 2016 12:11:44 +0000 (14:11 +0200)]
Comment out mdsal-signleton-service in controller

We'd like to have only one (DOM) ClusterSingletonServiceProvider in ODL.
So we have to commented out all actual reference for CSS DOM API in
controller and we'll able to clean MD-SAL CSS projects for DOM API
and stay with common API only.

So this patch has to be applay before
https://git.opendaylight.org/gerrit/#/c/42294/

Change-Id: I110b4554e4713802b9c261de8b1fd793eabb012a
Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
7 years agoChange default value of parameter "auto-down-unreachable-after" 94/42094/6
Sai MarapaReddy [Tue, 19 Jul 2016 20:49:21 +0000 (13:49 -0700)]
Change default value of parameter "auto-down-unreachable-after"

Akka documentation suggests not using auto-down feature
in production scenario.
Link - http://doc.akka.io/docs/akka/snapshot/java/cluster-usage.html

Change-Id: I24205a34e13c711791186b1e00d5203f623a0478
Signed-off-by: Sai MarapaReddy <sai.marapareddy@gmail.com>
Author: Sai MarapaReddy <sai.marapareddy@gmail.com>

7 years agoRemove global BindingToNormalizedNodeCodec instance 43/40743/7
Tom Pantelis [Wed, 22 Jun 2016 20:01:49 +0000 (16:01 -0400)]
Remove global BindingToNormalizedNodeCodec instance

The BindingToNormalizedNodeCodec was made a global static instance for
backwards compatibility for CSS users that inject the binding-dom-mapping-service
identity which defines the provided service as the concrete
BindingToNormalizedNodeCodec class instead of an interface. Therefore
the global static instance was created via blueprint and advertised via
its interfaces and was obtained via the static reference by the
RuntimeMappingModule for use by CSS users. The RuntimeMappingModule must
return an instance of BindingToNormalizedNodeCodec in order to provide
the binding-dom-mapping-service so obtaining the blueprint advertised OSGi
service via its interfaces and casting to BindingToNormalizedNodeCodec
failed b/c Aries creates a service proxy which loses the fact that it's a
BindingToNormalizedNodeCodec instance.

However the global BindingToNormalizedNodeCodec instance is not clean and
is problematic for supporting blueprint container restarts. Aries supports
concrete class proxies so I added an additional service export for the
BindingToNormalizedNodeCodec class in the binding-broker blueprint XML.
In addition, the blueprint XML now calls a new method, "newInstance", on the
BindingToNormalizedNodeCodecFactory to create a new instance and calls a
new method, "registerInstance", to register it with the SchemaService. The
returned ListenerRegistration instance is put into a bean so it can be
closed on destroy. The RuntimeMappingModule now obtains the
BindingToNormalizedNodeCodec instance from the OSGi registry as the other
blueprint-bridged CSS modules do. This eliminates the need for the global
instance.

Change-Id: I969ad5470967a81b37078393701c69d1898086cd
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoEnsure CSS modules are closed before blueprint containers on shutdown 01/40801/4
Tom Pantelis [Thu, 23 Jun 2016 14:35:07 +0000 (10:35 -0400)]
Ensure CSS modules are closed before blueprint containers on shutdown

Change-Id: I9be36a819423e904030540b161437b6f2ffd091d
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoExtend clustered-app-config to read default data from XML file 02/41902/4
Tom Pantelis [Fri, 15 Jul 2016 15:28:48 +0000 (11:28 -0400)]
Extend clustered-app-config to read default data from XML file

The default data can be specified in the clustered-app-config element
but it's also useful for scripting/automation or convenience to be able
to specify the default data in external XML file. The
clustered-app-config will now look for a file of the form

  <yang module name>_<container name>.xml

in a well-known location, etc/opendaylight/datastore/initial/config.

The XML file name can also be explicitly specified via the
"default-config-file-name" attribute.

Change-Id: Id310ef5ae121b8b9444a2102b93c3e382e421687
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoFix compiler error due to removal of InMemoryDataTreeFactory.create 60/42160/1
Tom Pantelis [Wed, 20 Jul 2016 05:38:12 +0000 (01:38 -0400)]
Fix compiler error due to removal of InMemoryDataTreeFactory.create

Change-Id: I016c1beeb55a438ba56b8076c7e792c79ac51294
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoAdd odl-mdsal-singleton-dom feature and CSS yang 93/42093/2
Tom Pantelis [Tue, 19 Jul 2016 01:30:39 +0000 (21:30 -0400)]
Add odl-mdsal-singleton-dom feature and CSS yang

Added the odl-mdsal-singleton-dom feature to the odl-mdsal-local-broker
feature.

Also added CSS yang and Module class for the
DOMClusterSingletonServiceProvider and added the XML to the existing
06-clustered-entity-ownership.xml file.

Change-Id: I69c7224fd7aa12742778670e7aec53118bf98332
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoConvert distributed EOS impl to use new DOM EOS interfaces 75/35475/14
Tom Pantelis [Fri, 26 Feb 2016 11:01:10 +0000 (06:01 -0500)]
Convert distributed EOS impl to use new DOM EOS interfaces

Change-Id: I5b2a6098a0c15f74ec2f16cb5451f3831ed913bf
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoAdd legacy pre-Boron EntityOwnershipService adapter 42/35442/15
Tom Pantelis [Fri, 26 Feb 2016 04:54:38 +0000 (23:54 -0500)]
Add legacy pre-Boron EntityOwnershipService adapter

Added a class that bridges between the legacy pre-Boron EntityOwnershipService
and DOMEntityOwnershipService interfaces. Also added the config yang and
Module class.

Change-Id: I77d02cb98a7dd5a713269907af7f269171c93fa8
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
7 years agoAdded config model for mdsal.binding.codec. 15/41815/6
Tony Tkacik [Thu, 14 Jul 2016 09:56:44 +0000 (11:56 +0200)]
Added config model for mdsal.binding.codec.

Change-Id: I3ee74461e79c3332a0e8e41afe1d56af4b942a74
Signed-off-by: Tony Tkacik <tony.tkacik@gmail.com>
7 years agoSwitch to StandardCharsets 81/41981/2
Robert Varga [Mon, 18 Jul 2016 14:40:07 +0000 (16:40 +0200)]
Switch to StandardCharsets

Guava's Charsets should not be used when StandardCharsets are
available.

Change-Id: I7c52bd3070bb48857cbba82e8d4bc5993d7aea9d
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoUse mdsal.dom.codec instead of yangtools.data.codec 11/41811/2
Tony Tkacik [Thu, 14 Jul 2016 09:26:24 +0000 (11:26 +0200)]
Use mdsal.dom.codec instead of yangtools.data.codec

Change-Id: I90d7ccd2e9c994305931288fc39f0c990a28a866
Signed-off-by: Tony Tkacik <tony.tkacik@gmail.com>
7 years agoBUG-4167: fall back to unknown module for empty YangInstanceIdentifier 87/41887/2
Robert Varga [Fri, 15 Jul 2016 10:43:58 +0000 (12:43 +0200)]
BUG-4167: fall back to unknown module for empty YangInstanceIdentifier

When we encounter an empyt YangInstanceIdentifier (for
example during listener registration), we cannot extract
a module name -- fallback to unknown, which will cause
us to talk to the default shard.

Change-Id: I2162884c5ce0d2c2f714bb66afd82f699c52d789
Signed-off-by: Robert Varga <rovarga@cisco.com>
7 years agoBug 6186 - fix testCandidateSerialization() 91/41691/5
Isaku Yamahata [Tue, 12 Jul 2016 04:15:53 +0000 (21:15 -0700)]
Bug 6186 - fix testCandidateSerialization()

The changeset of 97ff7dff8e58531065833736d5788808ca9e0396 make
LocalHistoryIdentifier#write() use WritableObjects#writeLongs()
instead of WritableObjects#writeLong(). In some situations, the
header length of object may be shorter.
As a result CommitTransactionPayloadTest#testCandidateSerialization()
failes. This patch fixes it by setting transaction id,
history transaction id to known value when setting up test.

Change-Id: I7fbd912564a25c92bc29f7e10bdae8ce1be52b8f
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
7 years agoReduce ConflictingVersionException log level to debug 47/41847/2
Sai MarapaReddy [Thu, 14 Jul 2016 16:50:11 +0000 (09:50 -0700)]
Reduce ConflictingVersionException log level to debug

In general it happens when there is  a ConflictingVersionException,
it retries and if it times out while retrying, it will log the error

The ConflictingVersionException is similar to the OptimisticLockFailuerEx
in the data store, i.e. the current config version is incremented and
recorded at the start of a push and if a second config is pushed before
 the first completes, the version changes and it detects that and
 re-pushes the first config. The CSS pushes one config at a time
 but this can happen during dependency resolution if it finds a
 dependent module that wasn't created yet or its config changed
 and needs to be dynamically recreated. The dependent module is
 pushed which results in a conflicting version. This
 happens with BGP.

Signed-off-by: Sai MarapaReddy <sai.marapareddy@gmail.com>
Author: Sai MarapaReddy <sai.marapareddy@gmail.com>
Change-Id: Ic1d4639625fa54ccc3d54331a960f421ad6fa1dd

7 years agoSet karaf.delay.console=true in etc/config.properties 62/40262/2
Michael Vorburger [Mon, 13 Jun 2016 23:18:11 +0000 (01:18 +0200)]
Set karaf.delay.console=true in etc/config.properties

$ ./karaf
Apache Karaf starting up. Press Enter to open the shell now...
 78% [=========================================>          ]
(...)
100% [====================================================]

Karaf started in 23s. Bundle stats: 276 active, 276 total

Change-Id: Iad04b7d03aa5be5dc17e28c4d60a56ded8a2f774
Signed-off-by: Michael Vorburger <vorburger@redhat.com>