Bump upstreams Adopt: - odlparent-13.0.1 - yangtools-11.0.0-SNAPSHOT - mdsal-12-0.0-SNAPSHOT Since the format of NormalizedNode tree is updated to not include AugmentationNode (and AugmentationIdentifier), we must force an incompatible update, as we do not have enough information to produce compatible payload. Change-Id: Ibf444540ed052b49651ba66ca7b93c806a6c8a97 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Switch to use PayloadVersion.CHLORINE_SR2 PayloadVersion.CHLORINE_SR2 is more efficient format, switch to using it and deprecate old versions for removal. JIRA: CONTROLLER-2056 Change-Id: Id05a34a28e4d3e817fc6ff783669493ee80af3b7 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Define PayloadVersion.CHLORINE_SR2 Add the definition of PayloadVersion.CHLORINE_SR2. This is backwards compatiblein most senses. JIRA: CONTROLLER-2056 Change-Id: Ia39ccf117ec1a0eb0edf909fbaea903f13c185f3 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Remove support for payloads older than Sodium SR1 Older payloads are not as efficient, nor can they really support all our datatypes. Prune supported versions to Sodium SR1-and-newer, retaining 2 years worth of backwards compatibility. JIRA: CONTROLLER-1985 Change-Id: I5ebabee1ddff37af7c0e0a5dca91db433ca9d5ac Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Bump odlparent/yangtools/mdsal Adopt latest versions, namely; - odlparent-9.0.1 - yangtools-7.0.1 - mdsal-8.0.0-SNAPSHOT There are a few adjustments needed, which mostly deal with the interface to NormalizedNode. Change-Id: I918fb885a6df62e16e17119a7e04ba1672ef7c39 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Switch current {ABI,DataStore,Payload}Version to Magnesium This mass-switches versioned resources to use Magnesium streaming format. It also updates the hand-written test data to use Uint64 instead of BigInteger, as Magnesium does not support BigIntegers. JIRA: CONTROLLER-1919 Change-Id: I4734fe3fe0064809fde899ea127e98380c8aacf4 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Switch current {ABI,DataStore,Payload}Version to Sodium SR1 This mass-switches versioned resources to use Sodium SR1 streaming format. JIRA: CONTROLLER-1919 Change-Id: Ic712fa04e16c68481eeecccc61d8e6a7dfabac52 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Define PayloadVersion.MAGNESIUM This adds the PayloadVersion which uses Magnesium normalized node stream. JIRA: CONTROLLER-1919 Change-Id: Ie43e9932a19bd19c92f14890da13646bf56e2f6f Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Define PayloadVersion.SODIUM_SR1 This adds the PayloadVersion which uses Sodium SR1 normalized node stream. JIRA: CONTROLLER-1919 Change-Id: I3dfa6ea053ea356bb68b458f3d6787904b2904c0 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Rename SODIUM versions to NEON_SR2 We are shipping these improvements in Neon SR2, which means we should reserve this version not as Sodium. Change-Id: If5caa0dd7510c6e1ffe3c483c74c456798ca6bb3 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Bump persisted PayloadVersion Since the NormalizedNodeStream format has changed, shard persisted state is affected. This patch bumps PayloadVersion and applies that bump to snapshots and CommitTransactionPayload. On recovery, a snapshot's need to migrate is reflected in its state, and it is examined just as MigratedSerializables are. CommitTransactionPayload is not examined, as understanding its stream version would require deserializing at least its header. JIRA: CONTROLLER-1888 Change-Id: I678527be4487ee1729123ba8b9dcd2269e6cf262 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Migrate nullness annotations This mass-migrates sal-distributed-datastore to use JDT nullness annotations instead of JSR305. Change-Id: I2cd739e3db8c074a55e335e17297a23065775d6a Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Remove deprecated Snapshot and related code Carbon will create a new snapshot when it encounters a pre-Carbon Snapshot so we can remove the pre-Carbon Snapshot and related code. Change-Id: Iae5f140aadb458eaa59ea4cc8be6054bbde090e4 Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
Bug 7521: Convert DatastoreSnapshot.ShardSnapshot to store Snapshot Converted the new DatastoreSnapshot.ShardSnapshot class to store a Snapshot instance instead of a serialized byte[] so the Snapshot can be serialized directly to the file when saved. The prior DatastoreSnapshotList was deprecated and readResolves to the new DatastoreSnapshotList. Change-Id: I656eca93bfed07ae99055c67cc3f195b25ea2b11 Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
sal-distributed-datastore Checkstyle fixes (for next version) Even though sal-distributed-datastore currentla already has CS enforcement, these violations don't show up in the build, but I see them as red in Eclipse; I think this must be becaues the CS version used in the Eclipse plugin-in is more recent than the CS used in odlparent, and one of the existing activated checks got a little more stringent. Cleaning this up thus helps both to (a) not have read in Eclipse; (b) pave the way to upgrade Checkstyle in odlparent some day. Change-Id: Ib5649a95a1b26b5791f2c3f3f83924b569f965a2 Signed-off-by: Michael Vorburger <vorburger@redhat.com>
Bug 7521: Convert Snapshot to store a State instance Created a new Snapshot class that stores the state as a Serializable State instance instead of a byte[] to lay the groundwork for handling snapshots > 2G. In this manner, State implementations can serialize their data directly to the ObjectOutputStream. The previous Snapshot class was deprecated but, unfortunately, it can't simply "read resolve" to the new class on de-serialization as the deserialization code doesn't know how to convert the data from byte[] to a State instance. Therefore a new method, deserializePreCarbonSnapshot, was added to RaftActorRecoveryCohort. The introduction of the State interface necessitated changes throughout the snapshot capture and snapshot recovery code paths. The Shard implementations were also changed accordingly. For follower snapshot install, an optional OutputStream is also passed to createSnapshot. The new API contract is: if the OutputStream is present, the implementation must serialize its state to the OutputStream and return the stream instance in the CaptureSnapshotReply along with the snapshot State instance. The snapshot State is serialized directly to the snapshot store while the OutputStream is used to send the state data to follower(s) in chunks. The deserializeSnapshot method is used to convert the serialized data back to a State instance on the follower end. The serialization for snapshot install is passed off so the cost of serialization is not charged to the raft actor's thread. The OutputStream is converted to a ByteSource when passed to AbstractLeader. AbstractLeader still converts it to a byte[] but will be changed to use the ByteSource's InputStream to chunk the data in a subsequent patch. Also the SnapshotManager currently passes a ByteArrayOutputStream to createSnapshot but this will be changed to a FileBackedOutputStream in a subsequent patch. Change-Id: I2ea30870b54478d7ef5669335ca4b444539f8d56 Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
Fix warnings/javadocs in sal-distributed-datastore First off, I apologize for the size of this patch. There's a ton of classes in this project and I didn't even get to all of them (will follow-up). While a lot of files were touched, the changes were mostly small. Fixed a lot of checkstyle warnings and cleaned up javadocs. Most of the warnings/changes were for: - white space before if/for/while/catch - white space before beginning brace - line too long - illegal catching of Exception (suppressed) - variable name too short - indentation - missing period after first sentence in javadoc - missing first sentence in javadoc - missing <p/> in javadoc Change-Id: Id56d874a8fbcbbc9285279a71c0a5aba393653a9 Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
BUG-5280: centralize ShardSnapshot operations Current shard snapshotting mechanism does not allow for evolution of the snapshot contents and contains only the root node. Also the serialization and deserialization operations are scattered in multiple places, making coordinated changes a bit troublesome. This patch introduces a versioned snapshot abstraction and moves serdes operations into a single place. A new serialization format is introduced, which holds the root node and some additional metadata. No concrete metadata is defined in this patch, but this will be used to transfer frontend protocol state from shard leader to shard follower. It also moves the act of creating a snapshot into ShardDataTree and creates a dedicated actor to handle the snapshotting task, which is used for all snapshot requests for a particular Shard. Also makes the actor message internal to the ShardSnapshotActor, providing a convenience method to create and dispatch it. Change-Id: I6d9680b6ef08672c363092a649255013980c0bd6 Signed-off-by: Robert Varga <rovarga@cisco.com>