Split up transaction chunks 53/84553/13
authorRobert Varga <robert.varga@pantheon.tech>
Thu, 19 Sep 2019 23:50:56 +0000 (01:50 +0200)
committerRobert Varga <robert.varga@pantheon.tech>
Mon, 7 Oct 2019 10:44:23 +0000 (12:44 +0200)
commitafe114674227071a2598dd3a3f6589a99573e075
tree164f80e0ddd5beca373f1da87c14b7267b9e1c8d
parent9112031a13f4bc73ea5504da290ddeca0f7d2c17
Split up transaction chunks

This adds the infrastructure to prevent allocation of large byte[]s,
as when those exceed 0.5-16MiB, under G1GC they end up in humongous
object region. Not only that, but after some cut-off point, the
copying of arrays starts to dominate performance.

What we do here is ensure we always receive up to a configurable
number of bytes, defaults to 256KiB, and keep those chunks in a list.
This way we may end up with larger overhead, but that really is
neglibeble -- even a 2GiB payload would end up using only about
8K arrays.

While the input/output streams are similar to
org.apache.commons.io.output.ByteArrayOutputStream, the design here
is geared towards having the intermediate representation available
as well as devolving to a single byte[] for memory efficiency
reasons.

JIRA: CONTROLLER-1920
Change-Id: I2b79a633ebf4fdf8d68d2accc644326e30b41f22
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/persisted/ChunkedByteArray.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/persisted/ChunkedInputStream.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/persisted/ChunkedOutputStream.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/persisted/CommitTransactionPayload.java
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/persisted/ChunkedOutputStreamTest.java [new file with mode: 0644]