with FrontendIdentifier, it also uniquely identifies any transaction that has
ever executed in the cluster, e.g. forming a GlobalTransactionIdentifier.
-== Frontend persistence
+== Persistence model
+
+=== Frontend persistence
As noted above, implementation of the Frontend Generation requires each member
to persistently store a single 64 bit counter. To keep the persistence interfaces
within a member consistent, Frontend uses Akka Persistence to maintain this
it being completed. If an inconsistency occurs (due to messages getting lost
or Shard Leader failing), the Frontend will retransmit this state to the Backend.
-== Backend persistence
+=== Backend persistence
The Backend needs to persist and replicate enough state to ensure that Shard
Leader movement does not render Local Histories inoperable, as that has
a direct impact on applications, which need to have a state reconciliation
by the application and enough metadata about Local Histories and Transactions
to make state recovery across shard leadership movement possible.
-=== User Data persistence
+==== User Data persistence
The Snapshot needs to identify all transactions which have been integrated
into it. Enumerating all of them is clearly not practical, hence we exploit
the monotonic nature of identifiers' relationship to Global History. For each
delta carried in that entry. This information is used to summarize transaction
metadata when this entry is snapshotted.
-=== Internal state persistence
+==== Internal state persistence
The Backend creates a hierarchy of actors to track interactions with the Frontend.
Information necessary to recreate these actors needs to be persisted within
the journal. This boils down to existence of Local Histories and the last
until the actor transitions to Shard Leader state. It only tracks the information
required.
+== Actor model
+ Both Backend and Frontend are implemented as a set of cooperating actors. This section
+ outlines the structure and how it relates to user application interactions.
+
+=== Frontend actor model
+ The current implementation does not have explicit actors, which means that every
+ interaction with the Backend is done via Patterns.ask(), which creates implicit
+ actors. This model proposes to change that, making the model explicit -- allowing
+ us to better track state of interactions.
+
+==== FrontendActor
+ FrontendActor acts as the single, root-level actor. It is tied to
+ a ConcurrentDataBroker instance. Its resposibility is to maintain the generation
+ number (hence it is a PersistentActor), maintain information about individual
+ ShardLeader locations and direct this information to its children. It maintains
+ a set of LocalHistoryActors and a set of SingleTransactionActors.
+
+ Its responsibility is to talk to ShardManager and propagate information about
+ Shard Leader location towards its child actors.
+
+==== LocalHistoryActor
+ A LocalHistoryActor is tied to a DOMTransactionChain, maintains all local state
+ to it and also tracks its propagation to the Shard Leader. This is not a persistent
+ actor.
+
+ It maintains a relationship to its Backend counterpart and routes operations
+ (remote) and transactions (local) towards it.
+
+ Requests sent from the frontend to the local history actor are subjects to timeouts.
+ If a timeout occurs, the frontend is expected to perform full backend leader
+ discovery, contact the leader and reconcile transaction state with it.
+
+ In order to minimize messaging latency while maintaining resiliency to message
+ loss (due to TCP connectivity issues), there are two messaging patterns:
+
+ -- Frontend talks to the Shard. This is used during instantiation of a local
+ history, as well as state reconciliation after a retriable failure has
+ occurred (such as a request timeout).
+
+ -- Frontend talks to the Local History Actor. This is used during transaction
+ operation, e.g. when a transaction is being built up. Once the transaction
+ is readied, the LHA will send a message to the Shard, which takes ownership
+ of that transaction's state. Since this handoff is not observable by
+ the frontend (which would require additional round-trip), the Shard
+ replicates its responses to the LHA. In case a timeout occurs, the frontend
+ will contact the LHA (as that's the last endpoint it knows about), which
+ will replay the Shard message.
+
+ Messaging between a Local History Actor and its associated Shard is expected
+ to be reliable. This is guaranteed by the parent/child relationship, where
+ Shard is the parent. If a Shard fails, all of its children will be restarted,
+ losing all state. If an LHA fails, Shard will be notified with Akka DeathWatch
+ pattern.
+
+
+==== SingleTransactionActor
+ A SingleTransactionActor takes care of transactions which are not tied to a
+ LocalHistory, e.g. single transactions instatiated via
+ DOMDataBroker.newXXXTransaction()). These actors instantiated for each transaction
+ separately and get terminated once the transaction is completed, successfully or not.
+
+
Code Review / controller.git / blobdiff