Merge pull request diem#202 from zekun000/dip201

[DIP-201] 2-chain DiemBFT

dips/dip-201.md

@@ -0,0 +1,78 @@
+---
+dip: 201
+title: 2-chain DiemBFT
+authors: Zekun Li (@zekun000)
+status: Draft
+type: Standard
+created: 08/09/2020
+---
+
+# Summary
+
+This DIP describes a major latency improvement - 2-chain commit rule instead of 3-chain - as described in  DiemBFT v4.
+
+# Abstract
+
+Currently, DiemBFT inherits the 3-chain commit rules from [HotStuff](https://dl.acm.org/doi/pdf/10.1145/3293611.3331591) for optimistic responsiveness. However, with timeout
+and round synchronization mechanism from DiemBFT, our research shows that we can use a 2-chain commit rule with a minor increase in communication overhead when validators time out of a consensus round.
+We only discuss engineering impact and omit the theoretic part from this DIP, readers are encouraged to read the full white paper.
+
+# Required Changes
+
+## Types
+In order to provide safety under a 2-chain protocol, two new timeout related structs are added to represent the highest quorum certificate the validator node has when it times out of a round.
+
+```rust
+pub struct TwoChainTimeout {
+    /// Epoch number corresponds to the set of validators that are active for this round.
+    epoch: u64,
+    /// The consensus protocol executes proposals (blocks) in rounds, which monotonically increase per epoch.
+    round: Round,
+    /// The highest quorum cert the signer has seen.
+    quorum_cert: QuorumCert,
+}
+pub struct TwoChainTimeoutCertificate {
+    timeout: TwoChainTimeout,
+    signatures: BTreeMap<Author, (Round, Ed25519Signature)>,
+}
+```
+
+TwoChainTimeoutCertificate only carries a single quorum cert instead of 2f+1 as an optimization to save both bandwidth and verification cost.
+It keeps the TwoChainTimeout with the highest quorum cert and the signature is signed on the derived struct
+```rust
+pub struct TimeoutSigningRepr {
+    pub epoch: u64,
+    pub round: Round,
+    pub hqc_round: Round,
+}
+```
+
+## Rules
+The [Safety Rules](https://github.com/diem/diem/tree/main/consensus/safety-rules) code has been changed accordingly and the proofs of safety and liveness are in the white paper.
+
+Voting Rule:
+1. block.round > last vote round
+2. block.round == block.qc.round + 1 || (block.round == tc.round + 1 && block.qc.round >= tc.highest_hqc_round)
+
+Timeout Rule:
+1. round >= last vote round
+2. (round == qc.round + 1 || round == tc.round + 1) && qc.round >= 1-chain round
+
+Commit Rule:
+ - It's safe to commit block A if there exist a 2-chain A <- QC <- B <- QC  such that b.round == a.round + 1.
+
+## Upgrade
+Switching from existing DiemBFT to the newer version requires reconfiguration. An on-chain consensus config will be added to support the transition from 3-chain to 2-chain at an epoch boundary. 
+We also switch from [BCS](https://github.com/diem/bcs) to a backward compatible wire protocol (JSON) to ease the complexity in migrating data structures.
+
+## Testing plan
+All consensus protocol changes must be thoroughly audited, proven, and tested since they are the foundation of safety and liveness of the blockchain. The current plan includes:
+1. Validate protocol and proof 
+2. Enable in all testing environment
+3. Add fault injection
+4. Stress test the reconfiguration switch
+5. Go through regular release pipeline
+
+## Client
+
+This change is client agnostic, other than an observable improvement in commit latency improvement due to that LedgerInfo which encapsulates the commit rule.