Skip to content

Commit

Permalink
cut feature branch for interpreter plugins (MystenLabs#13830)
Browse files Browse the repository at this point in the history
## Description 

Cut feature branch to extend interpreter to support plugins

## Test Plan 

ci
---
If your changes are not user-facing and not a breaking change, you can
skip the following section. Otherwise, please indicate what changed, and
then add to the Release Notes section as highlighted during the release
process.

### Type of Change (Check all that apply)

- [ ] protocol change
- [ ] user-visible impact
- [ ] breaking change for a client SDKs
- [ ] breaking change for FNs (FN binary must upgrade)
- [ ] breaking change for validators or node operators (must upgrade
binaries)
- [ ] breaking change for on-chain data layout
- [ ] necessitate either a data wipe or data migration

### Release notes
  • Loading branch information
wlmyng authored Sep 21, 2023
1 parent 4887749 commit d5a54b1
Show file tree
Hide file tree
Showing 214 changed files with 50,767 additions and 6 deletions.
7 changes: 7 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,13 @@ jobs:
- name: rustdoc
run: |
cargo doc --workspace --no-deps
- name: Install cargo-hakari, and cache the binary
uses: baptiste0928/cargo-install@30f432979e99f3ea66a8fa2eede53c07063995d8
with:
crate: cargo-hakari
locked: true
- name: Install rustfmt
run: rustup component add rustfmt
- name: sui-execution
run: |
./scripts/execution_layer.py generate-lib
Expand Down
121 changes: 121 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ exclude = [
"external-crates/move-execution/v0/move-bytecode-verifier",
"external-crates/move-execution/v0/move-stdlib",
"external-crates/move-execution/v0/move-vm/runtime",
"external-crates/move-execution/vm-rework/move-bytecode-verifier",
"external-crates/move-execution/vm-rework/move-stdlib",
"external-crates/move-execution/vm-rework/move-vm/runtime",
"external-crates/move/move-abstract-stack",
"external-crates/move/move-binary-format",
"external-crates/move/move-binary-format/serializer-tests",
Expand Down Expand Up @@ -162,6 +165,9 @@ members = [
"sui-execution/v0/sui-adapter",
"sui-execution/v0/sui-move-natives",
"sui-execution/v0/sui-verifier",
"sui-execution/vm-rework/sui-adapter",
"sui-execution/vm-rework/sui-move-natives",
"sui-execution/vm-rework/sui-verifier",
]

[workspace.package]
Expand Down
6 changes: 6 additions & 0 deletions crates/workspace-hack/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,7 @@ move-bytecode-source-map = { path = "../../external-crates/move/move-ir-compiler
move-bytecode-utils = { path = "../../external-crates/move/tools/move-bytecode-utils", default-features = false }
move-bytecode-verifier = { path = "../../external-crates/move/move-bytecode-verifier" }
move-bytecode-verifier-v0 = { path = "../../external-crates/move-execution/v0/move-bytecode-verifier" }
move-bytecode-verifier-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-bytecode-verifier" }
move-bytecode-viewer = { path = "../../external-crates/move/tools/move-bytecode-viewer" }
move-cli = { path = "../../external-crates/move/tools/move-cli", default-features = false }
move-command-line-common = { path = "../../external-crates/move/move-command-line-common", default-features = false }
Expand All @@ -415,13 +416,15 @@ move-stackless-bytecode = { path = "../../external-crates/move/move-prover/bytec
move-stackless-bytecode-interpreter = { path = "../../external-crates/move/move-prover/interpreter", default-features = false }
move-stdlib = { path = "../../external-crates/move/move-stdlib", default-features = false, features = ["testing"] }
move-stdlib-v0 = { path = "../../external-crates/move-execution/v0/move-stdlib", default-features = false }
move-stdlib-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-stdlib", default-features = false }
move-symbol-pool = { path = "../../external-crates/move/move-symbol-pool" }
move-transactional-test-runner = { path = "../../external-crates/move/testing-infra/transactional-test-runner", default-features = false }
move-unit-test = { path = "../../external-crates/move/tools/move-unit-test", default-features = false }
move-vm-config = { path = "../../external-crates/move/move-vm/config", default-features = false }
move-vm-profiler = { path = "../../external-crates/move/move-vm/profiler", default-features = false }
move-vm-runtime = { path = "../../external-crates/move/move-vm/runtime", features = ["debugging", "testing"] }
move-vm-runtime-v0 = { path = "../../external-crates/move-execution/v0/move-vm/runtime" }
move-vm-runtime-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-vm/runtime" }
move-vm-test-utils = { path = "../../external-crates/move/move-vm/test-utils", features = ["tiered-gas"] }
move-vm-types = { path = "../../external-crates/move/move-vm/types" }
multer = { version = "2" }
Expand Down Expand Up @@ -1173,6 +1176,7 @@ move-bytecode-source-map = { path = "../../external-crates/move/move-ir-compiler
move-bytecode-utils = { path = "../../external-crates/move/tools/move-bytecode-utils", default-features = false }
move-bytecode-verifier = { path = "../../external-crates/move/move-bytecode-verifier" }
move-bytecode-verifier-v0 = { path = "../../external-crates/move-execution/v0/move-bytecode-verifier" }
move-bytecode-verifier-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-bytecode-verifier" }
move-bytecode-viewer = { path = "../../external-crates/move/tools/move-bytecode-viewer" }
move-cli = { path = "../../external-crates/move/tools/move-cli", default-features = false }
move-command-line-common = { path = "../../external-crates/move/move-command-line-common", default-features = false }
Expand All @@ -1197,13 +1201,15 @@ move-stackless-bytecode = { path = "../../external-crates/move/move-prover/bytec
move-stackless-bytecode-interpreter = { path = "../../external-crates/move/move-prover/interpreter", default-features = false }
move-stdlib = { path = "../../external-crates/move/move-stdlib", default-features = false, features = ["testing"] }
move-stdlib-v0 = { path = "../../external-crates/move-execution/v0/move-stdlib", default-features = false }
move-stdlib-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-stdlib", default-features = false }
move-symbol-pool = { path = "../../external-crates/move/move-symbol-pool" }
move-transactional-test-runner = { path = "../../external-crates/move/testing-infra/transactional-test-runner", default-features = false }
move-unit-test = { path = "../../external-crates/move/tools/move-unit-test", default-features = false }
move-vm-config = { path = "../../external-crates/move/move-vm/config", default-features = false }
move-vm-profiler = { path = "../../external-crates/move/move-vm/profiler", default-features = false }
move-vm-runtime = { path = "../../external-crates/move/move-vm/runtime", features = ["debugging", "testing"] }
move-vm-runtime-v0 = { path = "../../external-crates/move-execution/v0/move-vm/runtime" }
move-vm-runtime-vm-rework = { path = "../../external-crates/move-execution/vm-rework/move-vm/runtime" }
move-vm-test-utils = { path = "../../external-crates/move/move-vm/test-utils", features = ["tiered-gas"] }
move-vm-types = { path = "../../external-crates/move/move-vm/types" }
multer = { version = "2" }
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[package]
name = "move-bytecode-verifier-vm-rework"
version = "0.1.0"
authors = ["Diem Association <[email protected]>"]
description = "Move bytecode verifier"
repository = "https://github.com/diem/diem"
homepage = "https://diem.com"
license = "Apache-2.0"
publish = false
edition = "2021"

[dependencies]
petgraph = "0.5.1"

move-borrow-graph = { path = "../../../move/move-borrow-graph" }
move-binary-format = { path = "../../../move/move-binary-format" }
move-core-types = { path = "../../../move/move-core/types" }
move-vm-config = { path = "../../../move/move-vm/config" }
move-abstract-stack = { path = "../../../move/move-abstract-stack" }

[dev-dependencies]
hex-literal = "0.3.4"
invalid-mutations = { path = "../../../move/move-bytecode-verifier/invalid-mutations" }

[features]
default = []
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
id: bytecode-verifier
title: Bytecode Verifier
custom_edit_url: https://github.com/move-language/move/edit/main/language/move-bytecode-verifier/README.md
---


## Overview

The bytecode verifier contains a static analysis tool for rejecting invalid Move bytecode. It checks the safety of stack usage, types, resources, and references.

The body of each function in a compiled module is verified separately while trusting the correctness of function signatures in the module. Checking that each function signature matches its definition is a separate responsibility. The body of a function is a sequence of bytecode instructions. This instruction sequence is checked in several phases described below.

## CFG Construction

A control-flow graph is constructed by decomposing the instruction sequence into a collection of basic blocks. Each basic block contains a contiguous sequence of instructions; the set of all instructions is partitioned among the blocks. Each block ends with a branch or return instruction. The decomposition into blocks guarantees that branch targets land only at the beginning of some block. The decomposition also attempts to ensure that the generated blocks are maximal. However, the soundness of the analysis does not depend on maximality.

## Stack Safety

The execution of a block happens in the context of a stack and an array of local variables. The parameters of the function are a prefix of the array of local variables. Arguments and return values are passed across function calls via the stack. When a function starts executing, its arguments are already loaded into its parameters. Suppose the stack height is *n* when a function starts executing; then valid bytecode must enforce the invariant that when execution lands at the beginning of a basic block, the stack height is *n*. Furthermore, at a return instruction, the stack height must be *n*+*k* where *k*, s.t. *k*>=0 is the number of return values. The first phase of the analysis checks that this invariant is maintained by analyzing each block separately, calculating the effect of each instruction in the block on the stack height, checking that the height does not go below *n*, and that is left either at *n* or *n*+*k* (depending on the final instruction of the block and the return type of the function) at the end of the block.

## Type Safety

The second phase of the analysis checks that each operation, primitive or defined function, is invoked with arguments of appropriate types. The operands of an operation are values located either in a local variable or on the stack. The types of local variables of a function are already provided in the bytecode. However, the types of stack values are inferred. This inference and the type checking of each operation can be done separately for each block. Since the stack height at the beginning of each block is *n* and does not go below *n* during the execution of the block, we only need to model the suffix of the stack starting at *n* for type checking the block instructions. We model this suffix using a stack of types on which types are pushed and popped as the instruction stream in a block is processed. Only the type stack and the statically-known types of local variables are needed to type check each instruction.

## Resource Safety

Resources represent the assets of the blockchain. As such, there are certain restrictions on these types that do not apply to normal values. Intuitively, resource values cannot be copied and must be used by the end of the transaction (this means that they are moved to global storage or destroyed). Concretely, the following restrictions apply:

* `CopyLoc` and `StLoc` require that the type of local is not of resource kind.
* `WriteRef`, `Eq`, and `Neq` require that the type of the reference is not of resource kind.
* At the end of a function (when `Ret` is reached), no local whose type is of resource kind must be empty, i.e., the value must have been moved out of the local.

As mentioned above, this last rule around `Ret` implies that the resource *must* have been either:

* Moved to global storage via `MoveTo`.
* Destroyed via `Unpack`.

Both `MoveTo` and `Unpack` are internal to the module in which the resource is declared.

## Reference Safety

References are first-class in the bytecode language. Fresh references become available to a function in several ways:

* Inputing parameters.
* Taking the address of the value in a local variable.
* Taking the address of the globally published value in an address.
* Taking the address of a field from a reference to the containing struct.
* Returning value from a function.

The goal of reference safety checking is to ensure that there are no dangling references. Here are some examples of dangling references:

* Local variable `y` contains a reference to the value in a local variable `x`; `x` is then moved.
* Local variable `y` contains a reference to the value in a local variable `x`; `x` is then bound to a new value.
* Reference is taken to a local variable that has not been initialized.
* Reference to a value in a local variable is returned from a function.
* Reference `r` is taken to a globally published value `v`; `v` is then unpublished.

References can be either exclusive or shared; the latter allow read-only access. A secondary goal of reference safety checking is to ensure that in the execution context of the bytecode program, including the entire evaluation stack and all function frames, if there are two distinct storage locations containing references `r1` and `r2` such that `r2` extends `r1`, then both of the following conditions hold:

* If `r1` is tagged as exclusive, then it must be inactive, i.e. it is impossible to reach a control location where `r1` is dereferenced or mutated.
* If `r1` is shared, then `r2` is shared.

The two conditions above establish the property of referential transparency, important for scalable program verification, which looks roughly as follows: consider the piece of code `v1 = *r; S; v2 = *r`, where `S` is an arbitrary computation that does not perform any write through the syntactic reference `r` (and no writes to any `r'` that extends `r`). Then `v1 == v2`.

### Analysis Setup

The reference safety analysis is set up as a flow analysis (or abstract interpretation). An abstract state is defined for abstractly executing the code of a basic block. A map is maintained from basic blocks to abstract states. Given an abstract state *S* at the beginning of a basic block *B*, the abstract execution of *B* results in state *S'*. This state *S'* is propagated to all successors of *B* and recorded in the map. If a state already existed for a block, the freshly propagated state is “joined” with the existing state. If the join fails an error is reported. If the join succeeds but the abstract state remains unchanged, no further propagation is done. Otherwise, the state is updated and propagated again through the block. An error may also be reported when an instruction is processed during the propagation of abstract state through a block.

**Errors.** As mentioned earlier, an error is reported by the checker in one of the following situations:

* An instruction cannot be proven to be safe during the propagation of the abstract state through a block.
* Join of abstract states propagated via different incoming edges into a block fails.

## How is this module organized?

```text
*
├── invalid-mutations # Library used by proptests
├── src # Core bytecode verifier files
├── tests # Proptests
```
Loading

0 comments on commit d5a54b1

Please sign in to comment.