x/blob

Abstract

The x/blob module enables users to pay for arbitrary data to be published to the Celestia blockchain. Users create a single BlobTx that is composed of:

1. Multiple Blobs (Binary Large OBjects): the data they wish to publish. A single Blob is composed of:
   1. NamespaceId []byte: the namespace this blob should be published to.
   2. Data []byte: the data to be published.
   3. ShareVersion uint32: the version of the share format used to encode this blob into a share.
2. A single sdk.Tx which encapsulates a MsgPayForBlobs message that is composed of:
   1. Signer string: the transaction signer
   2. NamespaceIds []byte: the namespaces they wish to publish each blob to. The namespaces here must match the namespaces in the Blobs.
   3. ShareCommitments [][]byte: a share commitment that is the root of a Merkle tree where the leaves are share commitments to each blob associated with this BlobTx.

After the BlobTx is submitted to the network, a block producer separates the transaction i.e., sdk.Tx from the blob. Both components get included in the data square in different namespaces: the sdk.Tx of the original BlobTx together with some metadata about the separated blobs get included in the PayForBlobNamespace (one of the reserved namespaces) and the associated blob gets included in the namespace the user specified in the original BlobTx. Further reading: Data Square Layout

After a block has been created, the user can verify that their data was included in a block via a blob inclusion proof. A blob inclusion proof uses the ShareCommitment in the original sdk.Tx transaction and subtree roots of the block's data square to prove to the user that the shares that compose their original data do in fact exist in a particular block.

TODO: link to blob inclusion (and fraud) proof

State

The blob module doesn't maintain it's own state outside of two params. Meaning that the blob module only uses the params and auth module stores.

Params

// Params defines the parameters for the module.
message Params {
  option (gogoproto.goproto_stringer) = false;
  uint32 gas_per_blob_byte = 1
      [ (gogoproto.moretags) = "yaml:\"gas_per_blob_byte\"" ];
  uint64 gov_max_square_size = 2
      [ (gogoproto.moretags) = "yaml:\"gov_max_square_size\"" ];
}

GasPerBlobByte

GasPerBlobByte is the amount of gas that is consumed per byte of blob data when a MsgPayForBlobs is processed. Currently, the default value is 8. This value is set below that of normal transaction gas consumption, which is 10.

GovMaxSquareSize

GovMaxSquareSize is the maximum size of a data square that is considered valid by the validator set. This value is superseded by the MaxSquareSize, which is hardcoded and cannot change without hardforking the chain. See ADR021 for more details.

Messages

• MsgPayForBlobs pays for a set of blobs to be included in the block. Blob transactions that contain this sdk.Msg are also referred to as "PFBs".

message MsgPayForBlobs {
  string signer = 1;
  repeated bytes namespaces = 2;
  repeated uint32 blob_sizes = 3;
  repeated bytes share_commitments = 4;
  repeated uint32 share_versions = 8;
}

MsgPayForBlobs pays for the inclusion of blobs in the block and consists of the following fields:

• signer: bech32 encoded signer address
• namespace: namespace is a byte slice of length 29 where the first byte is the namespaceVersion and the subsequent 28 bytes are the namespaceId.
• blob_sizes: sizes of each blob in bytes.
• share_commitments is a list of share commitments (one per blob).
• share_versions are the versions of the share format that the blobs associated with this message should use when included in a block. The share_versions specified must match the share_versions used to generate the share_commitment in this message. See ADR007 for more details on how this effects the share encoding and when it is updated.

Note that while the shares version in each protobuf encoded PFB are uint32s, the internal represantation of shares versions is always uint8s. This is because protobuf doesn't support uint8s.

Generating the ShareCommitment

The share commitment is the commitment to share encoded blobs. It can be used for cheap inclusion checks for some data by light clients. More information and rational can be found in the data square layout specs.

1. Split the blob into shares of size shareSize
2. Determine the SubtreeWidth by dividing the length in shares by the SubtreeRootThreshold.
3. Generate each subtree root by diving the blob shares into SubtreeWidth sized sets, then take the binary namespaced merkle tree (NMT) root of each set of shares.
4. Calculate the final share commitment by taking the merkle root (note: not an NMT, just a normal binary merkle root) of the subtree roots from the previous step.

See CreateCommitment for an implementation. See data square layout and ADR013 for details on the rational of the square layout.

Validity Rules

In order for a proposal block to be considered valid, each BlobTx, and thus each PFB, to be included in a block must follow a set of validity rules.

1. Signatures: All blob transactions must have valid signatures. This is state-dependent because correct signatures require using the correct sequence number(aka nonce).
2. Single SDK.Msg: There must be only a single sdk.Msg encoded in the sdk.Tx field of the blob transaction BlobTx.
3. Namespace Validity: The namespace of each blob in a blob transaction BlobTx must be valid. This validity is determined by the following sub-rules:
   1. The namepsace of each blob must match the respective (same index) namespace in the MsgPayForBlobs sdk.Msg field namespaces.
   2. The namespace is not reserved for protocol use.
4. Blob Size: No blob can have a size of 0.
5. Blob Count: There must be one or more blobs included in the transaction.
6. Share Commitment Validity: Each share commitment must be valid.
   1. The size of each of the share commitments must be equal to the digest of the hash function used (sha256 so 32 bytes).
   2. The share commitment must be calculated using the steps specified above in Generating the Share Commitment
7. Share Versions: The versions of the shares must be supported.
8. Signer Address: The signer address must be a valid Celestia address.
9. Proper Encoding: The blob transactions must be properly encoded.
10. Size Consistency: The sizes included in the PFB field blob_sizes, and each must match the actual size of the respective (same index) blob in bytes.

IndexWrappedTx

When a block producer is preparing a block, they must perform an extra step for BlobTxs so that end-users can find the blob shares relevant to their submitted BlobTx. In particular, block proposers wrap the BlobTx in the PFB namespace with the index of the first share of the blob in the data square. See Blob share commitment rules for more details.

Since BlobTxs can contain multiple blobs, the sdk.Tx portion of the BlobTx is wrapped with one share index per blob in the transaction. The index wrapped transaction is called an IndexWrapper and this is the struct that gets marshalled and written to the PayForBlobNamespace.

Events

The blob module emits the following events:

Blob Events

EventPayForBlobs

Attribute Key	Attribute Value
signer	{bech32 encoded signer address}
blob_sizes	{sizes of blobs in bytes}
namespaces	{namespaces the blobs should be published to}

Parameters

Key	Type	Default
GasPerBlobByte	uint32	8

Usage

celestia-app tx blob PayForBlobs <hex encoded namespace> <hex encoded data> [flags]

For submitting PFB transaction via a light client's rpc, see celestia-node's documention.

The steps in the SubmitPayForBlobs function can be reverse engineered to submit blobs programmatically.