Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fft semantics change at start #420

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

bxue-l2
Copy link
Contributor

@bxue-l2 bxue-l2 commented Mar 30, 2024

Why are these changes needed?

This PR makes EigenDA blob semantics compatible with 4844 spec. As the result, DA user (rollup) has a consistent upload format to EigenDA as to Ethereum.

There are 3 changes in this PR:

  • Add IFFT inside the api server of disperser, after rate limit has been checked. The transformed data are serialized as bytes and feed to shared blob store (S3). This approach treats the rest of EigenDA code as a black box, therefore introduce a minimal changes to the core.
  • Add a new interface to EigenDA encoder called "EncodeAndProveSymbols". The new interface is need for improving the coding efficiency by 2, whenever a rollup wants to use 4844 formats.
  • Add DecodeAsEval option as the default to the rs/Decoder, it takes the recovered data and additionally performs FFT to convert to evaluation format.

After the PR, all bytes are interpreted every 32 bytes.

See diagram delineates the changes.

Screenshot 2024-03-29 at 9 50 04 PM

Because 4844 requires taking IFFT on the data, every field element from the output of IFFT needs to use all 254 bits of information, and taking 32 bytes to store. Suppose it produces 1024 field elements. Because the old encoder interface of encoder treats every 31 bytes as a field element (Symbol). From EigenDA encoder's perspective, there are 1024*32/31 field elements, 1058. Some specific configuration of coding ratio and stake distribution leads to another 2x inefficiency, due to 31/32 bytes conversion. The 32/31 encoding also leads to a more confusing decoding process.

Most new codes are introduced at api server and decoder. There are many changes in the test code, most of them are converting the original data to the IFFT transformed data to test again the rs.Decode.

Checks

  • I've made sure the lint is passing in this PR.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
  • Testing Strategy
    • Unit tests
    • Integration tests
    • This PR is not tested :(

@bxue-l2 bxue-l2 requested review from jianoaix, mooselumph and ian-shim and removed request for jianoaix and mooselumph March 30, 2024 23:12
@bxue-l2 bxue-l2 marked this pull request as ready for review March 30, 2024 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant