feat: support for public ports (0xPolygon#325)

* feat: agglayer static port * chore: disable static ports by default and add ci test * fix: ci * ci: update job names * chore: rename file * feat: cdk node static port * chore: nit * fix: typos * fix: typo * feat: bridge static ports * feat: db static port * chore: lint * fix: typos * feat: more static ports * feat: add ethereum pkg static ports * fix: typo * fix: eth params * fix: eth pkg config * test * chore: nit * test * fix: typo * fix: dac typo * chore: update static ports * feat: add another flag to use default public ports * chore: update custom public ports * fix: typo * chore: clean up ports lib * fix: typos * fix: default public ports * chore: nit * chore: clean up * chore: clean up 2 * ci: rename job * feat: agglayer and agglayer prover public ports * feat: zkevm provers public port * refactor: cdk erigon * fix * fix * fix fix fix * feat: cdk-erigon public port * fix * fix: pless db * chore: refactor default public ports * ci: update custom public port test * refactor and rename cdk-erigon-node to cdk-erigon-rpc * ci: verify public ports if specified * fix: pless zkevm node * fix * fix * ci: fix * ci: nit * ci: fix * ci: nit * ci: debug * ci: fix * docs: document the public port feature * docs: improve public ports doc * chore: sort port configs to make sure public ports are deterministic * ci: last fix attempt * ci: fix issue * docs: mention to bump the docker engine memory limit if deployment is stuck * chore: merge default l1 and l2 public ports into one struct * chore: rename `public_ports` to `static_ports` and `use_default_public_ports` to `use_dynamic_ports` * ci: only test the default static port config * fix: typo * fix: ethereum pkg static ports
doutv · Nov 4, 2024 · 4704cc6 · 4704cc6
1 parent dcd6c7c
commit 4704cc6
Show file tree

Hide file tree

Showing 40 changed files with 748 additions and 343 deletions.
diff --git a/.github/tests/static-ports/custom-static-ports-cdk-erigon-sequencer-ds.yml b/.github/tests/static-ports/custom-static-ports-cdk-erigon-sequencer-ds.yml
@@ -0,0 +1,6 @@
+# Only expose the datastream port of the cdk-erigon sequencer to a static public port.
+# All the other ports will be allocated dynamically using Kurtosis.
+args:
+  use_dynamic_ports: false
+  static_ports:
+    cdk_erigon_sequencer_start_port: 61700
diff --git a/.github/tests/static-ports/custom-static-ports.yml b/.github/tests/static-ports/custom-static-ports.yml
@@ -0,0 +1,49 @@
+# All services will be exposed through static public ports, with custom ranges defined in this file.
+# - L1 services will be exposed on the range 60000-60999.
+# - L2 services on the range 61000-61999.
+args:
+  use_dynamic_ports: false
+  static_ports:
+    # L1 public ports (60000-60999).
+    l1_el_start_port: 60000
+    l1_cl_start_port: 60010
+    l1_vc_start_port: 60020
+    l1_additional_services_start_port: 60100
+
+    # L2/CDK public ports (61000-61999).
+    ## Agglayer
+    agglayer_start_port: 61000
+    agglayer_prover_start_port: 61010
+
+    ## CDK node
+    cdk_node_start_port: 61100
+
+    ## Bridge services
+    zkevm_bridge_service_start_port: 61200
+    zkevm_bridge_ui_start_port: 61210
+    reverse_proxy_start_port: 61220
+
+    ## Databases
+    database_start_port: 61300
+    pless_database_start_port: 61310
+
+    ## Pool manager
+    zkevm_pool_manager_start_port: 61400
+
+    ## DAC
+    zkevm_dac_start_port: 61500
+
+    ## ZkEVM Provers
+    zkevm_prover_start_port: 61600
+    zkevm_executor_start_port: 61610
+    zkevm_stateless_executor_start_port: 61620
+
+    ## CDK Erigon
+    cdk_erigon_sequencer_start_port: 61700
+    cdk_erigon_rpc_start_port: 61710
+
+    # L2 additional services (62000-62999).
+    arpeggio_start_port: 62000
+    blutgang_start_port: 62010
+    erpc_start_port: 62020
+    panoptichain_start_port: 62030
diff --git a/.github/tests/static-ports/default-static-ports.yml b/.github/tests/static-ports/default-static-ports.yml
@@ -0,0 +1,5 @@
+# All services will be exposed through static ports, with default ranges defined in `input_parser.star`.
+# - L1 services will be exposed on the range 50000-50999.
+# - L2 services on the range 51000-51999.
+args:
+  use_dynamic_ports: false
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -53,7 +53,7 @@ jobs:
         run: |
           ./monitor-verified-batches.sh \
             --enclave ${{ env.ENCLAVE_NAME }} \
-            --rpc-url $(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)
+            --rpc-url $(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)
 
       - name: Dump enclave
         if: ${{ !cancelled() }}
@@ -80,7 +80,10 @@ jobs:
 
       - id: set-matrix
         # List all yml files in the .github/tests directory, as well as test combinations, except for the additional-services.yml file.
-        run: echo "matrix=$(ls -R ./.github/tests/combinations/*.yml | grep -v 'additional-services.yml' | jq -R -s -c 'split("\n")[:-1]')" >> $GITHUB_OUTPUT
+        run: |
+          files=$(ls -R ./.github/tests/combinations/*.yml ./.github/tests/static-ports/custom-static-ports.yml ./.github/tests/static-ports/default-static-ports.yml | grep -v 'additional-services.yml')
+          matrix=$(echo "$files" | jq -R -s -c 'split("\n")[:-1]')
+          echo "matrix=$matrix" >> $GITHUB_OUTPUT
 
   run-with-args:
     needs: list-ymls
@@ -117,17 +120,35 @@ jobs:
 
       - name: Inspect enclave
         run: kurtosis enclave inspect ${{ env.ENCLAVE_NAME }}
+
+      - name: Verify static ports
+        run: |
+          if [[ ${{ matrix.file_name }} == "./.github/tests/static-ports/default-static-ports.yml" ]]; then
+            echo "Making sure public ports start by a 5 as defined by the static ports in the input_parser.star file."
+
+            ports=$(kurtosis enclave inspect ${{ env.ENCLAVE_NAME }} | sed -n '/^========================================== User Services ==========================================$/,$ p' | tail -n +3)
+            if ! wrong_ports=$(echo "$ports" | grep -vE '127.0.0.1:5|none'); then
+              echo "✅ Default static ports are set correctly."
+              exit 0
+            else
+              echo "wrong_ports: $wrong_ports"
+              echo "❌ Default static ports are not set correctly."
+              exit 1
+            fi
+          else
+            echo "Skipping."  
+          fi
 
       - name: Monitor verified batches (Central RPC)
         run: |
           sequencer_type=$(yq --raw-output '.args.sequencer_type' ${{ matrix.file_name }})
           rpc_name=""
           if [[ "$sequencer_type" == "erigon" ]]; then
-            rpc_name="cdk-erigon-node-001"
+            rpc_name="cdk-erigon-rpc-001"
           elif [[ "$sequencer_type" == "zkevm" ]]; then
             rpc_name="zkevm-node-rpc-001"
           elif [[ "$sequencer_type" == "null" ]]; then
-            rpc_name="cdk-erigon-node-001"
+            rpc_name="cdk-erigon-rpc-001"
           else
             echo "Unknown sequencer type: $sequencer_type"
             exit 1
@@ -167,7 +188,7 @@ jobs:
           name: ${{ env.ARCHIVE_NAME }}
           path: ./dump
 
-  additional_services:
+  additional-services:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
@@ -199,7 +220,7 @@ jobs:
         run: |
           ./monitor-verified-batches.sh \
             --enclave ${{ env.ENCLAVE_NAME }} \
-            --rpc-url $(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)
+            --rpc-url $(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)
 
       - name: Verify Arpeggio RPC
         run: |
@@ -270,7 +291,7 @@ jobs:
           name: dump_additional_services_${{ github.run_id }}
           path: ./dump
 
-  attach_cdks:
+  attach-second-cdk:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
@@ -303,7 +324,7 @@ jobs:
           kurtosis files download ${{ env.ENCLAVE_NAME }} agglayer-config-artifact
           cd agglayer-config-artifact
           # Update the config by adding the rpc and proof signer of the second chain.
-          tomlq -Y --toml-output --in-place '."full-node-rpcs" += {"2": "http://cdk-erigon-node-002:8123"}' agglayer-config.toml
+          tomlq -Y --toml-output --in-place '."full-node-rpcs" += {"2": "http://cdk-erigon-rpc-002:8123"}' agglayer-config.toml
           # Replace the agglayer config.
           agglayer_container_id="$(docker ps --filter name=agglayer --format json | jq -r -s '. | map(select(.Names | startswith("agglayer--"))) | .[].ID')"
           docker cp agglayer-config.toml "$agglayer_container_id:/etc/zkevm/agglayer-config.toml"
@@ -319,14 +340,14 @@ jobs:
         run: |
           ./monitor-verified-batches.sh \
             --enclave ${{ env.ENCLAVE_NAME }} \
-            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)"
+            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)"
 
       - name: Monitor verified batches of the second L2 chain (CDK Erigon Permissionless RPC)
         working-directory: .github/scripts
         run: |
           ./monitor-verified-batches.sh \
             --enclave ${{ env.ENCLAVE_NAME }} \
-            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-002 rpc)"
+            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-002 rpc)"
 
       - name: Dump enclave
         if: ${{ !cancelled() }}
@@ -339,7 +360,7 @@ jobs:
           name: dump_attach_ckds_${{ github.run_id }}
           path: ./dump
 
-  deploy_to_external_l1:
+  deploy-to-external_l1:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
@@ -375,7 +396,7 @@ jobs:
         run: |
           ./monitor-verified-batches.sh \
             --enclave ${{ env.ENCLAVE_NAME }} \
-            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)"
+            --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)"
 
       - name: Dump enclave
         if: ${{ !cancelled() }}

diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
@@ -104,7 +104,7 @@ jobs:
 
   #     - name: Monitor verified batches (CDK Erigon Permissionless RPC)
   #       working-directory: .github/scripts
-  #       run: ./monitor-verified-batches.sh --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)"
+  #       run: ./monitor-verified-batches.sh --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)"
 
   # Deploy the CDK environment incrementally, stage by stage.
   # TODO: Fix this job.
@@ -194,4 +194,4 @@ jobs:
 
   #     - name: Monitor verified batches (CDK Erigon Permissionless RPC)
   #       working-directory: .github/scripts
-  #       run: ./monitor-verified-batches.sh --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-node-001 rpc)"
+  #       run: ./monitor-verified-batches.sh --rpc-url "$(kurtosis port print ${{ env.ENCLAVE_NAME }} cdk-erigon-rpc-001 rpc)"
diff --git a/README.md b/README.md
@@ -88,10 +88,10 @@ kurtosis enclave inspect cdk
 
 That output, while quite useful, might also be a little overwhelming. Let's store the RPC URL in an environment variable.
 
-> You may need to adjust the various commands slightly if you deployed the legacy [zkevm-node](https://github.com/0xPolygonHermez/zkevm-node) as the sequencer. You should target the `zkevm-node-rpc-001` service instead of `cdk-erigon-node-001`.
+> You may need to adjust the various commands slightly if you deployed the legacy [zkevm-node](https://github.com/0xPolygonHermez/zkevm-node) as the sequencer. You should target the `zkevm-node-rpc-001` service instead of `cdk-erigon-rpc-001`.
 
 ```bash
-export ETH_RPC_URL="$(kurtosis port print cdk cdk-erigon-node-001 rpc)"
+export ETH_RPC_URL="$(kurtosis port print cdk cdk-erigon-rpc-001 rpc)"
 ```
 
 That is the same environment variable that `cast` uses, so you should now be able to run this command. Note that the steps below will assume you have the [Foundry toolchain](https://book.getfoundry.sh/getting-started/installation) installed.
@@ -162,11 +162,12 @@ For more information about the CDK stack, visit the [Polygon Knowledge Layer](ht
 
 This section features documentation specifically designed for advanced users, outlining complex operations and techniques.
 
+- How to use CDK [ACL](docs/acl-allowlists-blocklists.md).
 - How to deploy [additional services](docs/additional-services.md) alongside the CDK stack, such as transaction spammer, monitoring tools, permissionless nodes etc.
 - How to [attach multiple CDK chains to the AggLayer](docs/attach-multiple-cdks.md).
-- How to use CDK [ACL](docs/acl-allowlists-blocklists.md).
 - How to use the different [data availability modes](docs/data-availability-modes.md).
 - How to [deploy the stack to an external L1](docs/deploy-using-sepolia.org) such as Sepolia.
+- How to [deploy contracts with the deterministic deployment proxy](docs/deterministic-deployment-proxy.md).
 - How to [edit the zkevm contracts](docs/edit-contracts.md).
 - How to [perform an environment migration](docs/environment-migration.org) with clean copies of the databases.
 - How to [iterate and debug quickly](docs/fast-iteration-cycle.md) with Kurtosis.
@@ -177,10 +178,12 @@ This section features documentation specifically designed for advanced users, ou
 - How to use a [native token](docs/native-token/native-token.md).
 - How to [play with the network](docs/network-ops.org) to introduce latencies.
 - How to [set up a permissionless zkevm node](docs/permissionless-zkevm-node.md).
+- How to [resequence batches with the cdk-erigon sequencer](docs/resequence-sequencer/resequence-sequencer.md).
 - How to [run a debugger](docs/running-a-debugger/running-a-debugger.org).
+- How to [assign static ports](docs/static-ports/static-ports.md) to Kurtosis services.
 - How to work with the [timelock](docs/timelock.org).
 - How to [trigger a reorg](docs/trigger-a-reorg/trigger-a-reorg.md).
-- How to [deploy contracts with the deterministic deployment proxy](docs/deterministic-deployment-proxy.md).
+- How to [perform a trustless recovery the DAC and L1](docs/trustless-recovery-from-dac-l1.md).
 
 ## FAQ
 
@@ -299,6 +302,8 @@ kurtosis cluster set docker
 <details>
 <summary><b>Click to expand</b></summary>
 
+Occasionally, Kurtosis deployments may run indefinitely. Typically, deployments should complete within 10 to 15 minutes. If you experience longer deployment times or if it seems stuck, check the Docker engine's memory limit and set it to 16GB if possible. If this does not resolve the issue, please refer to the troubleshooting steps provided.
+
 1. Make sure the issue is related to Kurtosis itself. If you made any changes to the package, most common issues are misconfigurations of services, file artefacts, ports, etc.
 
 2. Remove the Kurtosis enclaves.