Valkey Automate Post-Release Tasks Using Centralized Bot.

[roshkhatri]

Problem Statement

While the release process has been automated up to building the official binaries and uploading them to the S3 bucket, the following steps are still performed manually:

1. Updating the valkey-hashes repository.
2. Running an update script on valkey-container, committing the changes, and triggering the container workflow to upload the images.
3. Updating the website downloads page to reflect the new releases.

Manual execution of these steps introduces delays and inconsistencies.

---

Proposed Solution

To achieve full automation, we propose creating a valkey-automation-bot and a centralized automation-bot repository. This setup will handle post-release tasks in a structured and automated way.

Workflow Overview:

1. Move the build-release-package workflow to the automation-bot repository.
2. Create a workflow that is triggered with the Valkey release to create a new issue to track the release process of the new version. Release Status for Valkey Versions X.Y.Z.
3. Use a repository_dispatch event triggered by a new Valkey release to initiate the automation workflow.
4. The bot will perform the following tasks:
   • Execute the build-release-package workflow to upload the binaries to the S3 bucket.
   • Create Pull Requests (PRs) with required updates in the following repositories:
     • valkey-hashes
     • valkey-container
     • valkey-website
5. Once PRs are reviewed and merged, the bot will trigger subsequent workflows or scripts using:
   • Webhooks for target repositories.
   • Additional repository_dispatch events to progress through the pipeline.
6. Maintain separate automation scripts for each repository step within the centralized automation-bot repository for better organization and reusability.

---

Expected Outcome

1. A fully automated release pipeline, eliminating manual intervention.
2. Improved consistency, accuracy, and efficiency for release updates.

---

Phased Implementation Plan

The automation can be implemented in phases for better tracking and incremental progress:

• Phase 1: PR to update the valkey-hashes repository (Issue #1396).
• Phase 2: Run automation scripts for valkey-container and open a PR.
• Phase 3: PR to update the downloads page on the valkey-website repository.

By following this approach, we can centralize automation workflows, simplify maintenance, and ensure a streamlined release process.

@valkey-io/core-team please share your thoughts and feedback on the same

Originally posted by @roshkhatri in #1396 (comment)

[vudiep411]

Automation for Valkey release

This is a Proof of Concept design to automate the process of release a new version for valkey. We can use a centralize repository to automate the process of build the binary files -> commit the new hash -> update the docker description

Current release process

Currently we still don't have a unified process to automate the workflow. These are the steps still happen manually:

1. Updating the valkey-hashes repository.
2. Running a update script on valkey-container and committing the changes which triggers containers workflow to upload the images.
3. Updating the website downloads page with the new releases.

Originally posted by @roshkhatri in #1396 (comment)

Below is the diagram of the current process:

Propose Design

To automated this process, we would create a centralized repository to take care of all these processes.

1. Create a valkey-automation-bot repo that will be triggered when the main valkey repo release a new version.

2. In this valkey-automation-bot repo, the workflow will run the following steps: build the binary files -> Open a PR for valkey-hashes -> Open a PR for valkey-container.

3. We can remove the build-release-packages.yml, call-build-linux-arm-packages.yml, call-build-linux-x86-packages.yml from valkey repo.

We need the PAT token that can give access to create a PR for valkey-container repo and valkey-hashes repo.

Below is the flow diagram of the design:

Github action jobs

Proof of concept reference

Valkey repo trigger

Valkey-automated-bot

Valkey-container

Valkey-hashes

Results

PR for valkey-hashes

PR for Docker Update

[hwware]

Some commits which comes from merged pr cause CI crash or failures in some platforms, I would suggest 2 points:

1. Before pushing the commit to container and website. it must pass the CI
2. Because CI failure always happens in code repo, we need first push valkey/unstable to container, then push the relative doc to valkey/valkey-doc if necessary.

[roshkhatri]

@hwware.This is not a continuous deployment.

This will only be executed once we have finalized the new release version on valkey-io/valkey and publish it. and then we will also wait for the CI to run on the PR of the respective repositories.

[vudiep411]

Yes CI will run before we merge into valkey-container.

[hwware]

I agree with this process, Thanks