Skip to content

Commit

Permalink
re: chore: ready workspace for publishing (near#5130)
Browse files Browse the repository at this point in the history
### The Plan

I closed near#5100 because manually bumping versions is understandably tasking, and automating this requires special tooling or at least patching existing tooling to support our use case. Otherwise, we'd run into issues from having certain specific dep relationships. Luckily `cargo-workspaces` had already done most of the work, I just needed to patch it to better suit our workflow.

Here's what we do want:

- Organized and automated publishing. Ideally, nearcore as an entity as maintained by all its core contributors should publish its crates. Not individual contributors.
- All crates in the workspace should share the same version.
- The versions of published crates on crates.io should always match their versions on GitHub.
- Unversioned crates like `nearcore`, `neard`, `integration-tests` that are on `0.0.0` should be excluded from version bumping.
- Runtime crates that are already on `3.0.0` (e.g [`crates.io:near-vm-logic`](https://crates.io/crates/near-vm-logic/versions)) should share a common version but be distinct from the rest of the workspace.
- All crates should, by default, be marked private. But to publish, it should be as easy as marking it publishable. (removing `publish = false`).

My fork of [`miraclx/cargo-workspaces`](pksunkara/cargo-workspaces@master...miraclx:grouping-and-exclusion) is now better suited for our use case.

#### How?

 - Unversioned crates that should, in no case, be versioned, are excluded in the workspace manifest.
   https://github.com/near/nearcore/blob/31763be351db59ac6f667d1fca21d91da0229536/Cargo.toml#L49-L58
 - Groups of related crates whose versions need to differ from the rest of the workspace can be grouped in the workspace manifest. (This ensures that while everything is versioned to `0.5.0`, the runtime group is ticked to `3.1.0`)
   https://github.com/near/nearcore/blob/31763be351db59ac6f667d1fca21d91da0229536/Cargo.toml#L60-L69

#### Versioning

The current common version across the workspace as at the current master (00138f2) is `0.4.0` and the current common version across the `[runtime]` crates is `3.0.0`

With this, we bump the versions of crates across the entire workspace to the next minor - `0.5.0` and in the `[runtime]` group to `3.1.0` and only publish however many are marked publishable:

- [near-account-id](https://crates.io/crates/near-account-id/versions) `0.1.0` => `0.5.0`
- [near-chain-configs](https://crates.io/crates/near-chain-configs/versions) `0.1.0` => `0.5.0`
- [near-chain-primitives](https://crates.io/crates/near-chain-primitives/versions) `0.1.0` => `0.5.0`
- [near-chunks-primitives](https://crates.io/crates/near-chunks-primitives/versions) `0.1.0` => `0.5.0`
- [near-client-primitives](https://crates.io/crates/near-client-primitives/versions) `0.1.0` => `0.5.0`
- [near-crypto](https://crates.io/crates/near-crypto/versions) `0.1.0` => `0.5.0`
- [near-jsonrpc-primitives](https://crates.io/crates/near-jsonrpc-primitives/versions) `0.2.0` => `0.5.0`
- [near-metrics](https://crates.io/crates/near-metrics/versions) `0.1.0` => `0.5.0`
- [near-network-primitives](https://crates.io/crates/near-network-primitives/versions) `0.1.0` => `0.5.0`
- [near-primitives-core](https://crates.io/crates/near-primitives-core/versions) `0.4.0` => `0.5.0`
- [near-primitives](https://crates.io/crates/near-primitives/versions) `0.1.0-pre.1` => `0.5.0`
- [near-rpc-error-core](https://crates.io/crates/near-rpc-error-core/versions) `0.1.0` => `0.5.0`
- [near-rpc-error-macro](https://crates.io/crates/near-rpc-error-macro/versions) `0.1.0` => `0.5.0`
- `[runtime]` [near-vm-errors](https://crates.io/crates/near-vm-errors/versions) `3.0.0` => `3.1.0`

#### Publishing Automation

I propose two systems;
 - The first one, let's call this Hermes (the messenger), runs on every push to master, checking for any version changes, publishing any unpublished versions.
 - The second, let's call this Chronos (for time), runs once at the end of the week. Bumping the versions across the entire workspace by one minor tick. Which then opens a PR with all the changes to be reviewed, approved and merged. Once merged, Hermes (the first system) takes over to publish them. This way, we ensure to have consistent versioning across the entire workspace.
  • Loading branch information
miraclx authored Nov 22, 2021
1 parent 6f2a290 commit 07b5264
Show file tree
Hide file tree
Showing 64 changed files with 175 additions and 168 deletions.
17 changes: 8 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,21 +87,20 @@ Refer to [this document](https://docs.nearprotocol.com/docs/contribution/nearcor

# Release Schedule

If a crate is to be published individually the following needs to be added to its `Cargo.toml`:

```toml
[package.metadata.workspaces]
independent = true
```

This is required by [cargo-workspaces](https://github.com/pksunkara/cargo-workspaces) which is used to publish non-private crates in the nearcore workspace.

Once your change ends up in master, it will be released with the rest of the changes by other contributors on the regular release schedules.

On [betanet](https://docs.near.org/docs/concepts/networks#betanet) we run nightly build from master with all the nightly protocol feature enabled.
Every six weeks, we stabilize some protocol features and make a release candidate for testnet.
After the release candidate has been running on testnet for 2 weeks and no issue is observed, we stabilize and publish the release for mainnet.

# Crate Versioning and Publishing

While all the crates in the workspace are directly unversioned (`v0.0.0`), they all share a unified variable version in the [workspace manifest](Cargo.toml). This keeps versions consistent across the workspace and informs their versions at the moment of publishing.

We also have CI infrastructure set up to automate the publishing process to crates.io. So, on every merge to master, if there's a version change, it's automatically applied to all the crates in the workspace and it attempts to publish the new versions of all non-private crates. All crates that should be exempt from this process should be marked `private`. That is, they should have the `publish = false` specification in their package manifest.

This process is managed by [cargo-workspaces](https://github.com/pksunkara/cargo-workspaces), with a [bit of magic](https://github.com/pksunkara/cargo-workspaces/compare/master...miraclx:grouping-and-exclusion#files_bucket) sprinkled on top.

## Issue Labels

Issue labels are of the following format `<type>-<content>` where `<type>` is a capital letter indicating the type of the label and `<content>` is a hyphened phrase indicating what is label is about.
Expand Down
Loading

0 comments on commit 07b5264

Please sign in to comment.