merge master

Signed-off-by: ashcherbakov <[email protected]>

CHANGELOG.md

@@ -1,5 +1,7 @@
 # Hyperledger Indy Node Release Notes
 
+* [1.9.1](#191)
+
 * [1.9.0](#190)
 
 * [1.8.1](#181)
@@ -44,6 +46,47 @@
 
 Although every attempt has been made to make this information as accurate as possible, please know there may be things that are omitted, not fully developed yet, or updates since this publication that were not included in the information below. Only the most pressing or significant items have been listed. For the entire list of tickets and or specific information about any given item, please visit the list at [Hyperleder Indy's Jira](https://jira.hyperledger.org/). Once logged in, simply navigate to Projects > Indy.
 
+## 1.9.1
+### Release date: Aug 02nd, 2019
+
+### Component Version Information
+| Components | Version Numbers |
+| --- | --- |
+| indy-plenum | 1.9.1 |
+| indy-node | 1.9.1 |
+| sovrin | 1.1.52 |
+
+### Additional Information:
+**There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.**
+
+### Major Changes
+- New DIDs can be created without endorsers
+- Transaction authors don't need to be endorsers
+- TAA acceptance should use date, not time
+- Bug fixes
+
+### Detailed Changelog
+
+#### Major Fixes
+| Description | Additional Information | Ticket Number |
+| --- | --- | --- |
+| Incorrect request validation | | [INDY-2164](https://jira.hyperledger.org/browse/INDY-2164) |
+| Need to make "reask_ledger_status" repeatable | | [INDY-2112](https://jira.hyperledger.org/browse/INDY-2112) |
+| When view change takes too long instance change should be sent periodically | | [INDY-2143](https://jira.hyperledger.org/browse/INDY-2143) |
+
+#### Changes and Additions
+| Description | Additional Information | Ticket Number |
+| --- | --- | --- |
+| New DIDs can be created without endorsers | | [INDY-2171](https://jira.hyperledger.org/browse/INDY-2171) |
+| Transaction authors don't need to be endorsers | | [INDY-2173](https://jira.hyperledger.org/browse/INDY-2173) |
+| Grab pool data for failed system tests | | [INDY-2141](https://jira.hyperledger.org/browse/INDY-2141) |
+| Memory profiling needs to be removed from GET_VALIDATOR_INFO output | | [INDY-2182](https://jira.hyperledger.org/browse/INDY-2182) |
+| Implement PBFT viewchanger service with most basic functionality | | [INDY-2147](https://jira.hyperledger.org/browse/INDY-2147) |
+| Extract Orderer service from Replica | | [INDY-2136](https://jira.hyperledger.org/browse/INDY-2136) |
+| Extract and integrate ConsensusDataProvider from Replica | | [INDY-2139](https://jira.hyperledger.org/browse/INDY-2139) |
+| TAA acceptance should use date, not time | | [INDY-2157](https://jira.hyperledger.org/browse/INDY-2157) |
+| Clean-up Pluggable Request Handlers | | [INDY-2154](https://jira.hyperledger.org/browse/INDY-2154) |
+
 ## 1.9.0
 ### Release date: July 04th, 2019
 

Jenkinsfile.cd

@@ -92,7 +92,7 @@ def systemTests = { component, releaseVersion ->
             ['test_vc.py'],
             ['test_consensus.py']
         ]
-        testVersion = 'v0.8.7'
+        testVersion = 'v0.8.10'
         testVersionByTag = true
         delegate.gatherLogs = gatherLogs
     }

Jenkinsfile.nightly

@@ -46,7 +46,7 @@ try {
             ['test_ledger.py'],
             ['test_vc.py'],
             ['test_consensus.py', 'TestTAASuite.py'],
-            ['test_upgrade.py', 'test_roles.py', 'test_freshness.py', 'TestMultiSigSuite.py'],
+            ['test_roles.py', 'test_freshness.py', 'TestMultiSigSuite.py'],
             ['TestAuditSuite.py'],
             // set of authmap tests
             // TODO might be groupped in parts once https://github.com/docker/docker-py/issues/2278 is resolved
@@ -60,7 +60,7 @@ try {
             ['TestAuthMapSchemaSuite.py'],
             ['TestAuthMapUpgradeSuite.py']
         ]
-        testVersion = 'v0.8.7'
+        testVersion = 'v0.8.10'
         testVersionByTag = true
         delegate.gatherLogs = gatherLogs
     }

ci/ubuntu.dockerfile

@@ -8,7 +8,7 @@ ARG venv=venv
 RUN apt-get update -y && apt-get install -y \
     python3-nacl \
     libindy-crypto=0.4.5 \
-    libindy=1.10.0~1198 \
+    libindy=1.10.1~1220 \
 # rocksdb python wrapper
     libbz2-dev \
     zlib1g-dev \

docs/source/ci-cd.md

@@ -127,36 +127,84 @@ Each `build-scripts` folder includes `Readme.md`. Please check them for more det
 
 ## Release workflow
 
-1. Release candidate preparation
-    1. [**Maintainer**] Creates a new release branch `release-X.Y.Z` based on `stable`.
-    2. [**Contributor**]
-        - Creates a new release candidate branch (e.g. `rc-X.Y.Z.rc1`) based on that release branch.
-        - Merges `master` branch.
-        - Sets stable version of `indy-plenum` in `setup.py` (for `indy-node` only).
-        - Sets new version `X.Y.Z.rc1` (`./bump_version.sh X.Y.Z.rc1`).
-        - Commits and pushes changes.
-        - Creates a release candidate PR to `release-X.Y.Z`.
-    3. [**Maintainer**] Waits for CI, reviews the release candidate PR and either merges the PR or asks for changes.
-2. Release candidate acceptance
-    1. [**Maintainer**] Once the release candidate PR is merged the maintainer **starts release candidate pipeline manually**.
-    2. [**build server**] Once the CD pipeline is started (manually triggered) for branch `release-X.Y.Z` it does the following:
-        - creates and pushes release commit to `release-X.Y.Z`;
-        - publishes release candidates packages to PyPI and debian `rc` components;
-        - performs system testing (`indy-node` only);
-        - creates a release PR to merge `release-X.Y.Z` to `stable`;
-        - waits for an approval to proceed.
-    3. [**Maintainer/QA**] Waits for CI, reviews the release PR and either approves or rejects:
-        - may run additional tests against the release candidate before approval;
-        - in case of approval lets build server to proceed but **does not merge the release PR manually**;
-        - otherwise stops the pipeline and previous steps are repeated for new release candidate `X.Y.Z.rc1` and possible future ones.
-    4. [**build server**]
-        - once it is approved to proceed performs fast-forward merging to stable and creates tag `vX.Y.Z`;
-        - otherwise rollbacks release commit pushed to release branch `release-X.Y.Z`.
-3. Publishing
-    1. [**build server**] Once the release PR is merged stable pipeline is triggered and it:
-        - publishes to PyPI;
-        - re-packs `rc` debian package and publishes to debian `stable` components.
-
-Hotfix releases are quite similar except the following difference:
-  - hotifx branches `hotfix-X.Y.Z` are created from git tag `vX.Y.(Z-1)`;
-  - `master` is not merged since hotfixes (as a rule) should include only fixes for stable code.
+### Feature Release
+
+#### 1. Release Candidate Preparation
+
+1. [**Maintainer**]
+    - Create `release-X.Y.Z` branch from `stable` (during the first RC preparation only).
+2. [**Contributor**]
+    - Create `rc-X.Y.Z.rcN` branch from `release-X.Y.Z` (`N` starts from `1` and is incremented for each new RC).
+    - Apply necessary changes from `master` (either `merge` or `cherry-pick`).
+    - (_optional_) [`indy-node`] Set `indy-plenum` version in `setup.py`.
+    - Set the package version `./bump_version.sh X.Y.Z.rcN`.
+    - Commit, push and create a PR to `release-X.Y.Z`.
+3. Until PR is merged:
+    1. [**build server**]
+        - Run CI for the PR and notifies GitHub.
+    2. [**Maintainer**]
+        - Review the PR.
+        - Either ask for changes or merge.
+    3. [**Contributor**]
+        - (_optional_) Update the PR if either CI failed or reviewer asked for changes.
+        - (_optional_) [**indy-node**] Bump `indy-plenum` version in `setup.py` if changes require new `indy-plenum` release.
+
+#### 2. Release Candidate Acceptance
+
+**Note** If any of the following steps fails new release candidate should be prepared.
+
+1. [**Maintainer**]
+    - **Start release candidate pipeline manually**.
+2. [**build server**]
+    - Checkout the repository.
+    - Publish to PyPI as `X.Y.Z.rcN`.
+    - Bump version locally to `X.Y.Z`, commit and push as the `release commit` to remote.
+    - Build debian packages:
+        - for the project: source code version would be `X.Y.Z`, debian package version `X.Y.Z~rcN`;
+        - for the 3rd party dependencies missed in the official debian repositories.
+    - Publish the packages to `rc-latest` debian channel.
+    - [`indy-node`] Copy the package along with its dependencies (including `indy-plenum`)
+      from `rc-latest` to `rc` channel.
+    - [`indy-node`] Run system tests for the `rc` channel.
+    - Create **release PR** from `release-X.Y.Z` (that points to `release commit`) branch to `stable`.
+    - Notify maintainers.
+    - Wait for an approval to proceed. **It shouldn't be provided until `release PR` passes all necessary checks** (e.g. DCO, CI testing, maintainers reviews etc.).
+3. [**build server**]
+    - Run CI for the PR and notify GitHub.
+4. [**QA**]
+    - (_optional_) Perform additional testing.
+5. [**Maintainer**]
+    - Review the PR but **do not merge it**.
+    - If approved: let build server to proceed.
+    - Otherwise: stop the pipeline.
+6. [**build server**]
+    - If approved:
+        - perform fast-forward merge;
+        - create and push tag `vX.Y.Z`;
+        - Notify maintainers.
+    - Otherwise rollback `release commit` by moving `release-X.Y.Z` to its parent.
+
+#### 3. Publishing
+
+1. [**build server**] triggered once the `release PR` is merged
+    - Publish to PyPI as `X.Y.Z`.
+    - Download and re-pack debian package `X.Y.Z~rcN` (from `rc-latest` channel) to `X.Y.Z` changing only the package name.
+    - Publish the package to `rc-latest` debian channel.
+    - Copy the package along with its dependencies from `rc-latest` to `stable-latest` channel.
+    - [`indy-node`] Copy the package along with its dependencies (including `indy-plenum`) from `stable-latest` to `stable` channel.
+    - [`indy-node`] Run system tests for the `stable` channel.
+    - Notify maintainers.
+
+#### 4. New Development Cycle Start
+
+1. [**Contributor**]:
+    - Create PR to `master` with version bump to `X'.Y'.Z'.dev0`, where `X'.Y'.Z'` is next target release version. Usually it increments one of `X`, `Y` or `Z` and resets lower parts (check [SemVer](https://semver.org/) for more details), e.g.:
+        - `X.Y.Z+1` - bugfix release
+        - `X.Y+1.0` - feature release, backwards compatible API additions/changes
+        - `X+1.0.0` - major release, backwards incompatible API changes
+
+### Hotfix Release
+
+Hotfix release is quite similar except the following difference:
+  - hotfix branches named `hotfix-X.Y.Z`;
+  - `master` usually is not merged since hotfixes (as a rule) should include only fixes for stable code.

indy_node/__version__.json

@@ -1 +1 @@
-[1, 9, 1, "dev", 0]
+[1, 9, 2, "dev", 0]

indy_node/test/endorser/helper.py

@@ -1,23 +1,13 @@
-import json
+from indy.ledger import append_request_endorser
 
-from plenum.common.types import f
 from plenum.test.helper import sdk_get_and_check_replies, sdk_send_signed_requests, sdk_multisign_request_object
 
 
 def sdk_append_request_endorser(
         looper,
         request_json,
         endorser_did):
-    request = json.loads(request_json)
-    request[f.ENDORSER.nm] = endorser_did
-    return json.dumps(request)
-    # TODO call SDK
-    # return looper.loop.run_until_complete(
-    #     append_request_endorser(
-    #         request_json,
-    #         endorser_did=endorser_did,
-    #     )
-    # )
+    return looper.loop.run_until_complete(append_request_endorser(request_json, endorser_did))
 
 
 def sdk_submit_and_check_by_endorser(looper, sdk_pool_handle, sdk_wallet_author, sdk_wallet_endorser, request_json):

setup.py

@@ -27,7 +27,7 @@
 
 BASE_DIR = os.path.join(os.path.expanduser("~"), ".indy")
 
-tests_require = ['pytest==3.3.1', 'pytest-xdist==1.22.1', 'python3-indy==1.10.0-dev-1198', 'pytest-asyncio==0.8.0']
+tests_require = ['pytest==3.3.1', 'pytest-xdist==1.22.1', 'python3-indy==1.10.1-dev-1220', 'pytest-asyncio==0.8.0']
 
 setup(
     name=metadata['__title__'],