Skip to content

Commit

Permalink
Merge remote-tracking branch 'base/master' into indy_2144
Browse files Browse the repository at this point in the history
  • Loading branch information
ArtObr committed Jul 19, 2019
2 parents d6c8faf + 611a70b commit fb72ac7
Show file tree
Hide file tree
Showing 95 changed files with 1,220 additions and 2,233 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ cli_output.log
.cache
nosetests.xml
coverage.xml
tails/

# Translations
*.mo
Expand Down
51 changes: 51 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Hyperledger Indy Node Release Notes

* [1.9.0](#190)

* [1.8.1](#181)

* [1.8.0](#180)
Expand Down Expand Up @@ -42,6 +44,55 @@

Although every attempt has been made to make this information as accurate as possible, please know there may be things that are omitted, not fully developed yet, or updates since this publication that were not included in the information below. Only the most pressing or significant items have been listed. For the entire list of tickets and or specific information about any given item, please visit the list at [Hyperleder Indy's Jira](https://jira.hyperledger.org/). Once logged in, simply navigate to Projects > Indy.

## 1.9.0
### Release date: July 04th, 2019

### Component Version Information
| Components | Version Numbers |
| --- | --- |
| indy-plenum | 1.9.0 |
| indy-node | 1.9.0 |
| sovrin | 1.1.50 |

### Additional Information:
**There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.**
**Some nodes can fail to send a REJECT or REPLY to client under specific network conditions. See Know Issues for more details.**

### Major Changes
- Pluggable Request Handlers have been implemented

### Detailed Changelog

#### Major Fixes
| Description | Additional Information | Ticket Number |
| --- | --- | --- |
| Propagates with invalid requests can lead to node crashes | | [INDY-2144](https://jira.hyperledger.org/browse/INDY-2144) |
| There is no validation of the ISSUANCE_TYPE field for the transaction REVOC_REG_DEF | | [INDY-2142](https://jira.hyperledger.org/browse/INDY-2142) |
| Reduce CONS_PROOF timeout to speed up catchup under the load | | [INDY-2083](https://jira.hyperledger.org/browse/INDY-2083) |

#### Changes and Additions
| Description | Additional Information | Ticket Number |
| --- | --- | --- |
| As a Trustee(s), I need to have a way to set multiple AUTH_RULES by one command | | [INDY-2087](https://jira.hyperledger.org/browse/INDY-2087) |
| Make more system tests to be ready for Indy Node CD pipeline | | [INDY-2127](https://jira.hyperledger.org/browse/INDY-2127) |
| Integrate new handlers into the codebase | | [INDY-1861](https://jira.hyperledger.org/browse/INDY-1861) |
| Define Interfaces needed for View Change Service | | [INDY-1338](https://jira.hyperledger.org/browse/INDY-1338) |
| Rename TRUST_ANCHOR to ENDORSER | | [INDY-1950](https://jira.hyperledger.org/browse/INDY-1950) |
| Update PBFT view change plan of attack | | [INDY-2134](https://jira.hyperledger.org/browse/INDY-2134) |
| Apply a new Docker-in-docker approach for system tests | | [INDY-2131](https://jira.hyperledger.org/browse/INDY-2131) |
| More tests for pluggable request handlers | | [INDY-2108](https://jira.hyperledger.org/browse/INDY-2108) |
| Remove ANYONE_CAN_WRITE | | [INDY-1956](https://jira.hyperledger.org/browse/INDY-1956) |
| [Design] ViewChange protocol must be as defined in PBFT | | [INDY-1290](https://jira.hyperledger.org/browse/INDY-1290) |
| Batch containing some already executed requests should be applied correctly | | [INDY-1405](https://jira.hyperledger.org/browse/INDY-1405) |
| Update Pluggable Req Handlers | | [INDY-2097](https://jira.hyperledger.org/browse/INDY-2097) |
| As a Network Admin, I need to be able to forbid an action in AUTH_RULE, so that no changes in code are needed | | [INDY-2077](https://jira.hyperledger.org/browse/INDY-2077) |
| Create Builders for handlers | | [INDY-1860](https://jira.hyperledger.org/browse/INDY-1860) |

#### Known Issues
| Description | Additional Information | Ticket Number |
| --- | --- | --- |
| Incorrect request validation || [INDY-2164](https://jira.hyperledger.org/browse/INDY-2164) |

## 1.8.1
### Release date: June 06th, 2019

Expand Down
182 changes: 17 additions & 165 deletions Jenkinsfile.cd
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

String name = 'indy-node'
String pkgName = name
String mainModuleName = 'indy_node'

def nodeTestUbuntu = {
try {
Expand Down Expand Up @@ -74,173 +75,24 @@ def buildDebUbuntu = { releaseVersion, sourcePath, packageVersion=null, missedPk
}

def systemTests = { component, releaseVersion ->

String prefix = "System Tests ($component)"
String systemTestsNetwork = 'indy-test-automation-network'
String systemTestsDir = './system_tests'

List testsSchema = [
['test_ledger.py'],
['test_vc.py'],
['test_consensus.py', 'TestTAASuite.py'],
['test_upgrade.py', 'test_roles.py', 'test_freshness.py', 'TestMultiSigSuite.py']
]

Map indyPlenumVersions = [:]
Map indySDKVersions = [:]
Map indyCryptoVersions = [:]

def dockerClean = {
sh "./system/docker/clean.sh $systemTestsNetwork"

try {
sh "docker ps -q --filter network=$systemTestsNetwork | xargs -r docker rm -f"
} catch (Exception exc) {
echo "$prefix: failed to remove docker containers in $systemTestsNetwork network: $exc"
throw exc
}

try {
sh "docker network ls -q --filter name=$systemTestsNetwork | xargs -r docker network rm"
} catch (Exception exc) {
echo "$prefix: failed to remove docker $systemTestsNetwork network: $exc"
throw exc
}

sh "docker container prune -f"
sh "docker network prune -f"
}

def runTest = { testGroup ->

stage("[${testGroup}] Checkout system tests") {
testHelpers.getSystemTests(tag: 'v0.7.0', targetDir: systemTestsDir)
}

dir(systemTestsDir) {
stage("[${testGroup}] Patch system tests python requirements") {
sh """
sed -i 's/python3-indy.*/python3-indy==${indySDKVersions.pypi}/g' ./system/requirements.txt
#sed -i 's/indy-plenum.*/indy-plenum==${indyPlenumVersions.pypi}/g' ./system/requirements.txt
#sed -i 's/indy-crypto.*/indy-crypto==${indyCryptoVersions.pypi}/g' ./system/requirements.txt
"""
}

stage("[${testGroup}] Cleanup docker") {
dockerClean()
}

stage("[${testGroup}] Prepare docker env") {
withEnv([
"INDY_NODE_REPO_COMPONENT=$component",
"LIBINDY_CRYPTO_VERSION=${indyCryptoVersions.debian}",
"PYTHON3_LIBINDY_CRYPTO_VERSION=${indyCryptoVersions.debian}",
"INDY_PLENUM_VERSION=${indyPlenumVersions.debian}",
"INDY_NODE_VERSION=$releaseVersion",
"LIBINDY_REPO_COMPONENT=${indySDKVersions.debian == indySDKVersions.pypi ? 'stable' : 'master'}",
"LIBINDY_VERSION=${indySDKVersions.debian}",
]) {
sh "./system/docker/prepare.sh $systemTestsNetwork"
}
}

try {
def err
String testReportFileNameXml = "system_tests_${testGroup}_report.${component}.xml"
String testReportFileNamePlain = "system_tests_${testGroup}_report.${component}.txt"
String testTargets = testsSchema[testGroup].collect{"system/indy-node-tests/$it"}.join(' ')
try {
stage("[${testGroup}] Run tests") {
sh """
bash -c "\
set -o pipefail; \
./system/docker/run.sh \
\\"$testTargets\\" \
\\"-l -vv --junit-xml=$testReportFileNameXml\\" \
\\"$systemTestsNetwork\\" 2>&1 | tee $testReportFileNamePlain;\
"
"""
}
} catch (_err) {
err = _err
throw _err
} finally {
stage("[${testGroup}] Upload test report") {
sh "ls -la *report* || true"
if (err) {
archiveArtifacts artifacts: testReportFileNamePlain, allowEmptyArchive: true
}
junit testResults: testReportFileNameXml, allowEmptyResults: true
}
}
} catch (Exception exc) {
echo "$prefix: fail: $exc"
throw exc
} finally {
stage("[${testGroup}] Cleanup docker") {
dockerClean()
}
}
}
}

nodeWrapper("ubuntu") {
stage("Checkout SCM") {
def localLib
nodeWrapper('ubuntu') {
stage('Load local shared library') {
checkout scm
localLib = load 'ci/pipeline.groovy'
}
}

stage("Get versions of dependencies") {
String pipLogName = "pip.intsall.log"
def uid = sh(returnStdout: true, script: 'id -u').trim()
docker.build("hyperledger/indy-node-ci", "--build-arg uid=$uid -f ci/ubuntu.dockerfile ci").inside {
sh """
pip install .[tests] >$pipLogName
"""

indyPlenumVersions.pypi = sh(returnStdout: true, script: """
grep "^Collecting indy-plenum==" $pipLogName | awk '{print \$2}' | awk -F'==' '{print \$2}'
""").trim()
indyPlenumVersions.debian = indyPlenumVersions.pypi.replaceAll(/\.?(dev|rc)(.*)/, "~\$1\$2")
echo "indy-plenum versions: $indyPlenumVersions"

indySDKVersions.pypi = sh(returnStdout: true, script: """
grep "^Collecting python3-indy==" $pipLogName | awk '{print \$2}' | awk -F'==' '{print \$2}'
""").trim()
indySDKVersions.debian = indySDKVersions.pypi.replaceAll(/-(dev|rc)-(.*)/, "~\$2")
echo "indy-sdk version: ${indySDKVersions}"

indyCryptoVersions.pypi = sh(returnStdout: true, script: """
grep "^Collecting indy-crypto==" $pipLogName | awk '{print \$2}' | awk -F'==' '{print \$2}'
""").trim()
indyCryptoVersions.debian = indyCryptoVersions.pypi.replaceAll(/-(dev|rc)-(.*)/, "~\$2")
echo "indy-crypto version: ${indyCryptoVersions}"
}

if (!(indyPlenumVersions.debian && indySDKVersions.debian && indyCryptoVersions.debian)) {
error "Failed to get versions for indy-plenum or indy-crypto or indy-sdk"
}
}

Map builds = [:]
for (int i = 0; i < testsSchema.size(); i++) {
String testNames = testsSchema[i].join(' ')
Boolean isFirst = (i == 0)
int testGroup = i
builds[testNames] = {
stage("Run ${testNames}") {
if (isFirst) {
runTest(testGroup)
} else {
nodeWrapper('ubuntu') {
runTest(testGroup)
}
}
}
}
}
builds.failFast = false

parallel builds
localLib.systemTests {
repoChannel = component
pkgVersion = releaseVersion
testSchema = [
['test_ledger.py'],
['test_vc.py'],
['test_consensus.py']
]
testVersion = 'v0.8.4'
testVersionByTag = true
}
}

Expand Down Expand Up @@ -277,5 +129,5 @@ testAndPublish(
[
ubuntu: [node: nodeTestUbuntu, common: commonTestUbuntu]
],
true, options, [ubuntu: buildDebUbuntu], 'indy_node'
true, options, [ubuntu: buildDebUbuntu], mainModuleName
)
2 changes: 1 addition & 1 deletion Jenkinsfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ for (i = 0; i < _labels.size(); i++) {
// PIPELINE

try {
timeout(60) {
timeout(90) {
stage('Static code validation') {
if (config.codeValidation) {
node(labels.linux) {
Expand Down
73 changes: 73 additions & 0 deletions Jenkinsfile.nightly
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#!groovy

@Library('[email protected]') _

String pkgName = 'indy-node'
String mainModuleName = 'indy_node'
String emailRecipients = params.INDY_NODE_RECIPIENTS ?: env.INDY_NODE_RECIPIENTS ?: ''

def localLib
def err
String buildPkgVersion
String buildSrcVersion

String scmRepoUrl
String scmSha1

try {
nodeWrapper('ubuntu') {
stage('Resolve version to build') {
docker.image('hyperledger/indy-core-baseci:0.0.3-master').inside('-u 0') {
sh "apt-get update && apt-get install -y $pkgName"
releaseVersion = getReleaseVersion(mainModuleName, false)
buildPkgVersion = "${releaseVersion.release}~${releaseVersion.pre}${releaseVersion.revision}"
buildSrcVersion = sh(returnStdout: true, script: """
python3 -c "from $mainModuleName import load_manifest; print(load_manifest()['sha1'])"
""").trim()
}
echo "Version to build: buildSrcVersion=$buildSrcVersion, buildPkgVersion=$buildPkgVersion"
}

stage('Load local shared library') {
checkout scm
localLib = load 'ci/pipeline.groovy'

scmRepoUrl = gitHelper.repoUrl()
scmSha1 = gitHelper.sha1()
}
}

localLib.systemTests {
repoChannel = 'master'
pkgVersion = buildPkgVersion
srcVersion = buildSrcVersion
testSchema = [
['test_ledger.py'],
['test_vc.py'],
['test_consensus.py', 'TestTAASuite.py'],
['test_upgrade.py', 'test_roles.py', 'test_freshness.py', 'TestMultiSigSuite.py'],
['TestAuditSuite.py'],
['TestAuthMapSuite.py']
]
testVersion = 'v0.8.4'
testVersionByTag = true
}
} catch(Exception _err) {
currentBuild.result = "FAILED"
err = _err
throw _err
} finally {
stage('Build result notification') {
sendNotification.email {
to = emailRecipients
subject = "[${pkgName}][nightly] Build #${this.env.BUILD_NUMBER} ${err ? 'failed' : 'succeed'} for version ${buildPkgVersion} (${buildSrcVersion})"
srcUrl = "${scmRepoUrl}/tree/${scmSha1}"
}
}

if (err) {
stage('Error dump') {
echo "Pipeline failed: $err"
}
}
}
Loading

0 comments on commit fb72ac7

Please sign in to comment.