[1.1.0] Merge master into 1.1.0 (mimblewimble#2720)

* cleanup legacy "3 dot" check (mimblewimble#2625) * Allow to peers behind NAT to get up to preferred_max connections (mimblewimble#2543) Allow to peers behind NAT to get up to preffered_max connections If peer has only outbound connections it's mot likely behind NAT and we should not stop it from getting more outbound connections * Reduce usage of unwrap in p2p crate (mimblewimble#2627) Also change store crate a bit * Simplify (and fix) output_pos cleanup during chain compaction (mimblewimble#2609) * expose leaf pos iterator use it for various things in txhashset when iterating over outputs * fix * cleanup * rebuild output_pos index (and clear it out first) when compacting the chain * fixup tests * refactor to match on (output, proof) tuple * add comments to compact() to explain what is going on. * get rid of some boxing around the leaf_set iterator * cleanup * [docs] Add switch commitment documentation (mimblewimble#2526) * remove references to no-longer existing switch commitment hash (as switch commitments were removed in ca8447f and moved into the blinding factor of the Pedersen Commitment) * some rewording (points vs curves) and fix of small formatting issues * Add switch commitment documentation * [docs] Documents in grin repo had translated in Korean. (mimblewimble#2604) * Start to M/W intro translate in Korean * translate in Korean * add korean translation on intro * table_of_content.md translate in Korean. * table_of_content_KR.md finish translate in Korean, start to translate State_KR.md * add state_KR.md & commit some translation in State_KR.md * WIP stat_KR.md translation * add build_KR.md && stratum_KR.md * finish translate stratum_KR.md & table_of_content_KR.md * rename intro.KR.md to intro_KR.md * add intro_KR.md file path each language's intro.md * add Korean translation file path to stratum.md & table_of_contents.md * fix difference with grin/master * Fix TxHashSet file filter for Windows. (mimblewimble#2641) * Fix TxHashSet file filter for Windows. * rustfmt * Updating regexp * Adding in test case * Display the current download rate rather than the average when syncing the chain (mimblewimble#2633) * When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process. * Replace the explicitly ignored variables in the pattern with an implicit ignore * remove root = true from editorconfig (mimblewimble#2655) * Add Medium post to intro (mimblewimble#2654) Spoke to @yeastplume who agreed it makes sense to add the "Grin Transactions Explained, Step-by-Step" Medium post to intro.md Open for suggestions on a better location. * add a new configure item for log_max_files (mimblewimble#2601) * add a new configure item for log_max_files * rustfmt * use a constant instead of multiple 32 * rustfmt * Fix the build warning of deprecated trim_right_matches (mimblewimble#2662) * [DOC] state.md, build.md and chain directory documents translate in Korean. (mimblewimble#2649) * add md files for translation. * start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md * add dandelion_KR.md && simulation_KR.md for Korean translation. * add md files for translation. * start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md * add dandelion_KR.md && simulation_KR.md for Korean translation. * remove some useless md files for translation. this is rearrange set up translation order. * add dot end of sentence & translate build.md in korean * remove fast-sync_KR.md * finish build_KR.md translation * finish build_KR.md translation * finish translation state_KR.md & add phrase in state.md to move other language md file * translate blocks_and_headers.md && chain_sync.md in Korean * add . in chain_sync.md , translation finished in doc/chain dir. * fix some miss typos * Api documentation fixes (mimblewimble#2646) * Fix the API documentation for Chain Validate (v1/chain/validate). It was documented as a POST, but it is actually a GET request, which can be seen in its handler ChainValidationHandler * Update the API V1 route list response to include the headers and merkleproof routes. Also clarify that for the chain/outputs route you must specify either byids or byheight to select outputs. * refactor(ci): reorganize CI related code (mimblewimble#2658) Break-down the CI related code into smaller more maintainable pieces. * Specify grin or nanogrins in API docs where applicable (mimblewimble#2642) * Set Content-Type in API client (mimblewimble#2680) * Reduce number of unwraps in chain crate (mimblewimble#2679) * fix: the restart of state sync doesn't work sometimes (mimblewimble#2687) * let check_txhashset_needed return true on abnormal case (mimblewimble#2684) * Reduce number of unwwaps in api crate (mimblewimble#2681) * Reduce number of unwwaps in api crate * Format use section * Small QoL improvements for wallet developers (mimblewimble#2651) * Small changes for wallet devs * Move create_nonce into Keychain trait * Replace match by map_err * Add flag to Slate to skip fee check * Fix secp dependency * Remove check_fee flag in Slate * Add Japanese edition of build.md (mimblewimble#2697) * catch the panic to avoid peer thread quit early (mimblewimble#2686) * catch the panic to avoid peer thread quit before taking the chance to ban * move catch wrapper logic down into the util crate * log the panic info * keep txhashset.rs untouched * remove a warning * [DOC] dandelion.md, simulation.md ,fast-sync.md and pruning.md documents translate in Korean. (mimblewimble#2678) * Show response code in API client error message (mimblewimble#2683) It's hard to investigate what happens when an API client error is printed out * Add some better logging for get_outputs_by_id failure states (mimblewimble#2705) * Switch commitment doc fixes (mimblewimble#2645) Fix some typos and remove the use of parentheses in a couple of places to make the reading flow a bit better. * docs: update/add new README.md badges (mimblewimble#2708) Replace existing badges with SVG counterparts and add a bunch of new ones. * Update intro.md (mimblewimble#2702) Add mention of censoring attack prevented by range proofs * use sandbox folder for txhashset validation on state sync (mimblewimble#2685) * use sandbox folder for txhashset validation on state sync * rustfmt * use temp directory as the sandbox instead actual db_root txhashset dir * rustfmt * move txhashset overwrite to the end of full validation * fix travis-ci test * rustfmt * fix: hashset have 2 folders including txhashset and header * rustfmt * (1)switch to rebuild_header_mmr instead of copy the sandbox header mmr (2)lock txhashset when overwriting and opening and rebuild * minor improve on sandbox_dir * add Japanese edition of state.md (mimblewimble#2703) * Attempt to fix broken TUI locale (mimblewimble#2713) Can confirm that on the same machine 1.0.2 TUI looks great and is broken on the current master. Bump of `cursive` version fixed it for me. Fixes mimblewimble#2676 * clean the header folder in sandbox (mimblewimble#2716) * forgot to clean the header folder in sandbox in mimblewimble#2685 * Reduce number of unwraps in servers crate (mimblewimble#2707) It doesn't include stratum server which is sufficiently changed in 1.1 branch and adapters, which is big enough for a separate PR. * rustfmt * change version to beta
6paklata · Apr 1, 2019 · 5cb8025 · 5cb8025
1 parent bd6c734
commit 5cb8025
Show file tree

Hide file tree

Showing 115 changed files with 3,590 additions and 1,367 deletions.
diff --git a/.auto-release.sh b/.auto-release.sh
diff --git a/.ci/general-jobs b/.ci/general-jobs
@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+
+# Copyright 2019 The Grin Developers
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script contains general jobs.
+
+case "${CI_JOB}" in
+    "test")
+        for dir in ${CI_JOB_ARGS}; do
+            printf "executing tests in directory \`%s\`...\n" "${dir}"
+            cd "${dir}" && \
+            cargo test --release && \
+            cd - > /dev/null || exit 1
+        done
+        ;;
+esac
diff --git a/.ci/release-jobs b/.ci/release-jobs
@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+
+# Copyright 2019 The Grin Developers
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script contains release-related jobs.
+
+# Redeclare CI and VCP specific environment variables
+# to make future migration to other providers easier.
+readonly JOB_ID="${TRAVIS_JOB_ID}"
+readonly OS_NAME="${TRAVIS_OS_NAME}"
+readonly TEST_RESULT="${TRAVIS_TEST_RESULT}"
+readonly VCP_AUTH_TOKEN="${GITHUB_TOKEN}"
+
+case "${CI_JOB}" in
+    "release")
+        # The release can only be triggered after successful completion of all tests.
+        [[ "${TEST_RESULT}" != 0 ]] && exit 1
+
+        readonly REPO_TAG="$(git describe --tags --exact-match 2> /dev/null || git symbolic-ref -q --short HEAD)"
+
+        case "${OS_NAME}" in
+            "linux")
+                cargo clean && \
+                cargo build --release
+                readonly ARCHIVE_CMD="tar zcf"
+                readonly BIN_SUFFIX=""
+                readonly PKG_NAME="grin-${REPO_TAG}-${JOB_ID}-linux-amd64"
+                readonly PKG_SUFFIX=".tgz"
+                ;;
+
+            "osx")
+                brew update
+                cargo clean && \
+                cargo build --release
+                readonly ARCHIVE_CMD="tar zcf"
+                readonly BIN_SUFFIX=""
+                readonly PKG_NAME="grin-${REPO_TAG}-${JOB_ID}-osx"
+                readonly PKG_SUFFIX=".tgz"
+                ;;
+
+            "windows")
+                cargo clean && \
+                cargo build --release
+                readonly ARCHIVE_CMD="7z a -tzip"
+                readonly BIN_SUFFIX=".exe"
+                readonly PKG_NAME="grin-${REPO_TAG}-${JOB_ID}-win-x64"
+                readonly PKG_SUFFIX=".zip"
+                ;;
+
+            *)
+                printf "Error! Unknown \$OS_NAME: \`%s\`" "${OS_NAME}"
+                exit 1
+        esac
+
+        printf "creating package \`%s\` for the release binary...\n" "${PKG_NAME}${PKG_SUFFIX}"
+
+        cd ./target/release/ || exit 1
+        rm -f -- *"${PKG_SUFFIX}"
+        ${ARCHIVE_CMD} "${PKG_NAME}${PKG_SUFFIX}" "grin${BIN_SUFFIX}"
+        ls -ls -- *.tgz | cut -d' ' -f6-
+        openssl md5 "${PKG_NAME}${PKG_SUFFIX}" > "${PKG_NAME}${PKG_SUFFIX}-md5sum.txt"
+        ls -ls -- *-md5sum.txt | cut -d' ' -f6-
+        cd - > /dev/null || exit 1
+
+        printf "%s package \`%s\` generated\n" "${OS_NAME}" "${PKG_NAME}${PKG_SUFFIX}"
+
+        # Generate changelog only on the Linux platform to avoid duplication.
+        [[ "${OS_NAME}" != "linux" ]] && exit 0
+
+        # Generate CHANGELOG.md
+        readonly REPO_SLUG="mimblewimble/grin"
+        readonly REPO_BRANCH="$(git symbolic-ref -q --short HEAD)"
+        readonly REPO_PREV_RELEASE_TAG="$(git describe --abbrev=0 --tags "$(git rev-list --tags --skip=0 --max-count=1)")"
+
+        gem install github_changelog_generator
+
+        # Needed by github_changelog_generator.
+        export CHANGELOG_GITHUB_TOKEN="${VCP_AUTH_TOKEN}"
+
+        github_changelog_generator \
+          --user "$(cut -d "/" -f1 <<< ${REPO_SLUG})" \
+          --project "$(cut -d "/" -f2 <<< ${REPO_SLUG})" \
+          --since-tag "${REPO_PREV_RELEASE_TAG}"
+
+        readonly CHANGELOG_CONTENT="$(<CHANGELOG.md)"
+
+        # Overwrite CHANGELOG.md with JSON data for release patch.
+        jq --null-input \
+          --arg body "${CHANGELOG_CONTENT}" \
+          --arg name "${REPO_TAG}" \
+          --arg tag_name "${REPO_TAG}" \
+          --arg target_commitish "${REPO_BRANCH}" \
+          '{
+            body: $body,
+            name: $name,
+            tag_name: $tag_name,
+            target_commitish: $target_commitish,
+            draft: false,
+            prerelease: false
+          }' > CHANGELOG.md
+
+        readonly HEADERS="Authorization: token ${VCP_AUTH_TOKEN}"
+        readonly RELEASE_URL="https://api.github.com/repos/${REPO_SLUG}/releases"
+        readonly RELEASE_ID="$(curl -0 --request GET -H "${HEADERS}" "${RELEASE_URL}/tags/${REPO_TAG}" 2> /dev/null | grep id | head -n 1 | sed 's/ *"id": *\(.*\),/\1/')"
+
+        printf "updating release changelog %s for repo: %s, branch: %s, release id: %s\n" "${REPO_TAG}" "${REPO_SLUG}" "${REPO_BRANCH}" "${RELEASE_ID}"
+        curl -H "${HEADERS}" --request PATCH --data @CHANGELOG.md "${RELEASE_URL}/${RELEASE_ID}"
+        printf "changelog uploaded.\n"
+        ;;
+esac
diff --git a/.editorconfig b/.editorconfig
@@ -1,6 +1,3 @@
-# top-most .editorconfig file
-root = true
-
 # use hard tabs for rust source files
 [*.rs]
 indent_style = tab
diff --git a/.travis.yml b/.travis.yml
@@ -1,24 +1,28 @@
-language: rust
-
-git:
-  depth: false
+# Copyright 2019 The Grin Developers
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 
 dist: trusty
 sudo: required
 
-cache:
-  cargo: true
-  timeout: 240
-  directories:
-    - $HOME/.cargo
-    - $TRAVIS_BUILD_DIR/target
-
-before_cache:
-  - rm -rf $TRAVIS_BUILD_DIR/target/tmp
+language: rust
 
 rust:
   - stable
 
+git:
+  depth: false
+
 addons:
   apt:
     sources:
@@ -33,6 +37,13 @@ addons:
       - gcc
       - binutils-dev
 
+cache:
+  cargo: true
+  timeout: 240
+  directories:
+    - $HOME/.cargo
+    - $TRAVIS_BUILD_DIR/target
+
 env:
   global:
     - RUST_BACKTRACE="1"
@@ -41,46 +52,29 @@ env:
 matrix:
   include:
     - os: linux
-      env: TEST_SUITE=servers
+      env: CI_JOB="test"        CI_JOB_ARGS="servers"
+    - os: linux
+      env: CI_JOB="test"        CI_JOB_ARGS="chain core"
     - os: linux
-      env: TEST_SUITE=chain-core
+      env: CI_JOB="test"        CI_JOB_ARGS="pool p2p src"
     - os: linux
-      env: TEST_SUITE=pool-p2p-src
+      env: CI_JOB="test"        CI_JOB_ARGS="keychain"
     - os: linux
-      env: TEST_SUITE=keychain
+      env: CI_JOB="test"        CI_JOB_ARGS="api util store"
     - os: linux
-      env: TEST_SUITE=api-util-store
+      env: CI_JOB="release"     CI_JOB_ARGS=
     - os: osx
-      env: TEST_SUITE=release
+      env: CI_JOB="release"     CI_JOB_ARGS=
 #    - os: windows
-#      env: TEST_SUITE=release
+#      env: CI_JOB="release"     CI_JOB_ARGS=
 
-script:
-  - IFS='-' read -r -a DIRS <<< "$TEST_SUITE"; DIR=${DIRS[0]};
-    echo "start testing on folder $DIR...";
-    if [[ -n "$DIR" ]] && [[ "$TRAVIS_OS_NAME" == "linux" ]];  then cd $DIR && cargo test --release && cd - > /dev/null; fi;
-  - IFS='-' read -r -a DIRS <<< "$TEST_SUITE"; DIR=${DIRS[1]};
-    if [[ -n "$DIR" ]];  then
-      echo "start testing on folder $DIR...";
-      cd $DIR && cargo test --release && cd - > /dev/null;
-    fi;
-  - IFS='-' read -r -a DIRS <<< "$TEST_SUITE"; DIR=${DIRS[2]};
-    if [[ -n "$DIR" ]];  then
-      echo "start testing on folder $DIR...";
-      cd $DIR && cargo test --release && cd - > /dev/null;
-    fi;
+script: .ci/general-jobs
+
+before_cache:
+  - rm -rf $TRAVIS_BUILD_DIR/target/tmp
 
 before_deploy:
-  - if [[ "$TEST_SUITE" == "pool-p2p-src" ]]; then
-      cargo clean && cargo build --release && ./.auto-release.sh;
-    fi
-  - if [[ "$TEST_SUITE" == "release" ]]  && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
-      brew update;
-      cargo clean && cargo build --release && ./.auto-release.sh;
-    fi
-  - if [[ "$TEST_SUITE" == "release" ]]  && [[ "$TRAVIS_OS_NAME" == "windows" ]]; then
-      cargo clean && cargo build --release && ./.auto-release.sh;
-    fi
+  - bash .ci/release-jobs
 
 deploy:
   provider: releases