Merge pull request iree-org#2633 from rsuderman:main-to-google

PiperOrigin-RevId: 322846874

.github/workflows/publish_docs.yml

@@ -33,9 +33,6 @@ jobs:
         uses: actions/checkout@v2
         with:
           token: ${{ secrets.GITHUB_WRITE_ACCESS_TOKEN }}
-      - name: Fetching gh-pages branch
-        run: |
-          git fetch origin gh-pages
       - name: Initializing submodules
         run: ./scripts/git/submodule_versions.py init
       - name: Installing Ninja build
@@ -45,6 +42,9 @@ jobs:
           ./build_tools/cmake/build_docs.sh
           # Patch the MarkDown files with front matter for rendering
           ./scripts/prepare_doc_publication.py ${IREE_DOC_BUILD_DIR}/doc
+      - name: Fetching gh-pages branch
+        run: |
+          git fetch origin gh-pages
       - name: Updating gh-pages branch
         run: |
           git checkout -f gh-pages

CONTRIBUTING.md

@@ -58,7 +58,7 @@ for them (e.g. because they don't have write access).
 ## Peculiarities
 
 Our documentation on
-[repository management](https://github.com/google/iree/blob/main/docs/repository_management.md)
+[repository management](https://github.com/google/iree/blob/main/docs/developing_iree/repository_management.md)
 has more information on some of the oddities in our repository setup and
 workflows. For the most part, these should be transparent to normal developer
 workflows.

README.md

@@ -37,8 +37,8 @@ For development, IREE supports both Bazel and CMake on Windows and Linux. We are
 working on enabling macOS support. For deployment, IREE aims to additionally
 cover Android and iOS.
 
-Please see the [Getting Started](https://google.github.io/iree/GetStarted) pages
-on IREE's [documentation hub](https://google.github.io/iree) to configure,
+Please see the [Getting Started](https://google.github.io/iree/get_started)
+pages on IREE's [documentation hub](https://google.github.io/iree) to configure,
 compile, and run IREE in your favorite development environment!
 
 ## Documentation and Talks
@@ -68,7 +68,7 @@ hardware/API-specific binaries like SPIR-V.
 
 The architecture of IREE is best illustrated by the following picture:
 
-![IREE Architecture](./docs/IREE-Architecture.svg)
+![IREE Architecture](./docs/iree_architecture.svg)
 
 Being compilation-based means IREE does not have a traditional runtime that
 dispatches "ops" to their fat kernel implementations. What IREE provides is a
@@ -100,8 +100,8 @@ IREE aims to
 ## Roadmap and Milestones
 
 IREE is still at its early stage; we have lots of exciting future plans. Please
-check out the [long-term design roadmap](./docs/roadmap_design.md) and
-[short-term focus areas](./docs/roadmap.md).
+check out the [long-term design roadmap](./docs/design_roadmap.md) and
+[short-term focus areas](./docs/milestones.md).
 
 We use [GitHub Projects](https://github.com/google/iree/projects) to track
 various IREE components and

build_tools/cmake/build_docs.sh

@@ -49,34 +49,12 @@ ninja iree-doc
 
 cd ${ROOT_DIR?}
 
+cp README.md ${BUILD_DIR}/doc/index.md
+cp -rf docs/* ${BUILD_DIR}/doc/
+
 # Update op_coverage.md
 scripts/update_op_coverage.py ${BUILD_DIR}
 
 # Update e2e_coverage.md
 PYTHON_BIN=`which python3` scripts/update_e2e_coverage.py ${BUILD_DIR}
 
-# Copy a curated list of docs to publish. This is expected to cover all docs
-# under docs/ after they are refreshed.
-
-cp README.md ${BUILD_DIR}/doc/index.md
-cp docs/IREE-Architecture.svg ${BUILD_DIR}/doc/
-
-cp docs/roadmap.md ${BUILD_DIR}/doc/
-cp docs/roadmap_design.md ${BUILD_DIR}/doc/
-cp docs/developer_overview.md ${BUILD_DIR}/doc/
-cp docs/testing_guide.md ${BUILD_DIR}/doc/
-cp docs/iree_community.md ${BUILD_DIR}/doc/
-
-mkdir -p ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_windows_bazel.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_windows_cmake.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_windows_vulkan.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_linux_bazel.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_linux_cmake.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_linux_vulkan.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_macos_bazel.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_macos_cmake.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_android_cmake.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/getting_started_python.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/generic_vulkan_env_setup.md ${BUILD_DIR}/doc/GetStarted/
-cp docs/GetStarted/cmake_options_and_variables.md ${BUILD_DIR}/doc/GetStarted/

build_tools/cmake/run_android_test.sh

@@ -35,7 +35,7 @@
 set -x
 set -e
 
-adb push $TEST_EXECUTABLE $TEST_ANDROID_ABS_DIR/$(basename $TEST_EXECUTABLE)
+adb push $TEST_EXECUTABLE $TEST_ANDROID_ABS_DIR/$(basename $TEST_EXECUTABLE) 1>/dev/null
 
 if [ -n "$TEST_DATA" ]; then
   adb push $TEST_DATA $TEST_ANDROID_ABS_DIR/$(basename $TEST_DATA)

colab/README.md

@@ -1,7 +1,7 @@
 # Google Colaboratory (Colab) Notebooks
 
 To run these notebooks with a local runtime, refer to the
-[Using Colab docs](../docs/using_colab.md).
+[Using Colab docs](../docs/using_iree/using_colab.md).
 
 Hosted/remote runtimes are not yet supported.
 

docs/function_abi.md → docs/design_docs/function_abi.md

@@ -1,4 +1,4 @@
-# Function signatures
+# Function Signatures
 
 A key job of the IREE compiler and runtime is capturing function call semantics
 from the originating system and providing mechanisms so that invocations can be

docs/simple_ir_walkthrough.md → docs/design_docs/simple_ir_walkthrough.md

@@ -1,5 +1,7 @@
 # Simple IR Walkthrough
 
+Note that this doc is quite outdated. We expect to update it soon.
+
 ## Overview
 
 This walks through the process of lowering TensorFlow python to an IREE module,

docs/contributor_tips.md → docs/developing_iree/contributor_tips.md

@@ -2,11 +2,11 @@
 
 This is an opinionated guide documenting workflows that some members of the team
 have found useful. It is focused on meta-tooling, not on IREE code specifically
-(you will find the latter in the [Developer Overview](../developer_overview.md))
-It is certainly possible to use workflows other than these, but some common
-tasks, especially for maintainers will likely be made easier if you use these
-flows. It assumes a basic knowledge of `git` and GitHub and suggests some
-specific ways of using it.
+(you will find the latter in the [Developer Overview](developer_overview.md)) It
+is certainly possible to use workflows other than these, but some common tasks,
+especially for maintainers will likely be made easier if you use these flows. It
+assumes a basic knowledge of `git` and GitHub and suggests some specific ways of
+using it.
 
 ## Git Structure
 

docs/developer_overview.md → docs/developing_iree/developer_overview.md

@@ -150,7 +150,7 @@ The `iree-check-module` program takes an already translated IREE module as input
 and executes it as a series of
 [googletest](https://github.com/google/googletest) tests. This is the test
 runner for the IREE
-[check framework](https://github.com/google/iree/tree/main/docs/testing_guide.md#end-to-end-tests).
+[check framework](https://github.com/google/iree/tree/main/docs/developing_iree/testing_guide.md#end-to-end-tests).
 
 ```shell
 $ bazel run iree/tools:iree-translate -- \
@@ -207,6 +207,28 @@ These flags can control IREE tool output verbosity. `--iree_minloglevel` and
 accept a number where 0, 1, 2, 3 stands for info, warning, error, and fatal
 error respectively.
 
+#### Read inputs from a file
+
+All the IREE tools support reading input values from a file. This is quite
+useful for debugging. Use `-help` for each tool to see what the flag to set. The
+inputs are expected to be newline-separated. Each input should be either a
+scalar or a buffer. Scalars should be in the format `type=value` and buffers
+should be in the format `[shape]xtype=[value]`. For example:
+
+```
+1x5xf32=1,-2,-3,4,-5
+1x5x3x1xf32=15,14,13,12,11,10,9,8,7,6,5,4,3,2,1
+```
+
+#### `iree-flow-trace-dispatch-tensors`
+
+This flag will enable tracing inputs and outputs for each dispatch function. It
+is easier to narrow down test cases, since IREE breaks a ML workload into
+multiple dispatch function. When the flag is on, IREE will insert trace points
+before and after each dispatch function. The first trace op is for inputs, and
+the second trace op is for outputs. There will be two events for one dispatch
+function.
+
 ### Useful Vulkan driver flags
 
 For IREE's Vulkan runtime driver, there are a few useful

docs/testing_guide.md → docs/developing_iree/testing_guide.md

@@ -30,7 +30,7 @@ $ bazel test iree/base:arena_test
 To use the Vulkan backend as test driver, you may need to select between a
 Vulkan implementation from SwiftShader and multiple Vulkan-capable hardware
 devices. This can be done via environment variables. See the
-[generic Vulkan setup](GetStarted/generic_vulkan_env_setup.md#useful-environment-variables)
+[generic Vulkan setup](get_started/generic_vulkan_env_setup.md#useful-environment-variables)
 page for details regarding these variables.
 
 For Bazel, you can persist the configuration in `user.bazelrc` to save typing.

docs/GetStarted/getting_started_linux_bazel.md → docs/get_started/getting_started_linux_bazel.md

@@ -123,7 +123,7 @@ $ ./bazel-bin/iree/tools/iree-run-mlir ./iree/tools/test/simple.mlir \
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md)
+    [Developer Overview](../developing_iree/developer_overview.md)
 *   To target GPUs using Vulkan, see
     [Getting Started on Linux with Vulkan](getting_started_linux_vulkan.md)
 *   To use IREE's Python bindings, see

docs/GetStarted/getting_started_linux_cmake.md → docs/get_started/getting_started_linux_cmake.md

@@ -110,7 +110,7 @@ $ ./build/iree/tools/iree-run-mlir $PWD/iree/tools/test/simple.mlir \
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md)
+    [Developer Overview](../developing_iree/developer_overview.md)
 *   To target GPUs using Vulkan, see
     [Getting Started on Linux with Vulkan](getting_started_linux_vulkan.md)
 *   To use IREE's Python bindings, see

docs/GetStarted/getting_started_macos_bazel.md → docs/get_started/getting_started_macos_bazel.md

@@ -126,8 +126,8 @@ $ ./bazel-bin/iree/tools/iree-run-mlir ./iree/tools/test/simple.mlir \
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md) <!-- TODO: Link to macOS
-    versions of these guides once they are developed.
+    [Developer Overview](../developing_iree/developer_overview.md) <!-- TODO:
+    Link to macOS versions of these guides once they are developed.
 *   To target GPUs using Vulkan, see
     [Getting Started on Linux with Vulkan](getting_started_linux_vulkan.md)
 *   To use IREE's Python bindings, see

docs/GetStarted/getting_started_macos_cmake.md → docs/get_started/getting_started_macos_cmake.md

@@ -110,8 +110,8 @@ $ ./build/iree/tools/iree-run-mlir $PWD/iree/tools/test/simple.mlir \
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md) <!-- TODO: Link to macOS
-    versions of these guides once they are developed.
+    [Developer Overview](../developing_iree/developer_overview.md) <!-- TODO:
+    Link to macOS versions of these guides once they are developed.
 *   To target GPUs using Vulkan, see
     [Getting Started on Linux with Vulkan](getting_started_linux_vulkan.md)
 *   To use IREE's Python bindings, see

docs/GetStarted/getting_started_windows_bazel.md → docs/get_started/getting_started_windows_bazel.md

@@ -118,7 +118,7 @@ and execute a function in the compiled module:
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md)
+    [Developer Overview](../developing_iree/developer_overview.md)
 *   To target GPUs using Vulkan, see
     [Getting Started on Windows with Vulkan](getting_started_windows_vulkan.md)
 *   To use IREE's Python bindings, see

docs/GetStarted/getting_started_windows_cmake.md → docs/get_started/getting_started_windows_cmake.md

@@ -107,7 +107,7 @@ and execute a function in the compiled module:
 ### Further Reading
 
 *   For an introduction to IREE's project structure and developer tools, see
-    [Developer Overview](../developer_overview.md)
+    [Developer Overview](../developing_iree/developer_overview.md)
 *   To target GPUs using Vulkan, see
     [Getting Started on Windows with Vulkan](getting_started_windows_vulkan.md)
 *   To use IREE's Python bindings, see

docs/roadmap.md → docs/milestones.md

@@ -1,11 +1,11 @@
-# IREE Roadmap
+# IREE Milestones
 
 ## Design
 
 Though many of the core dialects are now in place enough for correctness testing
 a large majority of the features we are most excited to demonstrate are still
 TODO and will be coming over the next few quarters. You can find a highlighted
-set of coming features in the [design roadmap](roadmap_design.md).
+set of coming features in the [design roadmap](design_roadmap.md).
 
 ## Spring/Summer 2020 Focus Areas
 
@@ -37,7 +37,7 @@ amount of glue mostly ready to accept it.
 ### HAL: Marl CPU Scheduling
 
 We want to plug in [marl](https://github.com/google/marl) to provide
-[CPU-side work scheduling](roadmap_design.md#gpu-like-cpu-scheduling) that
+[CPU-side work scheduling](design_roadmap.md#gpu-like-cpu-scheduling) that
 matches GPU semantics. This will enable improved CPU utilization and allow us to
 verify the approach with benchmarks.