Skip to content
This repository has been archived by the owner on Jul 1, 2024. It is now read-only.

Commit

Permalink
v0.5.0 beta rc0 with OV2021.3 and TF2.4.1 (#66)
Browse files Browse the repository at this point in the history 
* Using set Blob instead of Get Blob

* Upgrade to 2021.3

* Removed ngraph/chrome_trace.hpp

* Remove / rename occurrences of Add-On in code and documentation

* Build gtest with correct ABI, TF >r2.2 file struct changes, general build options

* Add whl download with personal token; Set resource util ratio for TF build

* update ci tensorflow setup, tf version change in documentation & code

* update tf 2.4.1 ci test patch

* rename project in cmake

* change openvino version in ci yml

* Fixed bug in encapsulate_op

* OCM submodule updated for OV2021.3

* Added backward compatibiliby with 2021.2

* SetBlob support in VAD-M Engine for OV 2021.3

* Condition corrected for OV 2021.2 compatibility

* update ocm to master

* assert output directory in build_tf, remove static ov versioning

* replace unsafe string functions, initialize some vars

* remove tf patch, assert non zero batch size

* nullptr check for output blobs

* Added names space to error types in VADMEngine

* Switching back to GetBlob for VAD-M

* nullptr check, add namespace, GetBlob for VAD-M

* nullptr check for output blobs

* Added names space to error types in VADMEngine

* Switching back to GetBlob for VAD-M

Co-authored-by: Mustafa Cavus <[email protected]>

* Model sources added for sprint models (#44)

* Model sources added for sprint models

* Links converted into hyperlinks

* Adding sources for the tf models

* Models are grouped based on the library/source

* Correction on the link of resnet 50 v1.5

* Some links are updated to the model's addresses instead of the library's main page. Instructions are added for some models to guide how to use models.

* Instructions for using model is updated to be more clear.

* Additional instructions are added for model usage.

* Sentence saying that which devices are supported is removed since it already include all devices.

The sentence was saying cpu, gpu, and myriad are supported. But it already includes all the models so we can remove it.

* Model list converted into tables

* Mobilenet V2 and FC DenseNet 103 are added

* Update MODELS.md

* Correction for se-resnext-50 link

Co-authored-by: bhadur <[email protected]>

* adding Object detection sample for yolo v3 model (#57)

* adding python object detection sample

* Object detection for Yolo V3 TF model

* Update ReadMe for object detection sample and pre-built packages

* Readme and code fixes

* update output result

* updating openvino_tensorflow to ovtf

* updated the build instructions for c++ example

* Fixed one of the headings

Co-authored-by: Chandrakant Khandelwal <[email protected]>

* Disabling additional ops for batching

* Environment variable descriptions are added to doc (#62)

* Environment variable descriptions are added to doc

* Typo corrections/updates on descriptions

* correct assert comparator

* docker manylinux builds (#65)

* add dockerfiles, take the right py version in setup.in.py

* make whl platlib compliant for auditwheel

* add python executable, abi0 and abi1 builds in dockerfile, proper py abi tagging in whl

* make prebuilt tf default, remove unwanted tf import in setup.in.py

* complete manylinux dockerfile to build py3.x release packages

* add flag for abi1 build

* add pypi readme doc

* ABI1 build requires latest numpy

* add env paths for py37, py38

* update tf_version for tf source builds

* don't run auditwheel, static manylinux tagging

* transpose sinking fix (#68)

* Modified op translations for Unpack and Fill op

* Fixed transpose sinking for binary operators

Co-authored-by: ynimmaga <[email protected]>

* Added op translation for FusedConv2D_BatchNormLeakyRelu (#69)

* Node limit update based on avg nodes (#70)

* Node limit update based on avg nodes

* Dyn min node limit fix for efficientnet

Co-authored-by: Mustafa Cavus <[email protected]>

* refactor main doc, add build options doc, manylinux2014 dockerfile instruction (#71)

* refactor main doc, add build options doc

* change cmake version in code

* disable tests that were tagged as defects

* * Commented one depthwiseconv2d test, as input params needs further analysis
* Modified readme for build from source python versions

Co-authored-by: suryasidd <[email protected]>
Co-authored-by: Chandrakant Khandelwal <[email protected]>
Co-authored-by: Mustafa Cavus <[email protected]>
Co-authored-by: Mustafa Cavus <[email protected]>
Co-authored-by: bhadur <[email protected]>
Co-authored-by: Sai Jayanthi <[email protected]>
Co-authored-by: ynimmaga <[email protected]>
  • Loading branch information
@sspintel @suryasidd
@ck-intel @cavusmustafa @bhadur @saipj @ynimmaga
8 people authored Apr 15, 2021
1 parent 4339cce commit 69afa83
Show file tree
Hide file tree
Showing 57 changed files with 1,316 additions and 471 deletions.
27 changes: 16 additions & 11 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Copyright (C) 2021 Intel Corporation

# SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.4)
cmake_minimum_required(VERSION 3.14)

project (ngraph_tensorflow_bridge CXX)
project (openvino_tensorflow CXX)

# set directory where the custom finders live
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake")
Expand Down Expand Up @@ -138,10 +138,8 @@ option(OPENVINO_ARTIFACTS_DIR "Where would OpenVINO be installed after build" FA
set(InferenceEngine_DIR ${OPENVINO_ARTIFACTS_DIR}/deployment_tools/inference_engine/share)
find_package(InferenceEngine REQUIRED)

if (USE_OPENVINO_FROM_LOCATION)
set(ngraph_DIR ${OPENVINO_ARTIFACTS_DIR}/deployment_tools/ngraph/cmake)
find_package(ngraph REQUIRED)
endif()
set(ngraph_DIR ${OPENVINO_ARTIFACTS_DIR}/deployment_tools/ngraph/cmake)
find_package(ngraph QUIET)

include_directories(${InferenceEngine_INCLUDE_DIRS})
if (THREADING STREQUAL "TBB" OR THREADING STREQUAL "TBB_AUTO")
Expand Down Expand Up @@ -171,14 +169,21 @@ endif()
message(STATUS "UNIT_TEST_ENABLE: ${UNIT_TEST_ENABLE}")
message(STATUS "OPENVINO_ARTIFACTS_DIR: ${OPENVINO_ARTIFACTS_DIR}")
message(STATUS "USE_PRE_BUILT_OPENVINO: ${USE_PRE_BUILT_OPENVINO}")

message(STATUS "OPENVINO_VERSION: ${OPENVINO_VERSION}")
if (${OPENVINO_VERSION} MATCHES "2021.3")
add_definitions(-DOPENVINO_2021_3=1)
elseif (${OPENVINO_VERSION} MATCHES "2021.2")
add_definitions(-DOPENVINO_2021_2=1)
else()
message(FATAL_ERROR "Unsupported OpenVINO version: ${OPENVINO_VERSION")
endif()
if(OS_VERSION STREQUAL "\"centos\"")
set(LIB "lib64")
else()
set(LIB "lib")
endif()

if(USE_OPENVINO_FROM_LOCATION)
if(ngraph_FOUND)
set(NGRAPH_INSTALL_DIR ${OPENVINO_ARTIFACTS_DIR}/deployment_tools/ngraph/)
else()
set(NGRAPH_INSTALL_DIR ${OPENVINO_ARTIFACTS_DIR})
Expand Down Expand Up @@ -215,7 +220,7 @@ if (DEFINED TF_SRC_DIR)
add_subdirectory(examples/classification_sample)
else()
message(
STATUS
STATUS
"TensorFlow source directory not provided. "
"C++ Examples won't be built"
)
Expand All @@ -231,7 +236,7 @@ if (UNIT_TEST_ENABLE)
else()
if (NOT EXISTS ${TensorFlow_SRC_DIR})
message(
STATUS
STATUS
"TensorFlow source directory doesn't exist"
)
endif()
Expand Down
97 changes: 50 additions & 47 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,84 +13,87 @@ This repository contains the source code of Intel<sup>®</sup> OpenVINO™ integ
**Check version numbers**
|Using pre-built packages| Building from source|
| -----------------------|-------------------|
|Python 3.6| Python 3.6|
|TensorFlow v2.2.2|GCC 7.5 (Ubuntu 18.04)|
| |cmake 3.15 or higher|
| |Bazelisk|
|Python 3.6, 3.7, or 3.8| Python 3.6, 3.7, or 3.8|
|TensorFlow v2.4.1|GCC 7.5 (Ubuntu 18.04)|
| |cmake >= 3.14|
| |Bazelisk v1.7.5|
| |virtualenv 16.0.0+|
| |patchelf|
| |patchelf 0.9|

### Use pre-built packages
### Use Pre-Built Packages

Intel<sup>®</sup> OpenVINO™ integration with TensorFlow has two releases: one built with CXX11_ABI=0 and another built with CXX11_ABI=1. TensorFlow packages available in PyPi are built with CXX11_ABI=0 and OpenVINO release packages are built with CXX11_ABI=1. To prevent ABI incompatibilities, we provide both packages built with CXX11_ABI=0 and CXX11_ABI=1.
Intel<sup>®</sup> OpenVINO™ integration with TensorFlow has two releases: one built with CXX11_ABI=0 and another built with CXX11_ABI=1. Since TensorFlow packages available in PyPi are built with CXX11_ABI=0 and OpenVINO release packages are built with CXX11_ABI=1, binary releases of these packages cannot be installed together and used as is. Based on your needs, you can choose one of the two available methods:

#### - Package built with CXX11_ABI=0
- Use Intel<sup>®</sup> OpenVINO™ integration with TensorFlow alongside PyPi TensorFlow
(CXX11_ABI=0, no OpenVINO installation required, disables VAD-M support)

- Use Intel<sup>®</sup> OpenVINO™ integration with TensorFlow alongside the Intel® Distribution of OpenVINO™ Toolkit
(CXX11_ABI=1, needs a custom TensorFlow package, enables VAD-M support)

Intel<sup>®</sup> OpenVINO™ integration with TensorFlow package built with ABI=0 is compatible with PyPi TensorFlow package 2.2.2. This OpenVINO integration with TensorFlow package comes with prebuilt libraries of OpenVINO version 2021.2. The users do not have to install OpenVINO separately. This package supports Intel CPUs, Intel integrated GPUs and Intel<sup>®</sup> Movidius<sup>TM</sup> Vision Processing Units (VPUs).
#### - Use Intel<sup>®</sup> OpenVINO™ integration with TensorFlow alongside PyPi TensorFlow

Users can use TensorFlow 2.2.2 from PyPi (`pip install -U tensorflow==2.2.2`). However, TensorFlow 2.2.2 package from PyPi does not have the latest security patches. We provide a ready-to-use TensorFlow package built with security patches using CXX11_ABI=0 and recommend users to use it to avoid any security issues.


Below are the steps needed to use the packages built with CXX11_ABI=0
This OpenVINO integration with TensorFlow package comes with pre-built libraries of OpenVINO version 2021.3. The users do not have to install OpenVINO separately. This package supports Intel CPUs, Intel integrated GPUs and Intel<sup>®</sup> Movidius<sup>TM</sup> Vision Processing Units (VPUs).

1. Ensure the following pip version is being used:

pip install --upgrade pip==21.0.1

2. Install `TensorFlow`:

pip install -U tensorflow==2.2.2
(or)

pip install tensorflow_security_patched-2.2.2-cp36-cp36m-linux_x86_64.whl (Recommended for security patches)

Download ([**tensorflow_security_patched-2.2.2-cp36-cp36m-linux_x86_64.whl**](https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/tensorflow_security_patched-2.2.2-cp36-cp36m-linux_x86_64.whl))
pip install -U tensorflow==2.4.1

3. Install `openvino-tensorflow`:

pip install openvino_tensorflow-0.5.0-py2.py3-none-manylinux2010_x86_64.whl

Download ([**openvino_tensorflow-0.5.0-py2.py3-none-manylinux2010_x86_64.whl**](https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/openvino_tensorflow-0.5.0-py2.py3-none-manylinux2010_x86_64.whl))
#### - Package built with CXX11_ABI=1
pip install openvino-tensorflow

Intel<sup>®</sup> OpenVINO™ integration with TensorFlow package built with ABI=1 is compatible with OpenVINO binary releases. This OpenVINO integration with TensorFlow package is currently compatible with OpenVINO version 2021.2. This package supports Intel CPUs, Intel integrated GPUs, Intel<sup>®</sup> Movidius<sup>TM</sup> Vision Processing Units (VPUs) and Intel<sup>®</sup> Vision Accelerator Design with Movidius<sup>TM</sup> (VAD-M).
#### - Use Intel<sup>®</sup> OpenVINO™ integration with TensorFlow alongside the Intel® Distribution of OpenVINO™ Toolkit

Users can build TensorFlow from source with CXX11_ABI=1 or they can use the TensorFlow package that we provide. We provide a ready-to-use TensorFlow package built with security patches using CXX11_ABI=1 and recommend users to use it to avoid any security issues.
This OpenVINO integration with TensorFlow package is currently compatible with OpenVINO version 2021.3. This package supports Intel CPUs, Intel integrated GPUs, Intel<sup>®</sup> Movidius<sup>TM</sup> Vision Processing Units (VPUs) and Intel<sup>®</sup> Vision Accelerator Design with Movidius<sup>TM</sup> (VAD-M).


Below are the steps needed to use the packages built with CXX11_ABI=1
Users can build TensorFlow from source with -D_GLIBCXX_USE_CXX11_ABI=1 or they can use the TensorFlow package that we provide below.

1. Ensure the following pip version is being used:

pip install --upgrade pip==21.0.1

2. Install `TensorFlow`:
2. Install `TensorFlow`. Based on your Python version, use the appropriate package below:

pip3.6 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/tensorflow_abi1-2.4.1-cp36-cp36m-manylinux2010_x86_64.whl

or

pip3 install tensorflow_security_patched_abi1-2.2.2-cp36-cp36m-linux_x86_64.whl
pip3.7 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/tensorflow_abi1-2.4.1-cp37-cp37m-manylinux2010_x86_64.whl

Download([**tensorflow_security_patched_abi1-2.2.2-cp36-cp36m-linux_x86_64.whl**](https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/tensorflow_security_patched_abi1-2.2.2-cp36-cp36m-linux_x86_64.whl))
or

3. Install the OpenVINO 2021.2 release along with its dependencies from ([https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit)).
pip3.8 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/tensorflow_abi1-2.4.1-cp38-cp38-manylinux2010_x86_64.whl

3. Download & install Intel® Distribution of OpenVINO™ Toolkit 2021.3 release along with its dependencies from ([https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit)).

4. Initialize the OpenVINO environment by running the `setupvars.sh` present in <code>\<openvino\_install\_directory\>\/bin</code> using the below command:

source setupvars.sh

3. Install `openvino-tensorflow`:
3. Install `openvino-tensorflow`. Based on your Python version, use the appropriate package below:

pip3.6 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/openvino_tensorflow_abi1-0.5.0-cp36-cp36m-manylinux2014_x86_64.whl

or

pip3.7 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/openvino_tensorflow_abi1-0.5.0-cp37-cp37m-manylinux2014_x86_64.whl

pip install openvino_tensorflow_abi1-0.5.0-py2.py3-none-manylinux2010_x86_64.whl
or

Download ([**openvino_tensorflow_abi1-0.5.0-py2.py3-none-manylinux2010_x86_64.whl**](https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/openvino_tensorflow_abi1-0.5.0-py2.py3-none-manylinux2010_x86_64.whl))
pip3.8 install https://github.com/openvinotoolkit/openvino_tensorflow/releases/download/v0.5.0/openvino_tensorflow_abi1-0.5.0-cp38-cp38-manylinux2014_x86_64.whl

#### - Summary of the Prebuilt packages



|TensorFlow Package| OpenVINO integration with TensorFlow Package|Supported OpenVINO Flavor|Supported Hardware Backends|Comments|
| -----------------|-----------------------------------|----------------------------|---------------------------|----------------|
|tensorflow-security-patched| openvino-tensorflow|OpenVINO built from source|CPU,GPU,MYRIAD|OpenVINO libraries are built from source and included in the wheel package|
|tensorflow-security-patched-abi1| openvino-tensorflow-abi1|Links to OpenVINO binary release|CPU,GPU,MYRIAD,HDDL|OpenVINO integration with TensorFlow libraries links to OpenVINO binaries|
|tensorflow| openvino-tensorflow|OpenVINO built from source|CPU,GPU,MYRIAD|OpenVINO libraries are built from source and included in the wheel package|
|tensorflow-abi1| openvino-tensorflow-abi1|Links to OpenVINO binary release|CPU,GPU,MYRIAD,VAD-M|OpenVINO integration with TensorFlow libraries links to OpenVINO binaries|


### Build from source
Expand All @@ -102,35 +105,33 @@ Once TensorFlow and its dependencies are installed, clone the `openvino_tensorfl
git submodule init
git submodule update --recursive

Run the following Python script to build TensorFlow, OpenVINO, and the OpenVINO integration with TensorFlow. Use Python 3
Running the following Python script will build OpenVINO integration with TensorFlow with PyPi TensorFlow, and OpenVINO sources

python3 build_ovtf.py

When the build finishes, a new `virtualenv` directory is created in `build_cmake/venv-tf-py3`. Build artifacts (i.e., the `openvino_tensorflow-<VERSION>-py2.py3-none-manylinux1_x86_64.whl`) are created in the `build_cmake/artifacts` directory.
When the build finishes, a new `virtualenv` directory is created in `build_cmake/venv-tf-py3`. Build artifacts (ex: the `openvino_tensorflow-<VERSION>-cp36-cp36m-manylinux2014_x86_64.whl`) are created in the `build_cmake/artifacts/` directory.

For more build options:
Select the help option of `build_ovtf.py` script to learn more about various build options.

python3 build_ovtf.py --help

To use `openvino-tensorflow`, activate the following `virtualenv` to start using OpenVINO with TensorFlow.
To use `openvino-tensorflow`, activate the following `virtualenv` to start using OpenVINO integration with TensorFlow.

source build_cmake/venv-tf-py3/bin/activate

Alternatively, you can also install the TensorFlow and OpenVINO integration with TensorFlow outside of a `virtualenv`. The Python `whl` files are located in the `build_cmake/artifacts/` and `build_cmake/artifacts/tensorflow` directories, respectively.

Select the help option of `build_ovtf.py` script to learn more about various build options.

Verify that `openvino-tensorflow` installed correctly:
Verify that `openvino-tensorflow` is installed correctly:

python -c "import tensorflow as tf; print('TensorFlow version: ',tf.__version__);\
import openvino_tensorflow; print(openvino_tensorflow.__version__)"

This will produce something like this:

TensorFlow version: 2.2.2
TensorFlow version: 2.4.1
OpenVINO integration with TensorFlow version: b'0.5.0'
OpenVINO version used for this build: b'2021.2'
TensorFlow version used for this build: v2.2.2
OpenVINO version used for this build: b'2021.3'
TensorFlow version used for this build: v2.4.1
CXX11_ABI flag used for this build: 1
OpenVINO integration with TensorFlow built with Grappler: False

Expand All @@ -144,6 +145,8 @@ Test the installation:

This command runs all C++ and Python unit tests from the `openvino_tensorflow` source tree. It also runs various TensorFlow Python tests using OpenVINO.

For more advanced build configurations, please refer to: ([OpenVINO integration with TensorFlow - Builds](https://github.com/openvinotoolkit/openvino_tensorflow/blob/master/docs/README.md))

## Usage

Once you have installed Intel<sup>®</sup> OpenVINO™ integration with TensorFlow, you can use TensorFlow to run inference using a trained model.
Expand Down
4 changes: 2 additions & 2 deletions build_ov.py
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# ==============================================================================
# Copyright (C) 2021 Intel Corporation

# SPDX-License-Identifier: Apache-2.0
# ==============================================================================

Expand All @@ -11,7 +11,7 @@


def main():
openvino_version = "releases/2021/2"
openvino_version = "releases/2021/3"
build_dir = 'build_cmake'
cxx_abi = "1"
print("openVINO version: ", openvino_version)
Expand Down
Loading

0 comments on commit 69afa83

Please sign in to comment.