Skip to content

Commit

Permalink
DOCS: Model Conversion Tutorials fix - Port to master (openvinotoolki…
Browse files Browse the repository at this point in the history
  • Loading branch information
msmykx-intel authored Nov 9, 2022
1 parent e043dcb commit 3127c28
Show file tree
Hide file tree
Showing 9 changed files with 192 additions and 179 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ BERT-NER model repository. The model with configuration files is stored in the `

To convert the model to ONNX format, create and run the following script in the root
directory of the model repository. If you download the pretrained model, you need
to download [`bert.py`](https://github.com/kamalkraj/BERT-NER/blob/dev/bert.py) to run the script.
to download [bert.py](https://github.com/kamalkraj/BERT-NER/blob/dev/bert.py) to run the script.
The instructions were tested with the commit-SHA: `e5be564156f194f1becb0d82aeaf6e762d9eb9ed`.

```python
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ mkdir rnnt_for_openvino
cd rnnt_for_openvino
```

**Step 3**. Download pretrained weights for PyTorch implementation from [https://zenodo.org/record/3662521#.YG21DugzZaQ](https://zenodo.org/record/3662521#.YG21DugzZaQ).
**Step 3**. Download pretrained weights for PyTorch implementation from [here](https://zenodo.org/record/3662521#.YG21DugzZaQ).
For UNIX-like systems, you can use `wget`:
```bash
wget https://zenodo.org/record/3662521/files/DistributedDataParallel_1576581068.9962234-epoch-100.pt
```
The link was taken from `setup.sh` in the `speech_recoginitin/rnnt` subfolder. You will get exactly the same weights as
if you were following the guide from [https://github.com/mlcommons/inference/tree/master/speech_recognition/rnnt](https://github.com/mlcommons/inference/tree/master/speech_recognition/rnnt).
if you were following the [guide](https://github.com/mlcommons/inference/tree/master/speech_recognition/rnnt).

**Step 4**. Install required Python packages:
```bash
Expand Down Expand Up @@ -103,4 +103,4 @@ mo --input_model rnnt_encoder.onnx --input "input[157 1 240],feature_length->157
mo --input_model rnnt_prediction.onnx --input "symbol[1 1],hidden_in_1[2 1 320],hidden_in_2[2 1 320]"
mo --input_model rnnt_joint.onnx --input "0[1 1 1024],1[1 1 320]"
```
> **NOTE**: The hardcoded value for sequence length = 157 was taken from the MLCommons, but conversion to IR preserves network [reshapeability](../../../../OV_Runtime_UG/ShapeInference.md). Therefore, input shapes can be changed manually to any value during either conversion or inference.
> **NOTE**: The hardcoded value for sequence length = 157 was taken from the MLCommons, but conversion to IR preserves network [reshapeability](@ref openvino_docs_OV_UG_ShapeInference). Therefore, input shapes can be changed manually to any value during either conversion or inference.
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Pretrained models for BERT (Bidirectional Encoder Representations from Transformers) are
[publicly available](https://github.com/google-research/bert).

## <a name="supported_models"></a>Supported Models
## <a name="supported-models"></a>Supported Models

The following models from the pretrained [BERT model list](https://github.com/google-research/bert#pre-trained-models) are currently supported:

Expand Down Expand Up @@ -43,7 +43,7 @@ Pretrained models are not suitable for batch reshaping out-of-the-box because of
# Converting a Reshapable TensorFlow BERT Model to OpenVINO IR

Follow these steps to make a pretrained TensorFlow BERT model reshapable over batch dimension:
1. Download a pretrained BERT model you want to use from the <a href="#supported_models">Supported Models list</a>
1. Download a pretrained BERT model you want to use from the <a href="#supported-models">Supported Models list</a>
2. Clone google-research/bert git repository:
```sh
https://github.com/google-research/bert.git
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This tutorial explains how to convert a CRNN model to Intermediate Representation (IR).

There are several public versions of TensorFlow CRNN model implementation available on GitHub. This tutorial explains how to convert the model from
the [https://github.com/MaybeShewill-CV/CRNN_Tensorflow](https://github.com/MaybeShewill-CV/CRNN_Tensorflow) repository to IR.
the [CRNN Tensorflow](https://github.com/MaybeShewill-CV/CRNN_Tensorflow) repository to IR.
If you have another implementation of CRNN model, it can be converted to OpenVINO IR in a similar way. You need to get inference graph and run Model Optimizer on it.

**To convert this model to the IR:**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ As a result, the frozen model file `savedmodeldir/efficientdet-d4_frozen.pb` wil

> **NOTE**: For custom trained models, specify `--hparams` flag to `config.yaml` which was used during training.
> **NOTE**: If you see an error `AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.initializers' has no attribute 'variance_scaling'`, apply the fix from the [patch](https://github.com/google/automl/pull/846).
> **NOTE**: If you see an error *AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.initializers' has no attribute 'variance_scaling'*, apply the fix from the [patch](https://github.com/google/automl/pull/846).
### Converting an EfficientDet TensorFlow Model to the IR

Expand All @@ -65,9 +65,9 @@ for the Model Optimizer on how to convert the model and trigger transformations
train the model yourself and modified the `hparams_config` file or the parameters are different from the ones used for EfficientDet-D4.
The attribute names are self-explanatory or match the name in the `hparams_config` file.

> **NOTE**: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the `RGB<->BGR` conversion specifying the command-line parameter: `--reverse_input_channels`. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the [Converting a Model to Intermediate Representation (IR)](../Converting_Model.md) guide.
> **NOTE**: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the `RGB<->BGR` conversion specifying the command-line parameter: `--reverse_input_channels`. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the [Converting a Model to Intermediate Representation (IR)](@ref openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model) guide.
OpenVINO&trade; toolkit provides samples that can be used to infer EfficientDet model.
OpenVINO toolkit provides samples that can be used to infer EfficientDet model.
For more information, refer to the [Open Model Zoo Demos](@ref omz_demos).

## <a name="efficientdet-ir-results-interpretation"></a>Interpreting Results of the TensorFlow Model and the IR
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ This tutorial assumes the use of the trained GNMT model from `wmt16_gnmt_4_layer

**Step 3**. Create an inference graph:

The OpenVINO&trade; assumes that a model is used for inference only. Hence, before converting the model into the IR, you need to transform the training graph into the inference graph.
The OpenVINO assumes that a model is used for inference only. Hence, before converting the model into the IR, you need to transform the training graph into the inference graph.
For the GNMT model, the training graph and the inference graph have different decoders: the training graph uses a greedy search decoding algorithm, while the inference graph uses a beam search decoding algorithm.

1. Apply the `GNMT_inference.patch` patch to the repository. Refer to the <a href="#patch-file">Create a Patch File</a> instructions if you do not have it:
Expand Down Expand Up @@ -215,7 +215,7 @@ Output cutting:

* `LookupTableFindV2` operation is cut from the output and the `dynamic_seq2seq/decoder/decoder/GatherTree` node is treated as a new exit point.

For more information about model cutting, refer to the [Cutting Off Parts of a Model](../Cutting_Model.md) guide.
For more information about model cutting, refer to the [Cutting Off Parts of a Model](@ref openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model) guide.

## Using a GNMT Model <a name="run_GNMT"></a>

Expand All @@ -234,7 +234,7 @@ Outputs of the model:
* `dynamic_seq2seq/decoder/decoder/GatherTree` tensor with shape `[max_sequence_length * 2, batch, beam_size]`,
that contains `beam_size` best translations for every sentence from input (also decoded as indices of words in
vocabulary).
> **NOTE**: The shape of this tensor in TensorFlow can be different: instead of `max_sequence_length * 2`, it can be any value less than that, because OpenVINO&trade; does not support dynamic shapes of outputs, while TensorFlow can stop decoding iterations when `eos` symbol is generated.
> **NOTE**: The shape of this tensor in TensorFlow can be different: instead of `max_sequence_length * 2`, it can be any value less than that, because OpenVINO does not support dynamic shapes of outputs, while TensorFlow can stop decoding iterations when `eos` symbol is generated.
#### Running GNMT IR <a name="run_GNMT"></a>

Expand Down Expand Up @@ -273,4 +273,4 @@ exec_net = ie.load_network(network=net, device_name="CPU")
result_ie = exec_net.infer(input_data)
```

For more information about Python API, refer to the [OpenVINO Runtime Python API](ie_python_api/api.html) guide.
For more information about Python API, refer to the [OpenVINO Runtime Python API](https://docs.openvino.ai/2022.2/api/api_reference.html) guide.
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Converting TensorFlow Object Detection API Models {#openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models}

> **NOTES**:
> * Starting with the 2022.1 release, Model Optimizer can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the [Proposal](../../../../ops/detection/Proposal_4.md) operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation [DetectionOutput](../../../../ops/detection/DetectionOutput_1.md) instead of `Proposal`. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal".
> * Starting with the 2021.1 release, Model Optimizer converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the [Using Shape Inference](../../../../OV_Runtime_UG/ShapeInference.md) guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
> * To generate IRs for TF 1 SSD topologies, Model Optimizer creates a number of `PriorBoxClustered` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape.
**NOTES**:

* Starting with the 2022.1 release, Model Optimizer can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the [Proposal](@ref openvino_docs_ops_detection_Proposal_4) operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation [DetectionOutput](@ref openvino_docs_ops_detection_DetectionOutput_1) instead of `Proposal`. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal".
* Starting with the 2021.1 release, Model Optimizer converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the [Using Shape Inference](@ref openvino_docs_OV_UG_ShapeInference) guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
* To generate IRs for TF 1 SSD topologies, Model Optimizer creates a number of `PriorBoxClustered` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape.

## Converting a Model

Expand Down Expand Up @@ -62,7 +63,7 @@ Open Model Zoo provides set of demo applications to show implementation of close
based on deep learning in various tasks, including Image Classification, Visual Object Detection, Text Recognition,
Speech Recognition, Natural Language Processing and others. Refer to the links below for more details.

* [OpenVINO Samples](../../../../OV_Runtime_UG/Samples_Overview.md)
* [OpenVINO Samples](@ref openvino_docs_OV_UG_Samples_Overview)
* [Open Model Zoo Demos](@ref omz_demos)

## Feeding Input Images to the Samples
Expand Down Expand Up @@ -134,5 +135,5 @@ It is also important to open the model in the [TensorBoard](https://www.tensorfl
* `--input_model <path_to_frozen.pb>` --- Path to the frozen model.
* `--tensorboard_logdir` --- Path to the directory where TensorBoard looks for the event files.

Implementation of the transformations for Object Detection API models is located in the file [https://github.com/openvinotoolkit/openvino/blob/releases/2022/1/tools/mo/openvino/tools/mo/front/tf/ObjectDetectionAPI.py](https://github.com/openvinotoolkit/openvino/blob/releases/2022/1/tools/mo/openvino/tools/mo/front/tf/ObjectDetectionAPI.py). Refer to the code in this file to understand the details of the conversion process.
Implementation of the transformations for Object Detection API models is located in the [file](https://github.com/openvinotoolkit/openvino/blob/releases/2022/1/tools/mo/openvino/tools/mo/front/tf/ObjectDetectionAPI.py). Refer to the code in this file to understand the details of the conversion process.

Loading

0 comments on commit 3127c28

Please sign in to comment.