Skip to content

Commit

Permalink
Docs g2p update (NVIDIA#5769) (NVIDIA#5775)
Browse files Browse the repository at this point in the history
* links update, riva docs link

Signed-off-by: ekmb <[email protected]>
Co-authored-by: Evelina <[email protected]>
  • Loading branch information
github-actions[bot] and ekmb authored Jan 11, 2023
1 parent c6b1ea5 commit ff7544b
Showing 1 changed file with 11 additions and 6 deletions.
17 changes: 11 additions & 6 deletions docs/source/nlp/text_normalization/wfst/wfst_customization.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,19 @@ Grammar customization

.. note::

*TN/ITN is transitioning from [NVIDIA/NeMo](https://github.com/NVIDIA/NeMo) repository to a standalone [NVIDIA/NeMo-text-processing](https://github.com/NVIDIA/NeMo-text-processing) repository. All updates and discussions/issues should go to the new repository.*
TN/ITN is transitioning from `NVIDIA/NeMo <https://github.com/NVIDIA/NeMo>`_ repository to a standalone `NVIDIA/NeMo-text-processing <https://github.com/NVIDIA/NeMo-text-processing>`_ repository. All updates and discussions/issues should go to the new repository.


All grammar development is done with [Pynini library](https://www.opengrm.org/twiki/bin/view/GRM/Pynini).
All grammar development is done with `Pynini library <https://www.opengrm.org/twiki/bin/view/GRM/Pynini>`_.
These grammars can be exported to .far files and used with Riva/Sparrowhawk, see :doc:`Text Processing Deployment <wfst_text_processing_deployment>` for details.

Steps to customize grammars
---------------------------

1. Install [NeMo-TN from source](https://github.com/NVIDIA/NeMo-text-processing#from-source)
2. Run [nemo_text_processing/text_normalization/normalize.py](https://github.com/NVIDIA/NeMo-text-processing/blob/main/nemo_text_processing/text_normalization/normalize.py) or [nemo_text_processing/inverse_text_normalization/inverse_normalize.py](https://github.com/NVIDIA/NeMo-text-processing/blob/main/nemo_text_processing/inverse_text_normalization/inverse_normalize.py) with `--verbose` flag to evaluate current behavior on the target case, see argument details in the scripts and [this tutorial](https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/text_processing/Text_(Inverse)_Normalization.ipynb)
3. Modify existing grammars or add new grammars to cover the target case using [Tutorial on how to write new grammars](https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/text_processing/WFST_Tutorial.ipynb)
4. Add new test cases [here](https://github.com/NVIDIA/NeMo-text-processing/tree/main/tests/nemo_text_processing):
1. Install `NeMo-TN from source <https://github.com/NVIDIA/NeMo-text-processing#from-source>`_
2. Run `nemo_text_processing/text_normalization/normalize.py <https://github.com/NVIDIA/NeMo-text-processing/blob/main/nemo_text_processing/text_normalization/normalize.py>`_ or `nemo_text_processing/inverse_text_normalization/inverse_normalize.py <https://github.com/NVIDIA/NeMo-text-processing/blob/main/nemo_text_processing/inverse_text_normalization/inverse_normalize.py>`_ with `--verbose` flag to evaluate current behavior on the target case, see argument details in the scripts and `this tutorial <https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/text_processing/Text_(Inverse)_Normalization.ipynb>`_
3. Modify existing grammars or add new grammars to cover the target case using `Tutorial on how to write new grammars <https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/text_processing/WFST_Tutorial.ipynb>`_
4. Add new test cases `here <https://github.com/NVIDIA/NeMo-text-processing/tree/main/tests/nemo_text_processing>`_:
- Run python tests:

.. code-block:: bash
Expand All @@ -35,3 +35,8 @@ Steps to customize grammars
WFST TN/ITN resources could be found in :doc:`here <wfst_resources>`.

Riva resources
--------------
- `Riva Text Normalization customization for TTS <https://riva-builder-01.nvidia.com/main/tts/tts-custom.html#custom-text-normalization>`_.
- `Riva ASR/Inverse Text Normalization customization <https://riva-builder-01.nvidia.com/main/asr/asr-customizing.html>`_.

0 comments on commit ff7544b

Please sign in to comment.