NVIDIA Neural Modules Developer Guide¶
Neural Modules (NeMo) is a framework-agnostic toolkit for building AI applications powered by Neural Modules. Current support is for PyTorch framework.
A “Neural Module” is a block of code that computes a set of outputs from a set of inputs.
Neural Modules’ inputs and outputs have Neural Type for semantic checking.
An application built with NeMo is a Directed Acyclic Graph (DAG) of connected modules enabling researchers to define and build new speech and nlp networks easily through API Compatible modules.
See this video for a walk-through.
Core Concepts and Features
NeuralModule class - represents and implements a neural module.
NmTensor - represents activations which flow between neural modules’ ports.
NeuralType - represents types of modules’ ports and NmTensors.
NeuralFactory - to create neural modules and manage training.
Lazy execution - when describing activation flow between neural modules, nothing happens until an “action” (such as optimizer.optimize(…) is called.
Collections - NeMo comes with collections - related group of modules such as nemo_asr (for Speech Recognition) and nemo_nlp for NLP
Python 3.6 or 3.7
PyTorch 1.2 with GPU support
NVIDIA APEX: https://github.com/NVIDIA/apex
You can use NVIDIA NGC PyTorch container which already includes all the requirements above.
# Pull the docker docker pull nvcr.io/nvidia/pytorch:19.11-py3 # Do one of the two following commands # Run Docker for docker version <19.03 nvidia-docker run -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/pytorch:19.11-py3 # Run Docker for docker version >=19.03 docker run --runtime=nvidia -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/pytorch:19.11-py3 cd /NeMo
and then continue with the following steps.
If you have all requirements installed (or are using NGC PyTorch container ), then you can simply use pip to install the latest released version (currently 0.9.0) of NeMo and its collections:
pip install nemo-toolkit # installs NeMo Core pip install nemo-asr # installs NeMo ASR collection pip install nemo-nlp # installs NeMo NLP collection pip install nemo-tts # installs NeMo TTS collection
Installing From Github
If you prefer to use NeMo’s latest development version (from GitHub) follow the steps below:
Note: For step 2 and 3, if you want to use NeMo in development mode, use:
pip install -e . instead of
pip install .
Clone the repository
git clone https://github.com/NVIDIA/NeMo.git
Go to NeMo folder and install the toolkit:
cd NeMo/nemo pip install .
Install the collection(s) you want.
# Install the ASR collection from collections/nemo_asr apt-get install libsndfile1 cd NeMo/collections/nemo_asr pip install . # Install the NLP collection from collections/nemo_nlp cd NeMo/collections/nemo_nlp pip install . # Install the TTS collection from collections/nemo_tts cd NeMo/collections/nemo_tts pip install .
This command runs unittests:
./reinstall.sh python -m unittest tests/*.py