FAQs

NAS/Pruning

Parallel search

If your score_func in mtn.search() supports parallel evaluation, you can make use of it by passing in DistributedDataParallel module to search.

Monkey-patched functions

During the conversion process (mtn.convert()), we use a monkey patch to augment the forward(), eval(), and train() methods of nn.Module. This renders the ModelOpt conversion process incompatible with other monkey patches to those methods.

# Internally in mtn.convert, we do:
model.forward = types.MethodType(nas_forward_func, model)
model.train = types.MethodType(nas_train_func, model)

Known Issues

1. Potential memory leak for FSDP with use_orig_params=True

When using FSDP with use_orig_params=True, there is a potential memory leak during training when using FSDP in conjunction with modelopt-converted models. Please use use_orig_params=False to avoid this issue.