Getting started

NeMo-Skills is a collection of pipelines to improve "skills" of large language models. We mainly focus on the ability to solve mathematical problems, but you can use our pipelines for many other tasks as well. Here are some of the things we support.

• Flexible inference: Seamlessly switch between API providers, local server and large-scale slurm jobs for LLM inference.
• Multiple formats: Use any of the NeMo, vLLM and TensorRT-LLM servers and easily convert checkpoints from one format to another.
• Model evaluation: Evaluate your models on many popular benchmarks
  • Math problem solving: gsm8k, math, amc23, aime24, omni-math (and many more)
  • Formal proofs in Lean: minif2f, proofnet
  • Coding skills: human-eval, mbpp
  • Chat/instruction following: ifeval, arena-hard, mt-bench
  • General knowledge: mmlu (generative), mmlu-pro
• Model training: Train models at speed-of-light using NeMo-Aligner.

To get started, follow the prerequisites and then run ns --help to see all available commands and their options.