RAPIDS Accelerator for Apache Spark Nvtx Range Glossary

The following is the list of Nvtx ranges that are used throughout the plugin. To add your own Nvtx range to the code, create an NvtxId entry in NvtxRangeWithDoc.scala and create an NvtxRangeWithDoc in the code location that you want to cover, passing in the newly created NvtxId.

See nvtx_profiling.md for more info.

Nvtx Ranges

Name	Description
getMapSizesByExecId	Call to internal Spark API for retrieving size and location of shuffle map output blocks
gpuKudoSerialize	Perform kudo serialization on the gpu
Acquire GPU	Time waiting for GPU semaphore to be acquired
Release GPU	Releasing the GPU semaphore
gpuKudoSliceBuffers	slice kudo serialized buffers on host into partitions
CommitShuffle	After all temporary shuffle writes are done, produce a single file (shuffle_[map_id]_0) in the commit phase
ParallelDeserializerIterator.next	Calling next on the MT shuffle reader iterator
queueFetched	MT shuffle manager is using the RapidsShuffleBlockFetcherIterator to queue the next set of fetched results
WaitingForWrites	Rapids Shuffle Manager (multi threaded) is waiting for any queued writes to finish before finalizing the map output writer
ThreadedWriter.write	Rapids Shuffle Manager (multi threaded) writing
ThreadedReader.read	Rapids Shuffle Manager (multi threaded) reading
gpuKudoCopyToHost	copy gpu kudo serialized outputs back to the host
BatchWait	Rapids Shuffle Manager (multi threaded) reader blocked waiting for batches to finish decoding
RapidsCachingWriter.write	Rapids Shuffle Manager (ucx) writing