cuda.bindings._v2.driver.dev_sm_resource_split#

cuda.bindings._v2.driver.dev_sm_resource_split( unsigned int nb_groups, input, unsigned int flags, group_params, ) → tuple#

Splits a CU_DEV_RESOURCE_TYPE_SM resource into structured groups.

This API will split a resource of CU_DEV_RESOURCE_TYPE_SM into nb_groups structured device resource groups (the result array), as well as an optional remainder, according to a set of requirements specified in the group_params array. The term “structured” is a trait that specifies the result has SMs that are co-scheduled together. This co-scheduling can be specified via the coscheduledSmCount field of the group_params structure, while the smCount will specify how many SMs are required in total for that result. The remainder is always “unstructured”, it does not have any set guarantees with respect to co-scheduling and those properties will need to either be queried via the occupancy set of APIs or further split into structured groups by this API. The API has a discovery mode for use cases where it is difficult to know ahead of time what the SM count should be. Discovery happens when the smCount field of a given group_params array entry is set to 0 - the smCount will be filled in by the API with the derived SM count according to the provided group_params fields and constraints. Discovery can be used with both a valid result array and with a NULL result pointer value. The latter is useful in situations where the smCount will end up being zero, which is an invalid value to create a result entry with, but allowed for discovery purposes when the result is NULL. The group_params array is evaluated from index 0 to nb_groups - 1. For each index in the group_params array, the API will evaluate which SMs may be a good fit based on constraints and assign those SMs to result. This evaluation order is important to consider when using discovery mode, as it helps discover the remaining SMs. For a valid call:.

result should point to a CUdevResource array of size nb_groups, or alternatively, may be NULL, if the developer wishes for only the group_params entries to be updated.
input should be a valid CU_DEV_RESOURCE_TYPE_SM resource that originates from querying the green context, device context, or device.
The remainder group may be NULL.
There are no API flags at this time, so the value passed in should be 0.
A CU_DEV_SM_RESOURCE_GROUP_PARAMS array of size nb_groups. Each entry must be zero-initialized.
smCount: must be either 0 or in the range of [2,inputSmCount] where inputSmCount is the amount of SMs the input resource has. smCount must be a multiple of 2, as well as a multiple of coscheduledSmCount if it is nonzero. When assigning SMs to a group (and if results are expected by having the result parameter set), smCount cannot end up with 0 or a value less than coscheduledSmCount otherwise CUDA_ERROR_INVALID_RESOURCE_CONFIGURATION will be returned.
coscheduledSmCount: allows grouping SMs together in order to be able to launch clusters on Compute Architecture 9.0+. The default value may be queried from the device’s CU_DEV_RESOURCE_TYPE_SM resource (8 on Compute Architecture 9.0+ and 2 otherwise). The maximum is 32 on Compute Architecture 9.0+ and 2 otherwise. A coscheduledSmCount of 0 uses the default value internally while preserving 0 in group_params. Cluster occupancy will be derived from the resulting SM topology.
preferredCoscheduledSmCount: Attempts to merge coscheduledSmCount groups into larger groups, in order to make use of preferredClusterDimensions on Compute Architecture 10.0+. The default value is set to coscheduledSmCount.
flags:.
CU_DEV_SM_RESOURCE_GROUP_BACKFILL: Treats constraints as a hint, ignoring them if necessary to reach the requested smCount. Lets smCount be a non-multiple of coscheduledSmCount, filling the difference between SM count and already assigned co-scheduled groupings with other SMs. This lets any resulting group behave similar to the remainder group for example. When used with CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID, backfill fills up to the requested smCount using the target locality domain first, then SMs not attributed to any locality domain, then SMs from other locality domains. If no SMs can be found in the requested locality domain, CUDA_ERROR_INVALID_RESOURCE_CONFIGURATION is returned.
CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID: Specifies that the SM partition should be localized to the specified localityDomainId.
localityDomainId: Specifies the locality domain that the partitioned SMs must be located on. Only valid when CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID is set in flags.

Example params and their effect:. A group_params array element is defined in the following order:. **View CUDA Toolkit Documentation for a C++ code example**. **View CUDA Toolkit Documentation for a C++ code example**. **View CUDA Toolkit Documentation for a C++ code example**. **View CUDA Toolkit Documentation for a C++ code example**. The difference between a catch-all param group as the last entry and the remainder is in two aspects:.

The remainder may be NULL / _TYPE_INVALID (if there are no SMs remaining), while a result group must always be valid.
The remainder does not have a structure, while the result group will always need to adhere to a structure of coscheduledSmCount (even if its just 2), and therefore must always have enough coscheduled SMs to cover that requirement (even with the CU_DEV_SM_RESOURCE_GROUP_BACKFILL flag enabled).

Splitting an input into N groups, can be accomplished by repeatedly splitting off 1 group and re-splitting the remainder (a bisect operation). However, it’s recommended to accomplish this with a single call wherever possible.

Parameters:

nb_groups (unsigned int) – Specifies the number of groups in result and group_params.
input (intptr_t) – Input SM resource to be split. Must be a valid CU_DEV_RESOURCE_TYPE_SM resource.
flags (unsigned int) – Flags specifying how the API should behave. The value should be 0 for now.
group_params (intptr_t) – Description of how the SMs should be split and assigned to the corresponding result entry.

Returns:

CUdevResource_v1: Output array of CUdevResource resources. Can
be NULL, alongside an smCount of 0, for discovery purpose.
CUdevResource_v1: If splitting the input resource leaves any SMs,
the remainder is placed in here.

Return type:

A 2-tuple containing