thrust::mr::disjoint_unsynchronized_pool_resource

Defined in thrust/mr/disjoint_pool.h

template<typename Upstream, typename Bookkeeper> class disjoint_unsynchronized_pool_resource : public thrust::mr::memory_resource<Upstream::pointer>, private thrust::mr::validator2<Upstream, Bookkeeper>

A memory resource adaptor allowing for pooling and caching allocations from Upstream, using Bookkeeper for management of that cached and pooled memory, allowing to cache portions of memory inaccessible from the host.

On a typical memory resource, calls to allocate and deallocate actually allocate and deallocate memory. Pooling memory resources only allocate and deallocate memory from an external resource (the upstream memory resource) when there’s no suitable memory currently cached; otherwise, they use memory they have acquired beforehand, to make memory allocation faster and more efficient.

The disjoint version of the pool resources uses a separate upstream memory resource, Bookkeeper, to allocate memory necessary to manage the cached memory. There may be many reasons to do that; the canonical one is that Upstream allocates memory that is inaccessible to the code of the pool resource, which means that it cannot embed the necessary information in memory obtained from Upstream; for instance, Upstream can be a CUDA non-managed memory resource, or a CUDA managed memory resource whose memory we would prefer to not migrate back and forth between host and device when executing bookkeeping code.

This is not the only case where it makes sense to use a disjoint pool resource, though. In a multi-core environment it may be beneficial to avoid stealing cache lines from other cores by writing over bookkeeping information embedded in an allocated block of memory. In such a case, one can imagine wanting to use a disjoint pool where both the upstream and the bookkeeper are of the same type, to allocate memory consistently, but separately for those two purposes.

Template Parameters

Upstream – the type of memory resources that will be used for allocating memory blocks to be handed off to the user
Bookkeeper – the type of memory resources that will be used for allocating bookkeeping memory

Public Types

using pointer = Upstream::pointer: Alias for the template parameter.

Public Functions

inline disjoint_unsynchronized_pool_resource(Upstream *upstream, Bookkeeper *bookkeeper, pool_options options = get_default_options())

Constructor.

Parameters

upstream – the upstream memory resource for allocations
bookkeeper – the upstream memory resource for bookkeeping
options – pool options to use

inline disjoint_unsynchronized_pool_resource(pool_options options = get_default_options())

Constructor. Upstream and bookkeeping resources are obtained by calling get_global_resource for their types.

Parameters: options – pool options to use

inline ~disjoint_unsynchronized_pool_resource(): Destructor. Releases all held memory to upstream.

inline void release(): Releases all held memory to upstream.

inline virtual void_ptr do_allocate(std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) override

Allocates memory of size at least bytes and alignment at least alignment.

Parameters

bytes – size, in bytes, that is requested from this allocation
alignment – alignment that is requested from this allocation

Throws

thrust::bad_alloc – when no memory with requested size and alignment can be allocated.

Returns

A pointer to void to the newly allocated memory.

inline virtual void do_deallocate(void_ptr p, std::size_t n, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) override

inline pointer allocate(std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT)

Allocates memory of size at least bytes and alignment at least alignment.

Parameters

bytes – size, in bytes, that is requested from this allocation
alignment – alignment that is requested from this allocation

Throws

thrust::bad_alloc – when no memory with requested size and alignment can be allocated.

Returns

A pointer to void to the newly allocated memory.

inline void deallocate(pointer p, std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) noexcept

Deallocates memory pointed to by p.

Parameters

p – pointer to be deallocated
bytes – the size of the allocation. This must be equivalent to the value of bytes that was passed to the allocation function that returned p.
alignment – the alignment of the allocation. This must be equivalent to the value of alignment that was passed to the allocation function that returned p.

inline bool is_equal(const memory_resource &other) const noexcept

Compares this resource to the other one. The default implementation uses identity comparison, which is often the right thing to do and doesn’t require RTTI involvement.

Parameters: other – the other resource to compare this resource to
Returns: whether the two resources are equivalent.

virtual void do_deallocate(pointer p, std::size_t bytes, std::size_t alignment) = 0

Deallocates memory pointed to by p.

Parameters

p – pointer to be deallocated
bytes – the size of the allocation. This must be equivalent to the value of bytes that was passed to the allocation function that returned p.
alignment – the size of the allocation. This must be equivalent to the value of alignment that was passed to the allocation function that returned p.

inline virtual bool do_is_equal(const memory_resource &other) const noexcept

Compares this resource to the other one. The default implementation uses identity comparison, which is often the right thing to do and doesn’t require RTTI involvement.

Parameters: other – the other resource to compare this resource to
Returns: whether the two resources are equivalent.

Public Static Functions

static inline pool_options get_default_options(): Get the default options for a disjoint pool. These are meant to be a sensible set of values for many use cases, and as such, may be tuned in the future. This function is exposed so that creating a set of options that are just a slight departure from the defaults is easy.