thrust::mr::unsynchronized_pool_resource

Defined in thrust/mr/pool.h

template<typename Upstream>
class unsynchronized_pool_resource : public thrust::mr::memory_resource<Upstream::pointer>, private thrust::mr::validator<Upstream>

A memory resource adaptor allowing for pooling and caching allocations from Upstream, using memory allocated from it for both blocks then allocated to the user and for internal bookkeeping of the cached memory.

On a typical memory resource, calls to allocate and deallocate actually allocate and deallocate memory. Pooling memory resources only allocate and deallocate memory from an external resource (the upstream memory resource) when there’s no suitable memory currently cached; otherwise, they use memory they have acquired beforehand, to make memory allocation faster and more efficient.

The non-disjoint version of the pool resource uses a single upstream memory resource. Every allocation is larger than strictly necessary to fulfill the end-user’s request, because it needs to account for the memory overhead of tracking the memory blocks and chunks inside those same memory regions. Nevertheless, this version should be more memory-efficient than the disjoint_unsynchronized_pool_resource, because it doesn’t need to allocate additional blocks of memory from a separate resource, which in turn would necessitate the bookkeeping overhead in the upstream resource.

This version requires that memory allocated from Upstream is accessible from device. It supports smart references, meaning that the non-managed CUDA resource, returning a device-tagged pointer, will work, but will be much less efficient than the disjoint version, which wouldn’t need to touch device memory at all, and therefore wouldn’t need to transfer it back and forth between the host and the device whenever an allocation or a deallocation happens.

Template Parameters

Upstream – the type of memory resources that will be used for allocating memory blocks

Public Types

using pointer = Upstream::pointer

Alias for the template parameter.

Public Functions

inline unsynchronized_pool_resource(Upstream *upstream, pool_options options = get_default_options())

Constructor.

Parameters

  • upstream – the upstream memory resource for allocations

  • options – pool options to use

inline unsynchronized_pool_resource(pool_options options = get_default_options())

Constructor. The upstream resource is obtained by calling get_global_resource<Upstream>.

Parameters

options – pool options to use

inline ~unsynchronized_pool_resource()

Destructor. Releases all held memory to upstream.

inline void release()

Releases all held memory to upstream.

inline virtual void_ptr do_allocate(std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) override

Allocates memory of size at least bytes and alignment at least alignment.

Parameters

  • bytes – size, in bytes, that is requested from this allocation

  • alignment – alignment that is requested from this allocation

Throws

thrust::bad_alloc – when no memory with requested size and alignment can be allocated.

Returns

A pointer to void to the newly allocated memory.

inline virtual void do_deallocate(void_ptr p, std::size_t n, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) override
inline pointer allocate(std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT)

Allocates memory of size at least bytes and alignment at least alignment.

Parameters

  • bytes – size, in bytes, that is requested from this allocation

  • alignment – alignment that is requested from this allocation

Throws

thrust::bad_alloc – when no memory with requested size and alignment can be allocated.

Returns

A pointer to void to the newly allocated memory.

inline void deallocate(pointer p, std::size_t bytes, std::size_t alignment = THRUST_MR_DEFAULT_ALIGNMENT) noexcept

Deallocates memory pointed to by p.

Parameters

  • p – pointer to be deallocated

  • bytes – the size of the allocation. This must be equivalent to the value of bytes that was passed to the allocation function that returned p.

  • alignment – the alignment of the allocation. This must be equivalent to the value of alignment that was passed to the allocation function that returned p.

inline bool is_equal(const memory_resource &other) const noexcept

Compares this resource to the other one. The default implementation uses identity comparison, which is often the right thing to do and doesn’t require RTTI involvement.

Parameters

other – the other resource to compare this resource to

Returns

whether the two resources are equivalent.

virtual void do_deallocate(pointer p, std::size_t bytes, std::size_t alignment) = 0

Deallocates memory pointed to by p.

Parameters

  • p – pointer to be deallocated

  • bytes – the size of the allocation. This must be equivalent to the value of bytes that was passed to the allocation function that returned p.

  • alignment – the size of the allocation. This must be equivalent to the value of alignment that was passed to the allocation function that returned p.

inline virtual bool do_is_equal(const memory_resource &other) const noexcept

Compares this resource to the other one. The default implementation uses identity comparison, which is often the right thing to do and doesn’t require RTTI involvement.

Parameters

other – the other resource to compare this resource to

Returns

whether the two resources are equivalent.

Public Static Functions

static inline pool_options get_default_options()

Get the default options for a pool. These are meant to be a sensible set of values for many use cases, and as such, may be tuned in the future. This function is exposed so that creating a set of options that are just a slight departure from the defaults is easy.