cuda::experimental::stf::cached_block_allocator_fifo

Defined in include/cuda/experimental/__stf/allocators/cached_allocator.cuh

class cached_block_allocator_fifo : public cuda::experimental::stf::block_allocator_interface 

A block allocator with FIFO caching strategy.

This allocator wraps a root allocator and implements a caching mechanism to efficiently reuse previously allocated memory blocks.

When a memory block is requested, it first attempts to find a suitable block in the internal cache (free_cache). If no suitable block is found, a larger buffer is allocated from the root allocator, split into smaller blocks, and stored in the cache for future reuse.

The caching policy is FIFO (First-In-First-Out): freed blocks are appended to a queue and reused in the order they were deallocated.

At deinitialization (deinit()), all cached blocks and their associated large allocations are properly released by delegating to the root allocator.

See also

block_allocator_interface

Note

The allocator is thread-safe and uses a mutex to protect access to its internal cache.

Note

Deallocation does not immediately release memory back to the root allocator. Instead, deallocated blocks are stored in the cache for future reuse.

Public Functions

inline cached_block_allocator_fifo(block_allocator_untyped root_allocator_): This constructor takes a root allocator which performs allocations when there is a cache miss.

inline virtual void *allocate(backend_ctx_untyped &ctx, const data_place &memory_node, ::std::ptrdiff_t &s, event_list &prereqs) override

Allocates a memory block.

Attempts to allocate a memory block from the internal cache if available.

Parameters

ctx – The backend context (unused in this implementation).
memory_node – Memory location where the allocation should happen.
s – Pointer to the size of memory required.
prereqs – List of events that this allocation depends on.

Returns

void* A pointer to the allocated memory block or nullptr if allocation fails.

inline virtual void deallocate(backend_ctx_untyped&, const data_place &memory_node, event_list &prereqs, void *ptr, size_t sz) override

Deallocates a memory block.

Puts the deallocated block back into the internal cache for future reuse.

Parameters

ctx – The backend context (unused in this implementation).
memory_node – Memory location where the deallocation should happen.
prereqs – List of events that this deallocation depends on.
ptr – Pointer to the memory block to be deallocated.
sz – Size of the memory block.

inline virtual event_list deinit(backend_ctx_untyped &ctx) override: De-initializes the allocator.

inline virtual ::std::string to_string() const override

Returns a string representation of the allocator.

Returns: std::string The string “cached_block_allocator”.

inline virtual void print_info() const override

Prints additional information about the allocator.

This function currently prints no additional information.

Protected Types

using per_place_map_t = ::std::unordered_map<size_t, ::std::queue<alloc_cache_entry>>: Maps sizes to cache entries for a given data_place.

Protected Attributes

block_allocator_untyped root_allocator: Underlying root allocator for base buffers.

::std::unordered_map<data_place, per_place_map_t, hash<data_place>> free_cache: We track actually allocated blocks (per data place), and then a map of.

::std::unordered_map<data_place, ::std::vector<::std::pair<void*, size_t>>, hash<data_place>> large_allocations

::std::mutex allocator_mutex

struct alloc_cache_entry

Struct representing an entry in the cache.

Holds the address of the allocated memory and a list of prerequisites for its reuse.

Public Members

void *ptr: Pointer to the allocated memory block.

event_list prereq: Prerequisites for reusing this block.