cuda::experimental::stf::cached_block_allocator_fifo
Defined in include/cuda/experimental/__stf/allocators/cached_allocator.cuh
-
class cached_block_allocator_fifo : public cuda::experimental::stf::block_allocator_interface
A block allocator with FIFO caching strategy.
This allocator wraps a root allocator and implements a caching mechanism to efficiently reuse previously allocated memory blocks.
When a memory block is requested, it first attempts to find a suitable block in the internal cache (
free_cache). If no suitable block is found, a larger buffer is allocated from the root allocator, split into smaller blocks, and stored in the cache for future reuse.The caching policy is FIFO (First-In-First-Out): freed blocks are appended to a queue and reused in the order they were deallocated.
At deinitialization (
deinit()), all cached blocks and their associated large allocations are properly released by delegating to the root allocator.See also
Note
The allocator is thread-safe and uses a mutex to protect access to its internal cache.
Note
Deallocation does not immediately release memory back to the root allocator. Instead, deallocated blocks are stored in the cache for future reuse.
Public Functions
-
inline cached_block_allocator_fifo(block_allocator_untyped root_allocator_)
This constructor takes a root allocator which performs allocations when there is a cache miss.
-
inline virtual void *allocate(backend_ctx_untyped &ctx, const data_place &memory_node, ::std::ptrdiff_t &s, event_list &prereqs) override
Allocates a memory block.
Attempts to allocate a memory block from the internal cache if available.
- Parameters
ctx – The backend context (unused in this implementation).
memory_node – Memory location where the allocation should happen.
s – Pointer to the size of memory required.
prereqs – List of events that this allocation depends on.
- Returns
void* A pointer to the allocated memory block or nullptr if allocation fails.
-
inline virtual void deallocate(backend_ctx_untyped&, const data_place &memory_node, event_list &prereqs, void *ptr, size_t sz) override
Deallocates a memory block.
Puts the deallocated block back into the internal cache for future reuse.
- Parameters
ctx – The backend context (unused in this implementation).
memory_node – Memory location where the deallocation should happen.
prereqs – List of events that this deallocation depends on.
ptr – Pointer to the memory block to be deallocated.
sz – Size of the memory block.
-
inline virtual event_list deinit(backend_ctx_untyped &ctx) override
De-initializes the allocator.
-
inline virtual ::std::string to_string() const override
Returns a string representation of the allocator.
- Returns
std::string The string “cached_block_allocator”.
-
inline virtual void print_info() const
Print information about the allocator.
This method prints information about the allocator to stderr.
Protected Types
-
using per_place_map_t = ::std::unordered_map<size_t, ::std::queue<alloc_cache_entry>>
Maps sizes to cache entries for a given data_place.
Protected Attributes
-
block_allocator_untyped root_allocator
Underlying root allocator for base buffers.
-
::std::unordered_map<data_place, per_place_map_t, hash<data_place>> free_cache
We track actually allocated blocks (per data place), and then a map of.
-
::std::unordered_map<data_place, ::std::vector<::std::pair<void*, size_t>>, hash<data_place>> large_allocations
-
::std::mutex allocator_mutex
-
struct alloc_cache_entry
Struct representing an entry in the cache.
Holds the address of the allocated memory and a list of prerequisites for its reuse.
Public Members
-
void *ptr
Pointer to the allocated memory block.
-
event_list prereq
Prerequisites for reusing this block.
-
void *ptr
-
inline cached_block_allocator_fifo(block_allocator_untyped root_allocator_)