CudaMemory
CudaMemory
Section titled “CudaMemory”#include <sbio/execution/cuda.hh>class CudaMemoryDefined in src/lib/sbio/execution/cuda.hh:166
CudaExecution is a policy for a single process to use one GPU.
IO is staged first through the host. After which results are moved to GPU, and returned in GPU device memory.
- All data buffers are device memory, and have a replica pinned host buffer by using
[CudaMirroredBuffer](api-sbio-CudaMirroredBuffer.md#cudamirroredbuffer). IO reads into the host replica. - Data is moved to device using
publish_broker_view. - Data format field resolution can happen on the host - so no copy back has to occur for this process. The pointers resolved on host are remapped directly on device.
- The Table for the BrokerGropu is pinned host memory so it can be read on device.
Iteration happens from the host from a single thread (serial).
get_data and friends stream synchronize before returns to guarantee results are ready.
For IO policies that support direct-to-GPU reads, CudaExecution will use that path without the need for the staged IO.
- If compiled as device code with nvcc, a single-threaded kernel will do all resolution/lookup directly on the device as well.
- Otherwise, the lookup will still be done on the host normally, by staging via the host replica buffer.
Public Static Attributes
Section titled “Public Static Attributes”| Return | Name | Description |
|---|---|---|
constexpr std::bitset< static_cast< std::size_t >(ParallelizationMethods::NUM_METHODS) > |
ParallelSupport static constexpr |
ParallelSupport
Section titled “ParallelSupport”static constexpr
constexpr std::bitset< static_cast< std::size_t >(ParallelizationMethods::NUM_METHODS) > ParallelSupport { 0x4 }Type: constexpr std::bitset< static_cast< std::size_t >(ParallelizationMethods::NUM_METHODS) >
Defined in src/lib/sbio/execution/cuda.hh:188
Public Static Methods
Section titled “Public Static Methods”| Return | Name | Description |
|---|---|---|
constexpr MemorySpace |
result_memory_space_impl static inline constexpr |
|
constexpr bool |
reads_to_device static inline constexpr |
Check if the IO policy can read into GPU memory. |
constexpr IOCapability |
required_io_capabilities_impl static inline constexpr |
IO currently gets staged through a pinned host buffer for this EPolicy. |
bool |
validate_environment_impl static inline |
The requirements are available CUDA devices (GPUs), and config selecting one of these. |
void |
init_logger static inline |
Setup a spdlog logger. |
void |
configure_impl static inline |
|
void |
use_unit_ordinal static inline |
Use the DistributionManager’s numbering parallel units. |
void |
use_io static inline |
|
void |
allocate_buffer static inline |
|
void |
release_buffer static inline `requires (std::is_same_v<Buffer, CudaMirroredBuffer> |
|
void * |
acquire_broker_view_impl static inline |
Get a host view of a device buffer (using the replica in the buffer). |
void |
publish_broker_view_impl static inline |
Move the bytes IO read from staging buffer (host) to device buffer. |
void |
after_get_data static inline |
Synchronize the stream so that results are returned only once copy completes. |
void |
synchronize static inline |
|
cudaStream_t |
get_stream static inline |
|
int |
unit_device static inline |
The device associated to the calling parallel unit. |
void |
bind_device static inline |
Make the calling unit’s device the current device for the host htread. |
result_memory_space_impl
Section titled “result_memory_space_impl”static inline constexpr
constexpr static inline constexpr MemorySpace result_memory_space_impl()Defined in src/lib/sbio/execution/cuda.hh:190
reads_to_device
Section titled “reads_to_device”static inline constexpr
template<typename IO> constexpr static inline constexpr bool reads_to_device()Defined in src/lib/sbio/execution/cuda.hh:196
Check if the IO policy can read into GPU memory.
required_io_capabilities_impl
Section titled “required_io_capabilities_impl”static inline constexpr
template<typename IO> constexpr static inline constexpr IOCapability required_io_capabilities_impl()Defined in src/lib/sbio/execution/cuda.hh:204
IO currently gets staged through a pinned host buffer for this EPolicy.
validate_environment_impl
Section titled “validate_environment_impl”static inline
static inline bool validate_environment_impl(const Config & config, const char *& error)Defined in src/lib/sbio/execution/cuda.hh:218
The requirements are available CUDA devices (GPUs), and config selecting one of these.
init_logger
Section titled “init_logger”static inline
static inline void init_logger()Defined in src/lib/sbio/execution/cuda.hh:243
Setup a spdlog logger.
configure_impl
Section titled “configure_impl”static inline
static inline void configure_impl(const Config & config)Defined in src/lib/sbio/execution/cuda.hh:253
use_unit_ordinal
Section titled “use_unit_ordinal”static inline
static inline void use_unit_ordinal(UnitOrdinalFn unit_ordinal)Defined in src/lib/sbio/execution/cuda.hh:266
Use the DistributionManager’s numbering parallel units.
use_io
Section titled “use_io”static inline
template<class IO> static inline void use_io()Defined in src/lib/sbio/execution/cuda.hh:271
allocate_buffer
Section titled “allocate_buffer”static inline
template<class Buffer> static inline void allocate_buffer(Buffer & buf, std::size_t size)Defined in src/lib/sbio/execution/cuda.hh:288
release_buffer
Section titled “release_buffer”static inline requires (std::is_same_v<Buffer, CudaMirroredBuffer> || std::is_same_v<Buffer, CudaPinnedBuffer> || std::is_same_v<Buffer, HostBuffer>)
template<class Buffer> static inline void release_buffer(Buffer & buf) requires (std::is_same_v<Buffer, CudaMirroredBuffer> || std::is_same_v<Buffer, CudaPinnedBuffer> || std::is_same_v<Buffer, HostBuffer>)Defined in src/lib/sbio/execution/cuda.hh:329
Release the memory of a buffer allocated by allocate_buffer.
acquire_broker_view_impl
Section titled “acquire_broker_view_impl”static inline
template<typename Buffer> static inline void * acquire_broker_view_impl(Buffer & buf)Defined in src/lib/sbio/execution/cuda.hh:360
Get a host view of a device buffer (using the replica in the buffer).
publish_broker_view_impl
Section titled “publish_broker_view_impl”static inline
template<typename Buffer> static inline void publish_broker_view_impl(Buffer & buf, void * view, std::size_t nbytes)Defined in src/lib/sbio/execution/cuda.hh:385
Move the bytes IO read from staging buffer (host) to device buffer.
after_get_data
Section titled “after_get_data”static inline
static inline void after_get_data()Defined in src/lib/sbio/execution/cuda.hh:402
Synchronize the stream so that results are returned only once copy completes.
synchronize
Section titled “synchronize”static inline
static inline void synchronize(const char * msg)Defined in src/lib/sbio/execution/cuda.hh:441
get_stream
Section titled “get_stream”static inline
static inline cudaStream_t get_stream()Defined in src/lib/sbio/execution/cuda.hh:446
unit_device
Section titled “unit_device”static inline
static inline int unit_device()Defined in src/lib/sbio/execution/cuda.hh:467
The device associated to the calling parallel unit.
When CudaMemory is configured with devices (as opposed to just device) a given parallelunit will be assigned a particular device to use from those available. Otherwise, every parallel unit just uses device.
bind_device
Section titled “bind_device”static inline
static inline void bind_device()Defined in src/lib/sbio/execution/cuda.hh:484
Make the calling unit’s device the current device for the host htread.
Public Types
Section titled “Public Types”| Name | Description |
|---|---|
UnitOrdinalFn |
|
PlacementFor |
UnitOrdinalFn
Section titled “UnitOrdinalFn”using UnitOrdinalFn = std::size_t(*)()Defined in src/lib/sbio/execution/cuda.hh:168
PlacementFor
Section titled “PlacementFor”using PlacementFor = std::conditional_t< std::is_same_v< typename Descriptor::role, roles::Data >, CudaMirroredBuffer, std::conditional_t< std::is_same_v< typename Descriptor::role, roles::Table >, CudaPinnedBuffer, HostBuffer > >Type: std::conditional_t< std::is_same_v< typename Descriptor::role, roles::Data >, CudaMirroredBuffer, std::conditional_t< std::is_same_v< typename Descriptor::role, roles::Table >, CudaPinnedBuffer, HostBuffer > >
Defined in src/lib/sbio/execution/cuda.hh:171
Private Static Attributes
Section titled “Private Static Attributes”| Return | Name | Description |
|---|---|---|
std::atomic< int > |
m_device static |
|
std::vector< int > |
m_devices static |
Only written once at configure. |
std::atomic< UnitOrdinalFn > |
m_unit_ordinal static |
|
std::atomic< RegisterFn > |
m_register_buffer static |
|
std::atomic< DeregisterFn > |
m_deregister_buffer static |
|
std::shared_ptr< spdlog::logger > |
m_logger static |
m_device
Section titled “m_device”static
std::atomic< int > m_device { 0 }Defined in src/lib/sbio/execution/cuda.hh:567
m_devices
Section titled “m_devices”static
std::vector< int > m_devices {}Defined in src/lib/sbio/execution/cuda.hh:568
Only written once at configure.
m_unit_ordinal
Section titled “m_unit_ordinal”static
std::atomic< UnitOrdinalFn > m_unit_ordinal { nullptr }Type: std::atomic< UnitOrdinalFn >
Defined in src/lib/sbio/execution/cuda.hh:569
m_register_buffer
Section titled “m_register_buffer”static
std::atomic< RegisterFn > m_register_buffer { nullptr }Type: std::atomic< RegisterFn >
Defined in src/lib/sbio/execution/cuda.hh:570
m_deregister_buffer
Section titled “m_deregister_buffer”static
std::atomic< DeregisterFn > m_deregister_buffer { nullptr }Type: std::atomic< DeregisterFn >
Defined in src/lib/sbio/execution/cuda.hh:571
m_logger
Section titled “m_logger”static
std::shared_ptr< spdlog::logger > m_loggerDefined in src/lib/sbio/execution/cuda.hh:572
Private Static Methods
Section titled “Private Static Methods”| Return | Name | Description |
|---|---|---|
void |
check static inline |
static inline
static inline void check(cudaError_t err, const char * what)Defined in src/lib/sbio/execution/cuda.hh:555