MPIExecution

MPIExecution

#include <sbio/execution/mpi.hh>

class MPIExecution

Defined in src/lib/sbio/execution/mpi.hh:58

Inherits: Execution< MPIExecution >

A basic shared MPI execution policy.

This policy implements some basic functionality for MPI-based workflows. The main features are:

• Allocations for types of the [IndexRole](api-sbio-IndexRole.md#indexrole) are made over shared MPI windows.

• Shared values are accordingly synchronized via the sync group mechanism on pre/post update hooks.

• As [IndexRole](api-sbio-IndexRole.md#indexrole) buffers are shared, redundant INDEXING is avoided by only allowing that Broker state for rank 0.

• Steps are processed in a simple round-robin fashion with a modulo world-size increment for each rank.

• Exhaustion of available indexed capacity automatically retriggers INDEXING if applicable for the Brokers.

List of all members

Name	Kind	Owner
MaxInactiveRanks	variable	Declared here
ParallelSupport	variable	Declared here
configure_impl	function	Declared here
allocate_storage_impl	function	Declared here
allocate_impl_helper	function	Declared here
pre_update_impl	function	Declared here
post_update_impl	function	Declared here
should_index_impl	function	Declared here
is_current_rank_inactive	function	Declared here
next_impl	function	Declared here
BufferTypeFor	typedef	Declared here
m_world_comm	variable	Declared here
m_active_comm	variable	Declared here
m_active_rank	variable	Declared here
m_active_size	variable	Declared here
m_inactive_ranks	variable	Declared here
m_num_inactive_ranks	variable	Declared here
m_main_rank	variable	Declared here
m_main_rank_loops	variable	Declared here
m_shmem_comm	variable	Declared here
m_rank	variable	Declared here
m_size	variable	Declared here
m_event_idx	variable	Declared here
m_logger	variable	Declared here
ParallelSupport	variable	Inherited from Execution
result_memory_space	function	Inherited from Execution
configure	function	Inherited from Execution
allocate_storage	function	Inherited from Execution
allocate_storage	function	Inherited from Execution
get_block_from_pool	function	Inherited from Execution
should_index	function	Inherited from Execution
should_process	function	Inherited from Execution
on_step	function	Inherited from Execution
pre_update	function	Inherited from Execution
post_update	function	Inherited from Execution
execute_read	function	Inherited from Execution
acquire_broker_view	function	Inherited from Execution
allocate_group_storage	function	Inherited from Execution
get_data	function	Inherited from Execution
get_data_steps	function	Inherited from Execution
next	function	Inherited from Execution
BufferTypeFor	typedef	Inherited from Execution

Inherited from Execution

Kind	Name	Description
variable	ParallelSupport static constexpr	The set of parallelization methods supported by the Execution policy.
function	result_memory_space static inline constexpr	The memory space that results (via get_data) are returned in.
function	configure static inline
function	allocate_storage static inline requires FormatTraits<FTraits, IO, Derived>	Controls the allocation of storage.
function	allocate_storage static inline requires FormatTraits<FTraits, IO, Derived>
function	get_block_from_pool static inline
function	should_index static inline	Decide whether or not this StreamBroker should proceed to the indexing state.
function	should_process static inline	Decide whether or not this StreamBroker should process this particular step.
function	on_step static inline	An opportunity to provide explicit synchronization immediately after every step.
function	pre_update static inline	An opportunity to provide explicit synchronization before any Storage changes.
function	post_update static inline	An opportunity to provide explicit synchronization after any Storage changes.
function	execute_read static inline
function	acquire_broker_view static inline
function	allocate_group_storage static inline requires FormatTraits<FTraits, IO, Derived>
function	get_data static inline	Run the data fetching and then querying of the filled buffers.
function	get_data_steps static inline	Run the data fetching and then querying of the filled buffers, for a BATCh of steps.
function	next static inline	Request the next step index to read data for.
typedef	BufferTypeFor	Sub-classes must define the used buffer types.

Public Static Attributes

Return	Name	Description
constexpr std::size_t	MaxInactiveRanks static constexpr
constexpr std::bitset< static_cast< std::size_t >(ParallelizationMethods::NUM_METHODS) >	ParallelSupport static constexpr

---

MaxInactiveRanks

static constexpr

constexpr std::size_t MaxInactiveRanks { 1024 }

Defined in src/lib/sbio/execution/mpi.hh:60

---

ParallelSupport

static constexpr

constexpr std::bitset< static_cast< std::size_t >(ParallelizationMethods::NUM_METHODS) > ParallelSupport { 0x2 }

Defined in src/lib/sbio/execution/mpi.hh:83

Public Static Methods

Return	Name	Description
void	configure_impl static inline
auto	allocate_storage_impl static inline requires FormatTraits<FTraits, IO, MPIExecution>
auto	allocate_impl_helper static inline
void	pre_update_impl static inline	This policy establishes a shared memory window over index data. The pre_update hook includes a fence on that window so we can synchronize updates to the memory across ranks.
void	post_update_impl static inline	Paired with the pre_update hook, the post_update ensures the synchronization of [IndexRole](api-sbio-IndexRole.md#indexrole) storage is completed.
bool	should_index_impl static inline	Check on if indexing should be done by the Broker.
bool	is_current_rank_inactive static inline
FTraits::StepIdxType	next_impl static inline	Retrieve the next step index to process.

---

configure_impl

static inline

static inline void configure_impl(constConfig & config)

Defined in src/lib/sbio/execution/mpi.hh:85

---

allocate_storage_impl

static inline requires FormatTraits<FTraits, IO, MPIExecution>

template<IsTypeList Requirements, class IO, class FTraits> static inline auto allocate_storage_impl(constAllocationRequest< FTraits > & request) requires FormatTraits<FTraits, IO, MPIExecution>

Defined in src/lib/sbio/execution/mpi.hh:179

---

allocate_impl_helper

static inline

template<typename... Descriptors, class FTraits> static inline auto allocate_impl_helper(TypeList< Descriptors... >, constAllocationRequest< FTraits > & request)

Defined in src/lib/sbio/execution/mpi.hh:192

---

pre_update_impl

static inline

template<classRole, classStorageT> static inline void pre_update_impl(StorageT & storage)

Defined in src/lib/sbio/execution/mpi.hh:245

This policy establishes a shared memory window over index data. The pre_update hook includes a fence on that window so we can synchronize updates to the memory across ranks.

Parameters

Parameter	Type	Description
storage	[StorageT](api-sbio-Execution.md#should_process) &	The storage to synchronize.

---

post_update_impl

static inline

template<classRole, classStorageT, classSyncT> static inline void post_update_impl(StorageT & storage, SyncT && sync_vars, IOStatus status)

Defined in src/lib/sbio/execution/mpi.hh:286

Paired with the pre_update hook, the post_update ensures the synchronization of [IndexRole](api-sbio-IndexRole.md#indexrole) storage is completed.

Any remaining sync_vars are synchronized via a simple MPI_Bcast.

Parameters

Parameter	Type	Description
storage	[StorageT](api-sbio-Execution.md#should_process) &	The storage to synchronize.
sync_vars	[SyncT](api-sbio-Execution.md#should_process) &&	The attributes that require synchronization
status	[IOStatus](api-sbio.md#iostatus)	The IOStatus from the INDEXING state.

---

should_index_impl

static inline

static inline bool should_index_impl()

Defined in src/lib/sbio/execution/mpi.hh:385

Check on if indexing should be done by the Broker.

File indexing is performed only by rank 0. This execution policy provides a shared memory buffer for the index storage - synchronization of that buffer ensures that other ranks will see any data rank 0 puts into it.

Returns

true for rank 0, else false.

---

is_current_rank_inactive

static inline

static inline bool is_current_rank_inactive()

Defined in src/lib/sbio/execution/mpi.hh:387

---

next_impl

static inline

template<class FTraits, classIndexTrigger> static inline FTraits::StepIdxType next_impl(typename FTraits::StepIdxType & max_capacity, IndexTrigger && trigger)

Defined in src/lib/sbio/execution/mpi.hh:413

Retrieve the next step index to process.

Steps are processed using a fixed offset of the rank, with increments of the world size. E.g. for a world size of 2:

• Step 1: Rank 0 processes 0, Rank 1 processes 1

• Step 2: Rank 0 processes 2, Rank 1 processes 3 … and so on.

Returns

The next step to process using the fixed offset of the world size.

Parameters

Parameter	Type	Description
max_capacity	[typename](api-sbio-Execution.md#should_process) FTraits::StepIdxType &	The current max capacity (i.e., already indexed steps).
trigger	[IndexTrigger](api-sbio-Execution.md#should_process) &&	A callback to reindex (if appropriate) when capacity is exhausted.

Public Types

Name	Description
BufferTypeFor	Map IndexRole to MPISharedBuffer and all other roles to HostBuffer.

---

BufferTypeFor

using BufferTypeFor = std::conditional_t< std::is_same_v< typename Descriptor::role, IndexRole >||std::is_same_v< typename Descriptor::role, GroupRole >||std::is_same_v< typename Descriptor::hint, Shareable >, MPISharedBuffer, HostBuffer >

Defined in src/lib/sbio/execution/mpi.hh:66

Map IndexRole to MPISharedBuffer and all other roles to HostBuffer.

Private Static Attributes

Return	Name	Description
MPI_Comm	m_world_comm static	Communicator for synchronizing across the whole MPI world.
MPI_Comm	m_active_comm static
int	m_active_rank static
int	m_active_size static
int	m_inactive_ranks static
std::size_t	m_num_inactive_ranks static
int	m_main_rank static
bool	m_main_rank_loops static
MPI_Comm	m_shmem_comm static	Communicator used when generating shareable buffers.
int	m_rank static	This processes rank in the MPI world.
int	m_size static	The size of the MPI world.
std::size_t	m_event_idx static	Rank-local index within the MPI world’s set of indices to distribute.
std::shared_ptr< spdlog::logger >	m_logger static

---

m_world_comm

static

MPI_Comm m_world_comm {  }

Defined in src/lib/sbio/execution/mpi.hh:448

Communicator for synchronizing across the whole MPI world.

---

m_active_comm

static

MPI_Comm m_active_comm {  }

Defined in src/lib/sbio/execution/mpi.hh:449

---

m_active_rank

static

int m_active_rank { -1 }

Defined in src/lib/sbio/execution/mpi.hh:450

---

m_active_size

static

int m_active_size { -1 }

Defined in src/lib/sbio/execution/mpi.hh:451

---

m_inactive_ranks

static

int m_inactive_ranks {}

Defined in src/lib/sbio/execution/mpi.hh:452

---

m_num_inactive_ranks

static

std::size_t m_num_inactive_ranks { 0 }

Defined in src/lib/sbio/execution/mpi.hh:453

---

m_main_rank

static

int m_main_rank { 0 }

Defined in src/lib/sbio/execution/mpi.hh:454

---

m_main_rank_loops

static

bool m_main_rank_loops {  }

Defined in src/lib/sbio/execution/mpi.hh:455

---

m_shmem_comm

static

MPI_Comm m_shmem_comm {  }

Defined in src/lib/sbio/execution/mpi.hh:459

Communicator used when generating shareable buffers.

---

m_rank

static

int m_rank { -1 }

Defined in src/lib/sbio/execution/mpi.hh:460

This processes rank in the MPI world.

---

m_size

static

int m_size { -1 }

Defined in src/lib/sbio/execution/mpi.hh:461

The size of the MPI world.

---

m_event_idx

static

std::size_t m_event_idx { 0 }

Defined in src/lib/sbio/execution/mpi.hh:465

Rank-local index within the MPI world’s set of indices to distribute.

---

m_logger

static

std::shared_ptr< spdlog::logger > m_logger

Defined in src/lib/sbio/execution/mpi.hh:467