Skip to main content

Crate tenferro_mpi

Crate tenferro_mpi 

Source
Expand description

Blocking point-to-point MPI transport for tenferro host tensors.

This crate moves one tensor value between two MPI ranks. It borrows the caller’s communicator and owns nothing else: it never calls MPI_Init or MPI_Finalize, never creates MPI_COMM_WORLD, never duplicates a communicator, and never transfers between host and device. Distributed scheduling stays with the caller (for example hataori); this is transport only.

§What can be sent

[Transport] covers the three values the adapter accepts, with dtype and shape preserved:

A Gpu-representation tensor does not implement [Transport], so an accidental device transfer is a compile error. Payloads must be compact column-major host slices; the adapter never materializes a view.

§Wire format

Each call sends a header, waits for the receiver’s ack, and only then sends the payload, so a receiver can refuse a tensor without ever receiving its payload and a sender always learns the outcome. Every frame uses the caller’s single tag, which therefore belongs to that pair for the duration of the call. The format is versioned (v1) and is specified in docs/design/mpi-transport.md of the tenferro-rs repository; the pieces a caller can observe are:

  • supported dtypes are bool, i32, i64, f32, f64, Complex32 and Complex64, plus nothing else: a caller-owned external scalar needs an explicit codec, because a process-local TypeId cannot be a wire identifier;
  • bool travels as one byte per element, 0 or 1, validated on receipt;
  • the payload is the compact column-major element representation in the host’s byte order, and the header carries a byte-order marker, so a peer with a different byte order is rejected explicitly rather than mis-decoded;
  • one message carries at most MAX_WIRE_ELEMENTS elements, and the payload also has to fit one MPI byte count; larger tensors are rejected, never chunked or truncated.

§Backend features

This is a workspace member with default = []: a default build compiles the wire format, the error taxonomy and the MpiBackend trait but selects no MPI binding, so the root cargo test --workspace graph needs neither MPI headers nor libclang.

Exactly one backend feature is selected when a caller wants transport. Both build the published rsmpi-rt 0.8.2 package, whose library target is still mpi:

  • system enables rsmpi-rt’s bindgen mpi-sys-backend (crates.io rsmpi-sys 0.2.4) and therefore needs system MPI headers and libclang at build time.
  • runtime enables its MPIABI mpi-rt-sys-backend (crates.io mpi-rt-sys 0.1.0), which loads MPI at run time and needs no headers or libclang.

The two backends are mutually exclusive; rsmpi-rt itself rejects enabling both backend features at once. engine compiles the protocol with no binding at all.

§Examples

The transport entry points exist only with a backend, so the example below is compiled in the system and runtime lanes:

use tenferro_mpi::mpi::traits::*;
use tenferro_mpi::{recv, send, Error};
use tenferro_tensor::{DynRank, Host, Tensor, TypedTensor};

let universe = tenferro_mpi::mpi::initialize().unwrap();
let world = universe.world();

let typed =
    TypedTensor::<f64, DynRank, Host>::from_host_vec_col_major(vec![2, 2], vec![1.0, 2.0, 3.0, 4.0])?;
let erased = Tensor::from_typed(
    TypedTensor::<f64, DynRank, Host>::from_host_vec_col_major(vec![2, 2], vec![1.0, 2.0, 3.0, 4.0])?
        .into_dynamic(),
);

// A blocking send to the calling rank could never be received, so it is refused
// before any message.
assert!(matches!(send(world.this_process(), 5, &typed), Err(Error::SelfMessage)));

// One rank sends while the other receives: a blocking send needs a posted
// receive, so both ranks may not send at each other.
if world.size() >= 2 {
    if world.rank() == 0 {
        send(world.process_at_rank(1), 5, &typed)?;
        send(world.process_at_rank(1), 6, &erased)?;
    } else if world.rank() == 1 {
        let typed_back: TypedTensor<f64, DynRank, Host> = recv(world.process_at_rank(0), 5)?;
        let erased_back: Tensor = recv(world.process_at_rank(0), 6)?;
        assert_eq!(typed_back.get(&[1, 1])?, &4.0);
        assert_eq!(erased_back.shape(), &[2, 2]);
    }
}

Enums§

AbortReason
Why a sender aborted before offering a payload.
Error
Failure of one transport call.
RejectReason
Why a receiver refused a tensor before its payload was sent.

Constants§

MAX_WIRE_ELEMENTS
Largest element count one message may carry.
MAX_WIRE_RANK
Largest rank a header can carry.
PROTOCOL_VERSION
Protocol version carried in every frame.

Traits§

MpiBackend
The MPI operations one transport call performs.

Type Aliases§

Result
Result alias for this crate.
Tag
A point-to-point tag, matching the MPI and rsmpi Tag type.