Struct CpuBackend 

pub struct CpuBackend { /* private fields */ }

tensor4all’s owned CPU execution handle: one [CpuEngine].

tenferro no longer has a backend session or a CpuContext; a caller builds a Rayon pool, hands it to [CpuEngine::new], and keeps the engine for as long as it wants the thread policy, the buffer pool, the prepared-plan cache and the contraction scratch to stay warm. This type is that caller-held owner for tensor4all: the implicit frontend’s resource owner during the transition to engine-explicit APIs, which will replace it later.

It is not Clone. Cloning an execution domain was how the previous session API shared one engine between callers; with a caller-held engine the caller owns it outright, and a second execution domain needs a second backend.

CpuBackend::new resolves its worker count in this order: the value set_default_cpu_threads fixed, else the Rayon environment (RAYON_NUM_THREADS, then the deprecated RAYON_RS_NUM_CPUS), else the platform’s parallelism, which is evaluated under the main thread’s mask and may be lower than that mask’s CPU count because of cgroup quotas. A resolved count of one uses [CpuEngine::serial] and owns no pool. The count is fixed by the first successful CpuBackend::new of the process, or by the setter. If the pool or the Linux mask snapshot cannot be used, CpuBackend::new falls back to [CpuEngine::serial] for that backend only instead of panicking, and a later call retries.

Every pool this type builds on Linux confines its workers to the main thread’s allowed CPU set, captured as a snapshot when the pool is built. This applies to the pool’s workers only: operations that run inline on the caller (small operations, Rayon-worker entries that use the serial inline engine, and hataori lane threads) are not moved. A cpuset or CPU-hotplug change needs a new backend, not a continuous watcher.

CpuBackend::pinned is an opt-in per-worker pinning mode whose benefit is unmeasured. Disjoint pinned backends confine their pools’ workers; they do not isolate all execution of their engines, because an entry from any Rayon worker runs inline on that worker. A provider team created from a pinned worker inherits that worker’s one CPU.

Examples

use tensor4all_tensorbackend::CpuBackend;
use tenferro_cpu::CpuEngine;

// A serial backend runs every operation on the caller's thread.
let backend = CpuBackend::with_threads(1)?;
assert_eq!(backend.num_threads(), 1);

// A caller that already owns an engine can hand it in unchanged.
let backend = CpuBackend::from(CpuEngine::serial());
assert_eq!(backend.num_threads(), 1);

Implementations

impl CpuBackend

pub fn new() -> Self

Build a backend with the default thread policy.

The worker count is set_default_cpu_threads if it was called; else RAYON_NUM_THREADS (then the deprecated RAYON_RS_NUM_CPUS); else the platform’s parallelism, evaluated under the main thread’s mask, which may be lower than that mask’s CPU count because of cgroup quotas. The pool’s default 16 MiB worker stack, which the CPU providers need, is kept. The count is fixed by the first successful call, or by the setter.

On Linux the pool is built on a short-lived construction thread whose mask is first set to the main thread’s mask snapshot, so building from a worker of a pinned pool does not inherit that worker’s single CPU. The pool’s workers inherit the snapshot; inline operations of the caller are not moved.

The pool build and the Linux affinity snapshot are the fallible steps, and this constructor does not report them: if either cannot be used, this backend is [CpuEngine::serial] instead, which is correct but uses no worker threads, and the failure is not stored as the default. A later call retries the construction. Use CpuBackend::with_threads or CpuBackend::pinned when that difference matters.

Examples

use tensor4all_tensorbackend::CpuBackend;

let backend = CpuBackend::new();
assert!(backend.num_threads() >= 1);

pub fn with_threads(threads: usize) -> Result<Self, CpuThreadError>

Build a backend that may use threads workers.

One worker builds a [CpuEngine::serial] engine, which owns no pool, so it stays usable from inside another pool. Larger counts build a Rayon pool with [tenferro_cpu::Pool::builder], so provider code gets the worker stack it needs; the pool’s workers are confined to the main thread’s allowed CPU set snapshot on Linux.

Errors

Returns CpuThreadError::ZeroThreads for zero, CpuThreadError::Affinity when the mask snapshot or its application fails, CpuThreadError::ConstructionThread when the construction thread cannot start, and CpuThreadError::Pool when the pool cannot be built.

Examples

use tensor4all_tensorbackend::{CpuBackend, CpuThreadError};

assert!(matches!(
    CpuBackend::with_threads(0),
    Err(CpuThreadError::ZeroThreads)
));
let backend = CpuBackend::with_threads(2)?;
assert_eq!(backend.num_threads(), 2);

pub fn pinned(cpus: &[usize]) -> Result<Self, CpuThreadError>

Build a backend whose worker i is pinned to cpus[i].

This is opt-in per-worker pinning on Linux; its performance benefit is unmeasured. Every listed CPU must be in the main thread’s allowed set (Self::allowed_cpus), which lets a caller split that set into disjoint lists, one per engine. Each worker gets exactly its one CPU, so a provider team created from a pinned worker inherits that one CPU. A one-element list still builds a one-worker pool rather than a serial engine; inline operations of a one-worker engine still run on the caller.

The mask is a snapshot: a cpuset or CPU-hotplug change needs a new backend. Only the pool’s workers are pinned; lane threads and inline operations are not moved.

Errors

Returns CpuThreadError::EmptyCpuList for an empty list, CpuThreadError::DuplicateCpu for a repeated CPU, CpuThreadError::CpuNotAllowed for a CPU outside the allowed set, CpuThreadError::UnsupportedPlatform off Linux, CpuThreadError::Affinity when the snapshot or a worker’s mask fails, CpuThreadError::WorkerAffinity when a worker cannot be pinned to exactly its CPU, and CpuThreadError::Pool when the pool cannot be built.

Examples

use tensor4all_tensorbackend::{CpuBackend, CpuThreadError};

assert!(matches!(
    CpuBackend::pinned(&[]),
    Err(CpuThreadError::EmptyCpuList)
));
let allowed = CpuBackend::allowed_cpus()?;
let backend = CpuBackend::pinned(&allowed[..1])?;
assert_eq!(backend.num_threads(), 1);

pub fn allowed_cpus() -> Result<Vec<usize>, CpuThreadError>

The CPUs the main thread is allowed to run on right now, in ascending order.

This is a snapshot, read with sched_getaffinity(getpid()) on Linux. A caller can split it into disjoint lists and pass one list to each CpuBackend::pinned engine. Off Linux this reports CpuThreadError::UnsupportedPlatform.

Errors

Returns CpuThreadError::UnsupportedPlatform off Linux and CpuThreadError::Affinity when the affinity query fails.

Examples

use tensor4all_tensorbackend::CpuBackend;

let allowed = CpuBackend::allowed_cpus()?;
assert!(!allowed.is_empty());
assert!(allowed.windows(2).all(|pair| pair[0] < pair[1]));

pub fn num_threads(&self) -> usize

The number of workers operations on this backend may use.

Examples

use tensor4all_tensorbackend::CpuBackend;

assert_eq!(CpuBackend::with_threads(1)?.num_threads(), 1);

pub fn worker_cpus(&self) -> Vec<Vec<usize>>

The CPU set each worker of this backend’s pool reported when it started, by worker index.

Empty for a serial engine, for a caller-supplied engine (CpuBackend::from) and off Linux. It is a snapshot taken at pool build time, so it shows what the workers actually inherited — the main thread’s allowed set for CpuBackend::new and CpuBackend::with_threads, and its one CPU for CpuBackend::pinned.

Examples

use tensor4all_tensorbackend::CpuBackend;

let backend = CpuBackend::with_threads(1)?;
assert!(backend.worker_cpus().is_empty());

pub fn engine(&self) -> &CpuEngine

Borrow the engine this backend owns.

Examples

use tensor4all_tensorbackend::CpuBackend;

let backend = CpuBackend::with_threads(1)?;
assert_eq!(backend.engine().buffer_pool_len(), 0);

pub fn engine_mut(&mut self) -> &mut CpuEngine

Borrow the engine mutably, for one operation or one session.

Examples

use tensor4all_tensorbackend::CpuBackend;

let mut backend = CpuBackend::with_threads(1)?;
backend.engine_mut().reset_buffer_pool();
assert_eq!(backend.engine().buffer_pool_len(), 0);

pub fn into_engine(self) -> CpuEngine

Consume the backend and return the engine it owns.

A caller that hands engines to another owner, such as a lane executor, uses this instead of borrowing and cloning: CpuBackend deliberately does not clone an execution domain.

Examples

use tensor4all_tensorbackend::CpuBackend;

let engine = CpuBackend::with_threads(1)?.into_engine();
assert_eq!(engine.num_threads(), 1);

Methods from Deref<Target = CpuEngine>

pub fn plan_cache(&self) -> &CpuPlanCache

The plan cache this engine reads plans from.

Examples

let engine = tenferro_cpu::CpuEngine::serial();
assert_eq!(engine.plan_cache().stats().entries, 0);

pub fn allocation_domain(&self) -> Option<AllocationDomainId>

The shared allocation domain this engine is bound to, if any.

Examples

assert_eq!(tenferro_cpu::CpuEngine::serial().allocation_domain(), None);

pub fn shared_allocation_domain( &self, ) -> Option<&Arc<dyn SharedTensorAllocationDomain>>

The shared allocator this engine is bound to, if any.

Examples

assert!(tenferro_cpu::CpuEngine::serial().shared_allocation_domain().is_none());

pub fn set_batch_policy(&mut self, policy: BatchPolicy) -> BatchPolicy

Replace the default batch policy, returning the previous one.

For a one-operation override use [crate::with_batch_policy].

Examples

use tenferro_cpu::{CpuBatchPolicy, CpuBatchStrategy, CpuEngine};

let mut engine = CpuEngine::serial();
let previous = engine.set_batch_policy(CpuBatchPolicy::new(CpuBatchStrategy::Outer));
assert_eq!(previous.strategy, CpuBatchStrategy::Auto);

pub fn batch_policy(&self) -> BatchPolicy

The default batch policy of this engine.

Examples

use tenferro_cpu::{CpuBatchStrategy, CpuEngine};
assert_eq!(CpuEngine::serial().batch_policy().strategy, CpuBatchStrategy::Auto);

pub fn num_threads(&self) -> usize

The number of workers operations may use: the pool size, or one.

Examples

assert_eq!(tenferro_cpu::CpuEngine::serial().num_threads(), 1);

pub fn has_team(&self) -> bool

Whether the engine runs its parallel work on a reserved threadteam::Team.

true for an engine built from a pool of two or more workers whose members were reserved. false for [CpuEngine::serial], for a one-worker pool, and for a pool whose members could not be reserved within about ten seconds of construction (the engine then runs serially and [CpuEngine::num_threads] reports one). Benchmarks should assert this after construction.

Examples

assert!(!tenferro_cpu::CpuEngine::serial().has_team());
let pool = tenferro_cpu::Pool::builder().threads(2).build()?;
assert!(tenferro_cpu::CpuEngine::new(pool).has_team());

pub fn is_broken(&self) -> bool

Whether an earlier operation broke this engine (see [CpuEngineError::Broken]).

Examples

assert!(!tenferro_cpu::CpuEngine::serial().is_broken());

pub fn exec(&mut self) -> Result<Exec<'_>, Error>

The provider execution bundle for this engine: a placement token over every worker of its pool, its workspace and its batch policy.

A downstream kernel that calls a provider (cpueinsum, tcontract, tlinalg) directly passes this bundle by &mut, so it uses the same threads, budget and batch policy as tenferro. The bundle is not Clone: it is the engine’s whole thread budget for that one call, and the provider splits it. Getting it applies the same checks as an operation.

Errors

Returns [crate::Error::RuntimeState] with a [CpuEngineError] source when the engine is broken.

Examples

let mut engine = tenferro_cpu::CpuEngine::serial();
let exec = engine.exec()?;
assert_eq!(exec.budget(), 1);

pub fn buffer_pool_len(&self) -> usize

Number of retained buffers in the engine’s buffer pool.

Examples

assert_eq!(tenferro_cpu::CpuEngine::serial().buffer_pool_len(), 0);

pub fn buffer_pool_stats(&self) -> BufferPoolStats

Retained-buffer statistics of the engine’s buffer pool.

Examples

assert_eq!(tenferro_cpu::CpuEngine::serial().buffer_pool_stats().buffers, 0);

pub fn buffer_pool_cache_stats(&self) -> CacheStats

The buffer pool as cache statistics (entries and retained bytes).

Examples

assert_eq!(tenferro_cpu::CpuEngine::serial().buffer_pool_cache_stats().entries, 0);

pub fn buffer_pool_limit_bytes(&self) -> usize

The retention limit of recycled buffers, in retained capacity bytes.

Examples

let mut engine = tenferro_cpu::CpuEngine::serial();
engine.set_buffer_pool_limit_bytes(0);
assert_eq!(engine.buffer_pool_limit_bytes(), 0);

pub fn set_buffer_pool_limit_bytes( &mut self, max_retained_capacity_bytes: usize, )

Bound the bytes of recycled buffers and contraction scratch retained between operations; zero disables retention.

Examples

let mut engine = tenferro_cpu::CpuEngine::serial();
engine.set_buffer_pool_limit_bytes(1 << 20);
assert_eq!(engine.buffer_pool_limit_bytes(), 1 << 20);

pub fn reset_buffer_pool(&mut self)

Drop every retained buffer and contraction scratch.

Examples

let mut engine = tenferro_cpu::CpuEngine::serial();
engine.reset_buffer_pool();
assert_eq!(engine.buffer_pool_len(), 0);

Trait Implementations

impl Debug for CpuBackend

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CpuBackend

fn default() -> Self

Returns the “default value” for a type.

impl Deref for CpuBackend

type Target = CpuEngine

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl DerefMut for CpuBackend

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

impl From<CpuEngine> for CpuBackend

fn from(engine: CpuEngine) -> Self

Converts to this type from the input type.