Expand description
Traced graph runtime and extension dispatch infrastructure for tenferro.
This crate owns graph construction, lowering to execution IR, graph
execution, and backend-parametric extension runtime dispatch. Standard
operations are lowered through the runtime’s internal operation vocabulary;
tensor storage and backend kernels live in tenferro-tensor.
Use this crate directly when you want concrete tensor helpers or reusable
traced graph execution without depending on tenferro-ad. Start with
TypedTensor when the scalar type is fixed in Rust, Tensor when dtype
is selected at runtime, and TracedTensor plus GraphCompiler and
Runtime when the same expression should be compiled once and run
repeatedly. Operation-family crates such as tenferro-einsum,
tenferro-linalg, and tenferro-fft register extension runtimes through
runtime engine registrations when compiled execution reaches those
operations.
User-facing guides live at https://tensor4all.org/tenferro-rs/guides/choosing-an-api.html and https://tensor4all.org/tenferro-rs/guides/execution-models.html.
add has no settings, so it has no options type and no _with form; only
operations with semantic options (such as dot_general) have one:
use tenferro_cpu::CpuEngine;
use tenferro_runtime::{CoreOps, Tensor};
let a = Tensor::from_vec_col_major(vec![2], vec![1.0_f64, 2.0]).unwrap();
let _ = a.add_with(&a, &mut CpuEngine::serial());use tenferro_runtime::AddOptions;§Examples
use tenferro_cpu::CpuEngine;
use tenferro_runtime::{TraceContext, ExecutionEngine, GraphCompiler, Runtime, TracedTensor};
let trace = TraceContext::new();
let x = trace.from_vec_col_major(vec![2], vec![1.0_f64, 2.0]).unwrap();
let y = (&x + &x).unwrap();
let mut compiler = GraphCompiler::new();
let program = compiler.compile(&y).unwrap();
// One caller-held engine runs every slot of the prepared program.
let mut engine = CpuEngine::serial();
let mut builder = Runtime::builder();
builder
.register_engine(tenferro_cpu::runtime_engine_registration(&engine).unwrap())
.unwrap();
let runtime = builder.build().unwrap();
let prepared = runtime.prepare(&program, &[], &[engine.target()]).unwrap();
let out = runtime
.run(&mut [&mut engine], &prepared, &[])
.unwrap()
.pop()
.unwrap()
.into_tensor(&mut engine)
.unwrap();
assert_eq!(out.as_slice::<f64>().unwrap(), &[2.0, 4.0]);Re-exports§
pub use error::ContextId;pub use error::Error;pub use error::ErrorPhase;pub use error::Result;pub use error::RuntimeFailureReasonRef;pub use error::ShapeConstraintEvalError;pub use extension_cache::ExtensionCacheKey;pub use extension_cache::ExtensionCacheLimits;pub use extension_cache::ExtensionCacheSelector;pub use extension_cache::ExtensionCacheStore;pub use graph::CompiledGraph;pub use graph::GraphCompiler;pub use runtime::assemble_executable_engine_registration;pub use runtime::assemble_preparation_only_engine_registration;pub use runtime::CacheInFlightBehavior;pub use runtime::CacheOwnerError;pub use runtime::CacheOwnerFailure;pub use runtime::CacheOwnerId;pub use runtime::CoreCapabilityBundle;pub use runtime::CoreCapabilityBundleBuilder;pub use runtime::CoreCapabilityKind;pub use runtime::CorePrepareContext;pub use runtime::Determinism;pub use runtime::DotGeneralPreparation;pub use runtime::DotGeneralPrepareRequest;pub use runtime::ElementwisePrepareRequest;pub use runtime::ElementwiseRuntime;pub use runtime::EngineExecutionContractError;pub use runtime::EngineId;pub use runtime::EngineRegistration;pub use runtime::EngineRegistrationMetadata;pub use runtime::EngineSnapshotView;pub use runtime::EngineTarget;pub use runtime::ErasedExecutionContext;pub use runtime::EventDomainDriver;pub use runtime::EventDomainError;pub use runtime::EventDomainId;pub use runtime::EventDomainOperation;pub use runtime::EventDomainRun;pub use runtime::EventToken;pub use runtime::ExecutableEngineRegistrationConfig;pub use runtime::ExecutionContextIdentity;pub use runtime::ExecutionContextMismatch;pub use runtime::ExecutionEngine;pub use runtime::ExecutionPolicy;pub use runtime::ExecutionPolicyError;pub use runtime::ExtensionEngine;pub use runtime::ExtensionModule;pub use runtime::ExtensionModuleError;pub use runtime::ExtensionModuleId;pub use runtime::ExtensionModuleRegistrar;pub use runtime::ExtensionPlanningConfig;pub use runtime::ExtensionPrepareRequest;pub use runtime::HardwareClassId;pub use runtime::IdentityError;pub use runtime::IdentityKind;pub use runtime::ImmediateEventDomainDriver;pub use runtime::IndexingPrepareRequest;pub use runtime::IndexingRuntime;pub use runtime::InputIngressContract;pub use runtime::InputIngressContractError;pub use runtime::InputPlacementContract;pub use runtime::InputSignature;pub use runtime::InputSignatureContract;pub use runtime::InputSignatureEntry;pub use runtime::InputSignatureError;pub use runtime::InputSpecializationProjection;pub use runtime::InputSpecializationRequirements;pub use runtime::InputSpecializationRequirementsBuilder;pub use runtime::InputSpecializationRequirementsError;pub use runtime::LayoutClass;pub use runtime::LayoutPrepareRequest;pub use runtime::LayoutProjection;pub use runtime::LayoutRuntime;pub use runtime::LayoutSpecialization;pub use runtime::Output;pub use runtime::PlacementConstraintError;pub use runtime::PlacementProjection;pub use runtime::PlacementSpecialization;pub use runtime::PreparationKeySummary;pub use runtime::PreparationOnlyEngineRegistrationConfig;pub use runtime::PrepareCapability;pub use runtime::PrepareError;pub use runtime::PrepareOptions;pub use runtime::PrepareOptionsKey;pub use runtime::Prepared;pub use runtime::PreparedOperation;pub use runtime::PreparedOperationBinding;pub use runtime::PreparedOperationHandle;pub use runtime::PreparedOperationPlan;pub use runtime::PreparedPlanCacheLimits;pub use runtime::PreparedPlanCacheStats;pub use runtime::ProgramPlacementConstraint;pub use runtime::ProviderContractError;pub use runtime::ProviderDeviceIdentity;pub use runtime::ProviderId;pub use runtime::RankRequirement;pub use runtime::ReductionPrepareRequest;pub use runtime::ReductionRuntime;pub use runtime::RegistrationIdentity;pub use runtime::RegistrationKey;pub use runtime::ResidentOutputContract;pub use runtime::ResolvedPlanningConfig;pub use runtime::ResolvedPlanningKey;pub use runtime::ResolvedProgramPlacement;pub use runtime::Runtime;pub use runtime::RuntimeCacheError;pub use runtime::RuntimeCacheOwner;pub use runtime::RuntimeCacheStats;pub use runtime::RuntimeConfigBuilder;pub use runtime::RuntimeConfigError;pub use runtime::RuntimeConfigSnapshot;pub use runtime::RuntimeEpoch;pub use runtime::RuntimeId;pub use runtime::RuntimeInputContract;pub use runtime::RuntimeReconfiguration;pub use runtime::RuntimeReconfigureError;pub use runtime::RuntimeStateError;pub use runtime::SpecializationError;pub use runtime::SpecializationProjection;pub use runtime::SpecializationRequirements;pub use runtime::StorageClass;pub use runtime::TransferEndpoint;pub use runtime::TransferError;pub use runtime::TransferProvider;pub use runtime::TransferProviderContractError;pub use runtime::TransferRequest;pub use runtime::UnsupportedReason;pub use traced::TracedTensor;
Modules§
- error
- Error types for the tenferro runtime crate.
- extension
- Public surface for out-of-tree extension primitives.
- extension_
cache - Generic runtime caches for extension executors.
- graph
- Graph compilation APIs.
- prelude
- Common runtime tensor and graph types plus the core operation grammar.
- program
- Immutable backend-neutral semantic programs.
- runtime
- sym_dim
- Symbolic dimensions of traced values.
- traced
Structs§
- Cache
Stats - Entry, retained-byte, and event accounting for one cache.
- Compiler
Options - DotGeneral
Config - DotGeneral dimension configuration.
- Extension
Execution Context - Backend and cache state passed to one prepared extension execution.
- Gather
Config - StableHLO gather dimension configuration.
- Optimizer
Config - PadConfig
- StableHLO pad configuration.
- Scatter
Config - StableHLO scatter dimension configuration.
- Slice
Config - Slice configuration.
- SymDim
- A symbolic tensor dimension expression used to build shape-agnostic graphs.
- Tensor
- Dynamic tensor over the supported scalar types.
- Tensor
Value - Owned tensor value with one move-only physical owner and metadata-only layout.
- Trace
Context - Shared, cloneable handle to one mutable semantic trace.
- Trace
Value - Opaque value handle owned by one
TraceContext. - Traced
Graph - Immutable backend-neutral output of one completed trace.
- Traced
Slice Builder - Rank-preserving traced tensor slicing builder.
- Typed
Tensor - Owned compact column-major typed tensor.
- Typed
Tensor View - Read-only borrowed view of typed tensor storage with arbitrary strides.
Enums§
- Compare
Dir - Comparison direction.
- DType
- Runtime scalar dtype tag.
- Memory
Kind - Memory location for tensor storage.
- Shape
Relation - A relation between two symbolic shape expressions.
- Tensor
Read - Read-only tensor input accepted by synchronous eager kernels.
- Tensor
View - Dynamic read-only borrowed tensor view.
Traits§
- CoreOps
- Core tensor operations on anything that implements
AsView: owned and borrowed tensors, typed tensors and strided views. Results areTensors for erased inputs and typed tensors for typed inputs. Axes are signed (-1is the last axis); the engine is always the last argument. - Core
Repr - The core result types for an input representation.
- Engine
- The execution trait every tensor operation runs on.
- Select
Condition - A representation that can be the condition of
where_select: the erased representation (checked at run time) orTyped<bool>. - Select
Operand - The branches of
where_selectfor a condition representationC: anyAsViewinput, available only whenCis aSelectCondition, so a typed non-boolreceiver does not offerwhere_select. - Tensor
Scalar - Sealed trait for scalar types that can be stored in a
Tensor.