Skip to main content

Getting Started

For multi-socket hosts, containers, or schedulers with restricted cpusets, read CPU Execution and Thread Pools before choosing the pool you hand to CpuEngine.

tenferro supports tensor computation without autodiff, immediate execution with optional backward() on scalar losses, functional eager grad, vjp, and jvp, traced graph execution, einsum, linear algebra, and CUDA execution through the feature-gated CUDA backend.

If an older example no longer compiles

Start with the API migration guide when a removed module, free function, constructor, or changed signature appears in an existing example. It maps the common pre-0.3 spellings to the current trait-based API.

Mental Model

tenferro has three independent choices. Pick the smallest tensor layer that matches the program, decide whether work should run immediately or through a compiled traced program, then choose a CPU or CUDA backend explicitly.

Choice Question Common starting point
Tensor layer What kind of value do I pass around? TypedTensor<T> for typed CPU values, Tensor for runtime dtype
Execution model When does computation run? Direct/eager for ordinary code and eager AD; traced for compiled graph reuse
Engine Where does computation run, and who owns it? CpuEngine; CudaEngine::open and explicit upload/download for CUDA

CUDA, eager execution, and traced graphs are not competing APIs. They compose when a workflow needs them, but the first CPU program below only needs the runtime crate and the CPU crate.

Setup

Start with the runtime crate, CPU backend crate, and standard linear algebra extension from crates.io:

[dependencies]
tenferro-runtime = "0.12"
tenferro-cpu = "0.12"
tenferro-linalg = "0.12"

For development against a local checkout of this repository, use path dependencies instead:

[dependencies]
tenferro-runtime = { path = "/path/to/tenferro-rs/crates/tenferro-runtime" }
tenferro-cpu = { path = "/path/to/tenferro-rs/crates/tenferro-cpu" }
tenferro-linalg = { path = "/path/to/tenferro-rs/crates/tenferro-linalg" }

If you create a scratch binary crate inside the tenferro-rs checkout, add an empty [workspace] table to that scratch crate’s Cargo.toml. Otherwise Cargo will treat it as an unlisted member of the parent workspace and report a workspace-membership error.

For a scratch crate directly under the checkout root, relative paths are less error-prone:

[workspace]

[dependencies]
tenferro-runtime = { path = "../crates/tenferro-runtime" }
tenferro-cpu = { path = "../crates/tenferro-cpu" }
tenferro-linalg = { path = "../crates/tenferro-linalg" }

The first build still needs network access unless dependencies are already vendored or cached. The workspace pins git dependencies and uses crates.io packages even when the tenferro crates themselves are local path dependencies. The CPU stack may take several minutes to compile the first time on a fresh machine; later incremental builds are much faster.

A CpuEngine always uses faer: the CPU stack declares no backend feature, so there is nothing to select and no system BLAS/LAPACK to install. External CPU providers may replace only a provider bundle slot, such as dot_general; they do not replace the CPU linalg path, which stays faer-backed.

Add tenferro-ad, tenferro-einsum, tenferro-linalg, tenferro-fft, or tenferro-gpu when a workflow needs those layers. Enable autodiff on operation crates such as tenferro-linalg when extension AD rules are needed. For CPU-only eager or traced AD, add:

[dependencies]
tenferro-ad = { path = "/path/to/tenferro-rs/crates/tenferro-ad" }

Enable concrete backend features such as cuda on each crate that needs GPU support:

[dependencies]
tenferro-ad = { path = "/path/to/tenferro-rs/crates/tenferro-ad", features = ["cuda"] }
tenferro-gpu = { path = "/path/to/tenferro-rs/crates/tenferro-gpu", features = ["cuda"] }
tenferro-linalg = { path = "/path/to/tenferro-rs/crates/tenferro-linalg", features = ["autodiff", "cuda"] }

Quickstart A: Direct Tensor And Linalg

use tenferro_cpu::CpuEngine;
use tenferro_linalg::prelude::*;
use tenferro_runtime::prelude::*;

fn assert_close(actual: &[f64], expected: &[f64]) {
    assert_eq!(actual.len(), expected.len());
    for (index, (actual, expected)) in actual.iter().zip(expected).enumerate() {
        let error = (actual - expected).abs();
        assert!(
            error < 1.0e-12,
            "value {index}: actual={actual}, expected={expected}, error={error}"
        );
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut engine = CpuEngine::serial();

    let a = TypedTensor::<f64>::from_vec_col_major(vec![2, 2], vec![3.0, 0.0, 0.0, 1.0])?;
    let identity = TypedTensor::<f64>::from_vec_col_major(vec![2, 2], vec![1.0, 0.0, 0.0, 1.0])?;
    let rhs = TypedTensor::<f64>::from_vec_col_major(vec![2, 1], vec![6.0, 2.0])?;

    // One borrowed engine carries every operation, the plan cache and the
    // buffer pool.
    let product = a.matmul(&identity, &mut engine)?;
    let solution = product.solve(&rhs, &mut engine)?;
    let singular_values = product.svdvals(&mut engine)?;
    assert_eq!(product.shape(), &[2, 2]);
    assert_close(product.host_data()?, &[3.0, 0.0, 0.0, 1.0]);
    assert_close(solution.as_slice()?, &[2.0, 2.0]);
    assert_close(singular_values.as_slice()?, &[3.0, 1.0]);

    Ok(())
}

Expected output: the program exits silently because the shape and value assertions pass. The same caller-held CpuEngine carries matmul, solve, and values-only SVD. The from_vec_col_major buffer is column-major: the leftmost axis varies fastest in memory.

Quickstart B: Traced AD

Add tenferro-ad when the same tensor stack needs graph-based derivatives:

[dependencies]
tenferro-runtime = "0.12"
tenferro-cpu = "0.12"
tenferro-ad = "0.12"
use tenferro_ad::TracedTensorAdExt;
use tenferro_cpu::CpuEngine;
use tenferro_runtime::{ExecutionEngine, GraphCompiler, Runtime, TraceContext, TracedTensor};

fn assert_close(actual: &[f64], expected: &[f64]) {
    assert_eq!(actual.len(), expected.len());
    for (index, (actual, expected)) in actual.iter().zip(expected).enumerate() {
        let error = (actual - expected).abs();
        assert!(
            error < 1.0e-12,
            "value {index}: actual={actual}, expected={expected}, error={error}"
        );
    }
}

fn run(tensor: &TracedTensor) -> Result<tenferro_runtime::Tensor, Box<dyn std::error::Error>> {
    let mut compiler = GraphCompiler::new();
    let program = compiler.compile(tensor)?;
    let mut engine = CpuEngine::serial();
    let mut builder = Runtime::builder();
    builder.register_engine(tenferro_cpu::runtime_engine_registration(&engine)?)?;
    let runtime = builder.build()?;
    let prepared = runtime.prepare(&program, &[], &[engine.target()])?;
    let mut outputs = runtime.run(&mut [&mut engine], &prepared, &[])?;
    assert_eq!(outputs.len(), 1);
    Ok(outputs.remove(0).into_tensor(&mut engine)?)
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let trace = TraceContext::new();
    let x = trace.from_vec_col_major(vec![3], vec![1.0_f64, 2.0, 3.0])?;
    let y = (&x * &x)?.reduce_sum(Some(&[0]))?;

    let y_value = run(&y)?;
    assert_eq!(y_value.shape(), &[] as &[usize]);
    assert_close(y_value.as_slice::<f64>().unwrap(), &[14.0]);

    let grad = y.grad(&x)?;
    let grad_value = run(&grad)?;
    assert_eq!(grad_value.shape(), &[3]);
    assert_close(grad_value.as_slice::<f64>().unwrap(), &[2.0, 4.0, 6.0]);

    let tangent = trace.from_vec_col_major(vec![3], vec![0.1_f64, 1.0, -2.0])?;
    let directional = y.jvp(&x, &tangent)?;
    let directional_value = run(&directional)?;
    assert_eq!(directional_value.shape(), &[] as &[usize]);
    assert_close(directional_value.as_slice::<f64>().unwrap(), &[-7.8]);

    Ok(())
}

Next Steps

After the first CPU program, read Core Concepts for the main mental model or Choosing a Tensor API to pick between TypedTensor, Tensor, EagerTensor, and TracedTensor. The sidebar links to the full guides for memory order, CUDA, tensor operations, autodiff, einsum, and linear algebra.