Struct PoolBuilder 

pub struct PoolBuilder { /* private fields */ }

Configuration of a Pool.

Examples

use threadteam::Pool;

let pool = Pool::builder()
    .threads(2)
    .stack_size(8 << 20)
    .on_start(|_worker| { /* e.g. pin the thread */ })
    .build()?;
assert!(pool.width().get() >= 1);

Implementations

impl PoolBuilder

pub fn threads(self, n: usize) -> PoolBuilder

Number of worker threads. Defaults to the available parallelism.

Zero is rejected by build; the pool never falls back to an environment-dependent default.

Examples

use threadteam::{Error, Pool};

assert_eq!(Pool::builder().threads(0).build().unwrap_err(), Error::ZeroThreads);

pub fn thread_name(self, f: impl Fn(usize) -> String + 'static) -> PoolBuilder

Names the workers; the closure receives the worker index.

Examples

let pool = threadteam::Pool::builder()
    .threads(1)
    .thread_name(|i| format!("my-pool-{i}"))
    .build()?;

pub fn stack_size(self, bytes: usize) -> PoolBuilder

Worker stack size in bytes. Defaults to 16 MiB, sized for recursive LU.

Examples

let pool = threadteam::Pool::builder().threads(1).stack_size(4 << 20).build()?;

pub fn on_start(self, f: impl Fn(usize) + Send + Sync + 'static) -> PoolBuilder

Hook run on each worker when it starts, with the worker index.

Typical use is pinning the worker to a core. The hook is not called without the parallel feature, which starts no workers.

Examples

let pool = threadteam::Pool::builder().threads(1).on_start(|_| {}).build()?;

pub fn idle_spin(self, spin: Duration) -> PoolBuilder

Default spin time of an unreserved idle team member before it returns its thread to Rayon. Defaults to 200 ms; zero returns it immediately. The member also returns once it has polled for the pool’s spin budget (spin_polls), whichever comes first.

Operations may override it with WaitOverride. A reserved member never leaves on a timeout.

Examples

use std::time::Duration;

let pool = threadteam::Pool::builder().threads(2).idle_spin(Duration::ZERO).build()?;
assert_eq!(pool.wait_policy().idle_spin(), Duration::ZERO);

pub fn idle_runs_rayon(self, on: bool) -> PoolBuilder

Default for whether unreserved idle members run Rayon jobs while they spin. Defaults to false: native team execution keeps them for the next reservation. true is the compatibility mode in which an idle member serves Rayon work (an acquisition then refuses a member that is inside such a job instead of waiting for it).

Examples

let pool = threadteam::Pool::builder().threads(2).idle_runs_rayon(true).build()?;
assert!(pool.wait_policy().idle_runs_rayon());

pub fn acquire_timeout(self, timeout: Duration) -> PoolBuilder

How long Exec::team waits for idle workers to wake before it refuses. Defaults to 2 ms.

This bounds the wait for workers that are free; the acquisition does not wait for a worker that is running a job, it only runs out this time and refuses. Raise it on a loaded host to reduce spurious refusals.

Examples

use std::time::Duration;
let pool = threadteam::Pool::builder()
    .threads(2)
    .acquire_timeout(Duration::from_millis(10))
    .build()?;

pub fn qos(self, qos: Qos) -> PoolBuilder

Scheduling class of the workers (see [Qos]).

Defaults to [Qos::UserInitiated] on macOS and to nothing elsewhere. The calling thread’s class is not changed by this crate; an engine that wants its leading thread raised too calls raise_current_thread_qos once on it.

Examples

use threadteam::{Pool, Qos};
let pool = Pool::builder().threads(1).qos(Qos::Default).build()?;

pub fn dispatch_costs(self, costs: DispatchCosts) -> PoolBuilder

Overrides the [DispatchCosts] of this pool’s tokens.

Defaults to [DispatchCosts::host_default]. The costs feed Exec::dispatch_cost and everything built on it (pays, width_for); they never change how work is placed.

Examples

use std::num::NonZeroUsize;
use std::time::Duration;
use threadteam::{DispatchCosts, DispatchKind, Exec, HostClass, Pool, WidthCosts};

let costs = DispatchCosts::for_class(HostClass::Generic)
    .with(DispatchKind::Outside, WidthCosts::flat(Duration::from_micros(7)));
let pool = Pool::builder().threads(2).dispatch_costs(costs).build()?;
let exec = Exec::new(&pool, 2)?;
// Without the `parallel` feature the pool has width 1: no section, no cost.
let expected = if pool.width().get() >= 2 { Duration::from_micros(7) } else { Duration::MAX };
assert_eq!(exec.dispatch_cost(NonZeroUsize::new(2).unwrap()), expected);

pub fn pin_participants(self, on: bool) -> PoolBuilder

Places the participants of a root team scope on the calling thread’s CPU set. On by default (see the reasons below); false turns it off for this pool. Linux only, where it does something.

With it on, the leader of a root team scope (Exec::team, or the root token of a Team) reads its own affinity mask S with sched_getaffinity, then:

• the leader pins itself onto S[0] for the life of the scope: one sched_setaffinity at scope entry and one at scope exit (which also runs on unwind) giving it S back, and nothing in between: no check and no move before a section;
• every member pins itself with sched_setaffinity onto one CPU of S: member k (participant k, counting the leader as 0) onto the k-th set CPU of S. The members take S minus S[0].

Nothing wraps. When S has fewer CPUs than the team has participants, nobody is placed: the scope runs with the scheduler’s placement, members that an earlier scope of a persistent team pinned restore their own masks, and Pool::pin_skips counts the scope. Participants stacked on one CPU measured slower than the scheduler’s placement and slower than one thread (evo2, LU f64 512 with 8 participants wrapped onto one CPU: 3.1 ms against 0.89 ms unpinned and 1.7 ms on one thread), so a team that cannot be placed is not placed badly.

The leader is pinned, and widened for vendor calls. A vendor library (OpenMP, OpenBLAS, a LAPACK provider) creates its thread team from the thread that calls it, and a team created from a one-CPU thread silently confines the whole provider team to that CPU — measured at 3-20x slower dgemm and LU at 4 threads. So a leader that calls such a library does it through TeamExec::external, which sets its mask to S for that call and back to {S[0]} when it returns (two sched_setaffinity calls per call, paid every time, inside the timed region; a refused syscall leaves the thread as it was). A member that makes such a call uses Exec::external, which restores the member’s own mask. A scope with no vendor call pays nothing.

A member saves its previous mask in its own slot and restores it itself before it stops being reserved: before it returns to Rayon, and before its thread serves work that is not its team’s — while it is lent by Exec::external, and when it runs such work itself (a member that leads a fork child and calls external). No thread ever changes another thread’s affinity.

What it is for

It keeps the participants of a team apart. Measured on broccoli (16 threads on 16 allowed CPUs, 20 processes, tensor4all/faer#107): with the scheduler free to place the threads, 11 of 20 processes ran a qr_factor 512 at the one-thread time, with 92 % of the samples showing two runnable team threads on one CPU; with the participants pinned, 0 of 20 did, and no sample was stacked. Giving the team one spare CPU did not help (7 of 20 slow). On evo2 (two L3 domains, LU f64 at 11 and 14 threads) unpinned teams stalled by 3 ms scheduler slices; pinning only the members and leaving the leader wherever it was put the leader on a member’s CPU in 74-94 % of the samples at 8 threads. Placing the leader on S[0] only when the scope opened, and leaving its mask whole, was not enough: in a long scope with serial leader work between sections (LU f64 4096) the leader drifted back onto a member’s CPU in 90 % of the samples and the call ran 38x slower. Pinning it to S[0] for the scope is the same as pinning every participant, which measured equal to placing it before every section; the simpler rule was chosen.

Assumptions and limits

• Linux only. The CPU set is a stack cpu_set_t and the placement is sched_setaffinity(0, ...). On every other target the option is accepted and does nothing.
• The caller’s CPU set is the source of placement, so the caller’s mask should belong to this team alone. Two teams whose leaders see the same mask place their members on the same CPUs; give each team (in practice: each process) its own CPU set (taskset -c).
• Best effort. A refused syscall (a mask larger than cpu_set_t, a cpuset that changed under us, a CPU offlined) leaves that thread where it was; an empty mask pins nobody; a failed pin or restore never fails the scope.
• A pinned member must not create a foreign thread team. A member that calls OpenMP, OpenBLAS or any library that builds its team from the calling thread directly inside a share confines that library’s team to one CPU. Call it through Exec::external, which restores the member’s mask first, or turn this option off.
• The members’ placement is the team’s and outlives a scope; the leader’s pin is the scope’s. A token of a Team that never opens a team scope (exec.for_ranges(...) directly) is not re-pinned: its members keep the placement of the last whole-team root scope (or of Pool::into_team), and its leader is not placed at all, so between and during such sections it may run on a member’s CPU. Open a team scope to get the leader on S[0]. The placement is changed only by a root scope of a token with the whole team’s budget: it lasts until such a scope re-places the members, or cannot place them and unpins them, or the Team ends. A root scope of a narrower token runs under the team’s current placement, changes nothing and pins no leader. One exception: members lent by Exec::external restore their own masks and are not re-pinned until the next whole-team root scope.

Why it is on by default

An unpinned team is the configuration that stalls, so a default of off gave it to every caller who did not know about this switch. The reasons it used to be off: it does not fight the caller’s taskset, because it never leaves the caller’s own mask; it does not confine a vendor team, because the leader widens itself with TeamExec::external and a member reaches vendor code through Exec::external, which restores the member’s mask first; it does take CPUs of S from other users of the process while a scope runs, bounded by S and by the no-wrap rule, so a pool that shares its CPUs with other work on purpose passes false. libgomp, libomp, OpenBLAS and faer leave threads unpinned unless asked; this crate differs on purpose.

Examples

let pool = threadteam::Pool::builder().threads(2).build()?;
assert!(pool.pin_participants()); // the default
let pool = threadteam::Pool::builder().threads(2).pin_participants(false).build()?;
assert!(!pool.pin_participants());

pub fn spin_polls(self, polls: u32) -> PoolBuilder

Spin budget of a team wait, in polls of one PAUSE (YIELD on aarch64): how long a reserved member waiting for work, a leader in a section’s join, or a participant in a barrier spins before it yields a few times and then sleeps until woken (docs/design/waiting.md; the shape of libgomp’s GOMP_SPINCOUNT). A count, so a waiter that is descheduled does not spend it while it waits for its CPU. Zero sleeps after the yields. An oversubscribed scope uses at most 100 polls. It also bounds how long an idle member released by a team keeps polling before it returns to Rayon, together with idle_spin.

Examples

let pool = threadteam::Pool::builder().threads(2).spin_polls(4096).build()?;
assert_eq!(pool.spin_polls(), 4096);

pub fn build(self) -> Result<Pool, Error>

Starts the workers.

Without the parallel feature no thread is started and the pool has width 1.

Errors

[Error::ZeroThreads] when threads(0) was requested, and [Error::PoolBuild] when the workers cannot be started.

Examples

let pool = threadteam::Pool::builder().threads(2).build()?;
assert!(pool.width().get() <= 2);

Trait Implementations

impl Debug for PoolBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.