The cubecl fork: what it carries, what stays upstream, and the own-backend contingency (#1945)
Decision record for the tensor4all/cubecl fork that tenferro-rs consumes, written after the held-CUDA session (#2059), the owning pending transfers (#2073, #2076) and the source publication (#2101) were delivered against it. tenferro depends on the fork’s published releases (t4a-cubecl* 0.11.0 / t4a-cubek* 0.3.0 from crates.io); the revisions in this table are what those releases carry.
1. What the pin is
| Item | Value |
|---|---|
| Fork | tensor4all/cubecl, release branch release/t4a-cubecl-0.11.0 at 8aeafdf50cb3db65cc727e4ab303ed5b504c5ab9 |
| Upstream | tracel-ai/cubecl, whose main (4362f783f) is 388 commits ahead of the fork’s merge base |
| Fork-only commits | 62 |
| Version line | releases v0.10.1, v0.10.2 and v0.11.0 of the fork’s packages |
| Downstream pin | cubek fork release branch release/t4a-cubek-0.3.0 (a1a106096a09fd27e1fbe9dbd2253d244f2df677), which carries one cubecl revision |
The fork is therefore not a thin overlay of the #1945 work. It is the release fork that this workspace’s CUDA stack builds on, and it already carried, before #1945: CUDA OOM reclaim, complex-plane warp reduction, exact wgpu submission completion, device-service wakeups, host-visible primary allocation, removal of redundant upload copies in ComputeClient::create*, a read-only loaded-kernel-id diagnostic, and the release/CI plumbing that keeps those packages publishable.
2. What #1945 added to it, and why each is needed here
| Change | Fork PR | Why tenferro-rs needs it |
|---|---|---|
ComputeServer::check_errors / Client::check_errors: report producer errors and dispatch the host queue without rotating the drop queue |
#27 | The held session’s publication boundary had to surface backend errors without a flush, which would wait for the previous fence. Retired by the 0.11 migration: upstream reports a failed launch on the buffers the launch never wrote, through the published buffer’s own get_resource claim and Client::check/sync_buffers, so the four publication sites rely on that claim instead (docs/design/cubecl-011-check-errors-disposition.md). |
| CUDA write path evaluates the staging policy | #28 | Command::write_to_gpu queued staged host bytes but only the kernel path checked drop_queue.should_flush(). With a publication boundary that no longer flushes, a write-only workload never reached the default 64-allocation / 64 MiB thresholds and retained staged bytes indefinitely. |
ComputeServer::write is unsafe fn with a write-authority contract |
#29 | A safe server write accepted a lifetime-free binding, so a borrowed value could be written through a binding derived from the borrow. Session::tensor_binding and Session::array_arg are unsafe on our side (#2093); this is the substrate half of the same contract. |
3. The residue, and why the fork will not carry it
The remaining hole is cubecl’s checked kernel launch: the launcher generated by #[cube(launch)] is a safe function whose arguments include writable tensor bindings, and launch_unchecked — which is unsafe — is required only for arguments the launcher cannot check, not for write authority. A user holding a lifetime-free TensorBinding can therefore launch a kernel that writes storage an outstanding borrow still reads, from safe code, inside cubecl itself.
tenferro-rs does not carry a fix for this, for three reasons:
- We cannot reach it. Our layer never hands out a cubecl handle, binding or client, and the accessors that derive a binding from a borrow are
unsafewith the write-authority contract (#2093). Making the launch safe-again-unsafe changes nothing about what our own code can do. - It is cubecl’s API decision, not a defect with one obvious fix. Marking every writable launch
unsafe, or introducing a separate write-authority binding type, changes the ergonomics of every cubecl kernel in every downstream project. That decision belongs to the upstream maintainers, who can weigh it against their whole surface; a fork-only version would diverge from them for no local gain. - The fork’s cost is real. It already diverges 62 commits from upstream. Carrying an unbounded API redesign on top of that increases the rebase and review surface of every future pin bump, which is the opposite of what the fork exists for.
What we owe upstream instead: a focused proposal that names the hole, the two candidate shapes, and the fact that crates/tenferro-linalg and the samples would have to annotate their launch sites (18 sites in gpu/linalg.rs plus the QR variants and the extension samples) if launches became unsafe. That is a proposal to tracel-ai/cubecl, not a fork commit.
4. When the fork goes away
The fork is retired when either holds:
- Upstream absorbs the changes we need. The soundness-relevant set is #27, #28 and #29; the rest are release-fork conveniences that upstream either has or does not want. Upstream
mainis 388 commits ahead, so any proposal is a rebase onto current upstream, not a cherry-pick of our history. - We migrate to an owned backend (§5), at which point the pin disappears with the backend.
Until then the pin stays, and every bump re-runs the CUDA test partition and the held-session / pending-transfer / publication tests against the new revision.
A rebase has to keep the upload de-duplication. Newer upstream create_from_slice / do_create_from_slices own their input with slice.to_vec() and then copy that owned vector again inside do_create_from_slices, so a pageable upload pays two extra payload-sized copies before the staging copy. The pinned revision does not contain that code: do_create hands the already-owned Bytes straight to the server. When the fork next rebases onto a newer upstream, the de-duplication has to be kept (or the upstream regression reported), and the staged-upload row of the transfer measurement is where the loss would show.
A bump also has to keep the declared version honest. scripts/check-git-pin-content.py fails closed when a git-pinned dependency’s version does not identify the content the pinned revision actually has: the crates.io release of that version must match the pinned revision file by file. The workspace once pinned t4a-cubecl-* at the fork revision that contains #27/#28/#29 while still declaring =0.10.2, and the published 0.10.2 came from eb2c8894d (before those fixes); that mismatch failed main’s CI configuration checks job. Resolved: the 0.11 fork releases publish the ported content, the workspace now requires t4a-cubecl* = =0.11.0 / t4a-cubek* = =0.3.0 from the registry, and no workspace git pin remains. The rule for any future pin is unchanged: advance the declared version together with the revision, or release the fork, and the exception file stays reserved for dependencies with no crates.io release.
5. Own-backend contingency inventory
If the fork ever becomes untenable — upstream rejects the soundness changes, or the divergence cost exceeds maintaining a backend — the replacement surface is bounded and identifiable:
| Surface | Where it lives now | Replacement cost |
|---|---|---|
| Kernel JIT, fusion codegen, launch plumbing | cubecl-core / cubecl-macros / cubecl-runtime, used across tenferro-gpu (60 files) and tenferro-linalg’s GPU kernels (84 files workspace-wide mention cubecl) |
Largest item: a codegen path plus a launch API, including the checked-launch hole above |
| Device memory pool, staging and its retirement bound | cubecl-runtime / cubecl-cuda (reserve_cpu, drop queue, flushing policy) |
Moderate: our own pool already owns retirement (TransferRetirementQueue, workspace retirement); the allocation pool and staging bound would move in |
| Streams, events, stream slotting, event domains | cubecl-runtime / cubecl-cuda (StreamId, drop_queue, event domain ids) |
Moderate: our runtime already identifies streams, events and domains by captured logical ids |
| Graph/execution-engine contract, compiled-kernel cache | cubecl-runtime (KernelId, memory management, scheduler) |
Bounded: our runtime and plan cache already own the tenferro-side contracts |
| cudarc / driver bindings | cudarc directly in tenferro-gpu (FFI, cuBLAS, cuSOLVER, events, pinned host memory) |
Already ours; unaffected by the fork |
| Extension/sample API | our unsafe bindings |
Small: it is our surface already |
The migration is a backend swap behind our own types, which is why every contract added for #1945 — held sessions, pending transfers, publications — deliberately mentions no cubecl type in its public API. No public tenferro-rs API breaks if the pin is replaced.
6. What this record does not claim
- It does not claim the fork is unsound for our use, or that upstream requires the soundness changes: it records that our layer cannot reach the residue and that the fix is upstream’s call.
- It does not claim the fork’s 62 commits are all upstreamable or all ours; the inventory above lists what the fork provides, and only #27/#28/#29 are proposal candidates.
- It does not schedule the own-backend migration. The inventory exists so the contingency is priced before it is needed, not to start it.