Host-Driven FFI #

future_form includes the future_form_ffi crate that lets FFI consumers drive Rust async state machines without an async runtime.

The Pattern #

In the host-driven (or sans-IO) pattern, the foreign host (Go, Java, Python, C, Swift) controls the execution loop. Instead of handing a future to a Rust runtime that polls it, the host calls poll_once() repeatedly:

+-------------+  poll_once()   +--------------+
|  FFI Host   | ------------->|  HostHandle  |
|  (Go, Java, |               |  or Effect-  |
|   Python…)  | <-------------|  Handle      |
+-------------+  Poll<T>      +--------------+
        |
        | If Pending: fulfill effect,
        |             then poll again
        |
        v
  Poll::Ready(value)

This is the async equivalent of an effect handler: the host interprets "pending" as "I need something from you," fulfills that need, and re-polls.

The future_form_ffi Crate #

All FFI building blocks live in the future_form_ffi crate (#![no_std], core + alloc):

Type	Module	Purpose
PollOnce	poll_once	Extension trait: poll a boxed future once with a no-op waker
HostHandle<F>	host_handle	Thin-pointer wrapper for passing boxed futures through C ABI
AtomicSlot<T>	atomic_slot	Lock-free, thread-safe slot for a single value
EffectSlot<E, R>	effect_slot	Shared-state channel for the effect protocol (built on AtomicSlot)
EffectHandle<F, E, Ctx>	effect_handle	Future handle + stashed effect + context pointer

PollOnce Trait #

Rather than introducing separate host-driven FutureForm variants, PollOnce is an extension trait implemented for all boxed futures:

pub trait PollOnce {
    type Output;
    fn poll_once(&mut self) -> Poll<Self::Output>;
}

impl<T> PollOnce for BoxFuture<'_, T> { ... }
impl<T> PollOnce for LocalBoxFuture<'_, T> { ... }

This means any Sendable or Local future can be polled by a foreign host — no special variant needed. The library stays at two marker types, and host-driven capability is orthogonal.

HostHandle and EffectHandle have inherent poll_once() methods so bridge code doesn't need to import the PollOnce trait. PollOnce remains as an extension trait for foreign types (BoxFuture, LocalBoxFuture) where inherent methods aren't possible.

HostHandle<F> #

A thin wrapper that adds an outer Box around a boxed future, producing a single thin pointer suitable for passing through C ABI:

BoxFuture<T> = Pin<Box<dyn Future<Output = T>>>  ← fat pointer (data + vtable)
HostHandle<F> = Box<F>                            ← thin pointer

HostHandle<F> is generic over the future type. F is typically BoxFuture<'static, T> or LocalBoxFuture<'static, T>, so Send vs !Send is carried by F itself — no separate LocalHostHandle needed. Bridge code uses type aliases for readability:

type CounterHandle = HostHandle<BoxFuture<'static, u64>>;

AtomicSlot<T> #

A lock-free, thread-safe slot that holds at most one value. This is the low-level building block that encapsulates all AtomicPtr unsafety behind a safe move-only API:

• put(&self, val: T) — stores a value, dropping any previous occupant
• take(&self) -> Option<T> — takes the value, leaving the slot empty

There is intentionally no peek(&self) -> &T — the value could be taken (and freed) on another thread while a reference exists. Only move-only operations are safe.

AtomicSlot is the only type in future_form_ffi that contains unsafe code. All other types (EffectSlot, EffectHandle, etc.) are built on safe abstractions.

EffectSlot<E, R> #

Built on two AtomicSlots — one for the pending effect, one for the response. It mediates the effect protocol between async code and the FFI host:

Async code                    Host (via FFI bridge)
    │                              │
    │  request(&effect)            │
    │──────────────────────────>   │
    │  Poll::Pending               │
    │                              │
    │              take_effect()   │
    │   <──────────────────────────│
    │              Some(effect)    │
    │                              │
    │              fulfill(resp)   │
    │   <──────────────────────────│
    │                              │
    │  request(&effect)            │
    │──────────────────────────>   │
    │  Poll::Ready(response)       │
    │                              │

EffectSlot contains zero unsafe code — all thread safety is delegated to AtomicSlot.

EffectHandle<F, E, Ctx> #

Bundles a HostHandle<F> with a stashed effect cache and a context pointer:

• poll_once(&mut self) — polls the underlying future
• stash_effect(&mut self, effect) — saves an effect snapshot for host inspection
• take_effect(&mut self) / last_effect(&self) — host reads the stashed effect
• context() — raw *const Ctx for application-specific state

The only unsafe in EffectHandle is unsafe impl Send for the *const Ctx pointer (raw pointers are !Send by default). The stashed effect is a plain Option<E> — no UnsafeCell, no interior mutability.

Why a Trait (Not Newtypes) #

An earlier design used separate FutureForm variants (HostSendable, HostLocal) backed by newtype wrappers around BoxFuture/LocalBoxFuture. This was removed in favor of PollOnce because:

• No variant proliferation. Four variants (Sendable, Local, HostSendable, HostLocal) collapsed to two (Sendable, Local). The macro, tests, and docs all simplified.
• Orthogonal concerns. "How to box the future" (Sendable vs Local) and "how to poll it" (runtime vs host) are independent axes. Coupling them in the type system was unnecessary.
• No orphan rule issues. PollOnce is a local trait, so implementing it for BoxFuture/LocalBoxFuture (from the futures crate) is allowed.
• Universal availability. poll_once() is useful beyond FFI — tests, debugging, any context where you want to manually step a future.

The Noop Waker #

poll_once() uses Waker::noop() — a &'static Waker that does nothing on wake(), stable since Rust 1.85.

This is correct for host-driven polling because:

1. The host decides when to re-poll, not a runtime scheduler.
2. There is no runtime to notify. The waker's job (telling the executor "this task is ready") is irrelevant — the host is the executor.
3. Futures that use poll_fn and check shared state directly work correctly with noop wakers.

FFI Bridge Pattern #

A typical FFI bridge exposes C-ABI functions that:

1. Create a HostHandle wrapping a BoxFuture via the Sendable variant
2. Convert to a thin raw pointer via Box::into_raw(Box::new(handle))
3. Call poll_once() through the opaque pointer
4. Return a status code (0 = pending, 1 = ready) and the result

// C API (declared in Go via CGo, Python via ctypes, etc.)
typedef void* CounterHandle;
typedef void* FutureHandle;

typedef struct {
    uint8_t  ready;
    uint64_t value;
} FfiPollResult;

CounterHandle counter_new(uint64_t start);
void          counter_free(CounterHandle counter);
FutureHandle  counter_next_value(CounterHandle counter);
FfiPollResult future_poll(FutureHandle future);
void          future_free(FutureHandle future);

The Rust bridge code follows a three-phase pattern:

// 1. Create — ownership moves to the host
let handle = HostHandle::new(fut);
let ptr = Box::into_raw(Box::new(handle));  // thin *mut for C ABI

// 2. Poll — borrow from the raw pointer
let handle = unsafe { &mut *ptr };
let result = handle.poll_once();

// 3. Free — reclaim and drop
unsafe { drop(Box::from_raw(ptr)) };

The host program calls counter_next_value to create the future, then loops on future_poll until it returns ready, then calls future_free to free the Rust allocation.

Composability #

Because BoxFuture and LocalBoxFuture implement both Future and PollOnce, the same trait implementation works for both runtime-driven and host-driven execution:

// Same Sendable impl works for both:

// Runtime-driven (tokio):
let result = <Counter as Counter<Sendable>>::next(&c).await;

// Host-driven (FFI):
let mut fut = <Counter as Counter<Sendable>>::next(&c);
assert_eq!(fut.poll_once(), Poll::Ready(42));

Library authors write one trait impl and get both FFI and native Rust support from the same code.

Effect Handling #

The basic counter example's futures resolve immediately — the host's poll loop never actually sees Pending. A richer example demonstrates the full sans-IO pattern, where futures yield effects that the host must fulfill.

See counter_effects/counter_effects/ for an async state machine that requests timestamps and logging from the host, and counter_effects/go_host/ (and corresponding Java/Python hosts) for the effect-dispatching poll loop.

For a detailed comparison of PollOnce with hand-coded pure sans-IO state machines, see Sans-IO Comparison.

Effect Slot Ownership #

An EffectSlot is a single request-response channel — one pending effect, one response. If two futures share the same slot, they can deliver responses to the wrong future:

Per-struct (one slot, current example):

  future_a ──┐
             ├──> EffectSlot  ← only one conversation at a time
  future_b ──┘

  1. future_a writes GetTimestamp to the slot
  2. future_b overwrites with Log(...), dropping GetTimestamp
  3. host reads Log, fulfills with LogAck
  4. future_a is re-polled, reads LogAck — but expected a Timestamp

There is no memory unsafety — AtomicSlot swaps are atomic and well-defined. The issue is purely semantic: futures receive responses meant for other futures.

This means the slot ownership strategy determines how many futures can be in flight concurrently from the same struct. Four approaches were considered:

Per-struct slot #

The counter_effects example uses this pattern. The EffectSlot lives as a field on the struct:

pub struct EffectCounter {
    current: u64,
    effects: EffectSlot<Effect, Response>,
}

All futures created from this counter share the one slot. This is simple and works well for sequential use — create a future, poll it to completion, then create the next one. The host already does this naturally in a single-threaded poll loop.

Limitation: Only one in-flight future per struct instance.

This is what the examples use because the goal is demonstrating the effect protocol (request → inspect → fulfill → re-poll), not concurrent effect handling.

Per-future Arc<EffectSlot> (recommended for concurrent use) #

Each future gets its own EffectSlot, shared via Arc between the async closure (which calls request()) and the EffectHandle (through which the host calls take_effect() / fulfill()):

Per-future (Arc per slot):

        ┌───────────┼───────────┐
        │           │           │
        ▼           ▼           ▼
   ┌─────────┐ ┌─────────┐ ┌─────────┐
   │future_a │ │future_b │ │future_c │
   └────┬────┘ └────┬────┘ └────┬────┘
        │           │           │
        │ Arc       │ Arc       │ Arc
        │ clone     │ clone     │ clone
        ▼           ▼           ▼
   ┌─────────┐ ┌─────────┐ ┌─────────┐
   │ slot_a  │ │ slot_b  │ │ slot_c  │    independent channels
   └────┬────┘ └────┬────┘ └────┬────┘
        │           │           │
        │ Arc       │ Arc       │ Arc
        │ clone     │ clone     │ clone
        ▼           ▼           ▼
   ┌─────────┐ ┌─────────┐ ┌─────────┐
   │handle_a │ │handle_b │ │handle_c │    EffectHandle per future
   └────┬────┘ └────┬────┘ └────┬────┘
        │           │           │
        └───────────┼───────────┘
                    │
                    │ take_effect() / fulfill()
                    ▼
            ┌───────────────┐
            │     Host      │
            │               │
            │ manages a set │
            │ of handles    │
            └───────────────┘

  Each future ↔ slot ↔ handle is an independent conversation.
  No cross-talk. No capacity limit.

The bridge creates the Arc<EffectSlot> when constructing each future, clones it into both the async block and the EffectHandle. The host interacts with effects through the handle — it never needs to know about slot identity or indices:

let slot = Arc::new(EffectSlot::new());
let slot_for_future = Arc::clone(&slot);

let fut = Box::pin(async move {
    let ts = poll_fn(|_cx| slot_for_future.request(&Effect::GetTimestamp)).await;
    // ...
});

let handle = EffectHandle::new(HostHandle::new(fut), context);
// Bridge stores `slot` alongside the handle for effect inspection/fulfillment.

The host already manages one opaque handle per future — the slot follows the same per-future ownership. No map, no indices, no capacity limit. The host can create as many concurrent futures as memory allows, each with its own independent effect channel.

The allocation cost is one Arc per future (two pointer-sized atomics + the slot's two AtomicPtrs). This is negligible alongside the Box::pin that already heap-allocates the future itself.

Slot registry (rejected) #

A BTreeMap<SlotId, EffectSlot> on the struct, where each future claims a slot by ID:

pub struct EffectCounter {
    current: u64,
    slots: BTreeMap<SlotId, EffectSlot<Effect, Response>>,
}

This was rejected for several reasons:

• API pollution. Every FFI function that touches effects needs a SlotId parameter. The host must track which ID belongs to which future — bookkeeping the host shouldn't need to care about.
• Requires &mut self for insertion. You can't create two futures from the same counter without releasing the mutable borrow in between.
• Manual cleanup. The host must call a cleanup function to remove slots when futures complete. If it forgets, slots leak.
• Reintroduces shared mutable state. The map itself needs synchronization for concurrent access, adding the very complexity AtomicSlot was designed to avoid.
• Duplicates existing structure. The host already has one handle per future via EffectHandle. A slot registry on the Rust side duplicates that per-future mapping.

Fixed-size array (rejected for general FFI) #

An inline [EffectSlot<E, R>; N] with an atomic bitmap for claiming slots:

pub struct EffectCounter<const N: usize> {
    current: u64,
    slots: [EffectSlot<Effect, Response>; N],
    // + AtomicU64 bitmap for slot allocation
}

This is viable for no_alloc / embedded contexts where Arc isn't available and the concurrency bound is known upfront. However, it was rejected as the general recommendation because:

• Arbitrary capacity. The right N depends on the application — too small and you hit the limit, too large and you waste memory (though at 16 bytes per slot, even N=64 is only 1KB).
• Exhaustion policy. Must decide what happens when all N slots are taken — return an error? Panic? Block?
• Index management. Claiming and releasing slots requires an atomic bitmap or counter, adding complexity.
• Arc cost is negligible. In FFI contexts where futures are already heap-allocated via Box::pin, one extra Arc per future is not a meaningful cost.

A SlotPool<N> type could be offered alongside Arc<EffectSlot> for constrained environments, but the general-purpose recommendation is per-future Arc.

Comparison #

	Per-struct	Per-future Arc	Registry map	Fixed array
Concurrent futures	No	Yes	Yes	Up to N
Extra allocation	None	One Arc	Map + slot	None
API surface	Clean	Clean	SlotId everywhere	Index management
Cleanup	Automatic	Automatic	Manual	Automatic
no_alloc compatible	Yes	No	No	Yes
Complexity	Low	Low	High	Medium

Guidance #

The counter_effects example uses per-struct for simplicity — it demonstrates the effect protocol, not concurrency. The key_value_store example uses per-future Arc<EffectSlot> — each kv_get, kv_put, etc. creates a fresh slot, enabling concurrent operations with independent effect channels. The host's interaction doesn't change — it still calls take_effect() and fulfill() through the EffectHandle, which routes to the correct slot automatically.

Handle Storage #

Rust does not assign or manage future IDs. The EffectHandle is the identity — an opaque token bundling the future, its slot, and its context. The host stores these however its concurrency model demands.

Host-side collection #

The host keeps a map of handles, keyed however it likes:

  Host (Go / Java / Python)
  ┌─────────────────────────────────────────────┐
  │                                             │
  │  handles: Map<FutureId, *mut EffectHandle>  │
  │                                             │
  │  handle_1 ──► EffectHandle ──► Arc<slot_1>  │
  │  handle_2 ──► EffectHandle ──► Arc<slot_2>  │
  │  handle_3 ──► EffectHandle ──► Arc<slot_3>  │
  │                                             │
  └─────────────────────────────────────────────┘

The FutureId is whatever the host wants — an integer counter, a UUID, a task name. It is entirely host-side bookkeeping. Rust does not know or care about it.

Inline in host tasks #

Each host-side task/goroutine/thread holds its handle directly — no map needed:

  Go goroutine A                Go goroutine B
  ┌──────────────────┐          ┌──────────────────┐
  │ handle *C.Handle │          │ handle *C.Handle │
  └────────┬─────────┘          └────────┬─────────┘
           │                             │
           ▼                             ▼
     EffectHandle                  EffectHandle
     └─► Arc<slot_a>              └─► Arc<slot_b>

No future ID at all — the handle is the identity.

Event loop with a vec #

  Host event loop
  ┌──────────────────────────────┐
  │  active: Vec<EffectHandle>   │
  │                              │
  │  for handle in &mut active { │
  │      match handle.poll() {   │
  │          Pending => check fx │
  │          Ready   => remove   │
  │      }                       │
  │  }                           │
  └──────────────────────────────┘

The index in the vec is the implicit ID, or the host uses a slab allocator. When a future completes, the host removes it — the handle drops, its Arc<EffectSlot> drops, and the slot is freed.

All three patterns work because EffectHandle is self-contained: the host never needs to coordinate with Rust about which handle maps to which future.

Safety Budget #

The future_form_ffi crate concentrates unsafe into the smallest possible surface:

Type	unsafe blocks	unsafe impl	Notes
AtomicSlot<T>	4	2 (Send, Sync)	All AtomicPtr unsafety lives here
EffectSlot<E, R>	0	0	Fully safe — delegates to AtomicSlot
HostHandle<F>	0	0	Pin::new requires Unpin, which is satisfied — no unsafe needed
EffectHandle<F, E, Ctx>	0	1 (Send)	Only for *const Ctx pointer
PollOnce impls	0	0	Pin::new requires Unpin, satisfied by BoxFuture/LocalBoxFuture

AtomicSlot has a self-contained safety proof (four invariants documented in the module). All other types are safe code built on top.