qxmx
A single-model (Bonsai-27B) inference engine for Intel Arc GPUs
Star
Fork
Watch

Commits on extra-debug
engine: abort with diagnostics on null USM allocations
committed
prefill: fix DEVICE_LOST on short prompts (< 16 tokens) main
committed
prefill: batch the tail in one M=16 call (fixes 1.58x serve slowdown)
committed
prefill_cached: fast path when no snapshot group fires + --driver-cpu pin
committed
coarsen prefix-cache snapshot stride + real device banner in CLI tools
committed
update AGENTS.md and README.md
committed
hoist loop-invariant address math in gemv + FFN GEMM kernels (decode +4.5%, prefill +6.7%)
committed
diff: relabel 'B70 eng' -> 'GPU eng' (Arc Battlemage, not B70-specific)
committed
decode gemv: tune split-K target to 512 WGs, add QXMX_GEMV_* knobs (ffn 16.9->16.0)
committed
decode gemv: raise split-K S cap to hide latency (ffn 19.7->16.9 ms)
committed
gguf: support Q2_0 g128 format alongside g64
committed
attn: v3 FD-in-WG decode scan (16.4 tok/s at 63k ctx, beats llama.cpp scan)
committed
attn: GQA-folded decode scan + micro-fixes (2.3x attn, 1.7x TG at 63k ctx)
committed
bench: add qxmx_bench (llama-bench-style pp/tg matrix)
committed
attn: native::exp in decode softmax (6.7% attn at 33k ctx)
committed
serve: add llama.cpp-compatible GET /props endpoint
committed
serve: emit chunked-body terminator on stream completion (llama-benchy fix)
committed
M4.1: multi-slot batched FFN spike (1.78x aggregate at N=16)
committed
attn: T13 deferred WHT in decode (12% attn, 3% decode)
committed
profile: QXMX_PROFILE=2 reports device-busy vs dispatch split
committed
serve: correct prefill tok/s in the request log (cache-aware)
committed
M4: FlashDecoding split-KV decode attention (7.4x attn, 2.8x decode)
committed
qxmx_serve: fix max_tokens
committed
Remove dead code and targets
committed
Add README.md and AGENTS.md
committed
MVP #7: Dockerfile for the post-reorg layout; server validated in-container
committed
docs: M3 complete -- ROADMAP checkmarks, AGENTS state, plan records
committed
M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle
committed
Add plan.md: M3 execution plan + M2.6 cache + M3.3 chat-template notes
committed
M2.6.2+2.6.3: engine restore path + inline SSM snapshotting
committed
M2.6.1: hash-chained KV block cache + SSM checkpoint store
committed
M3.1 follow-up: fix per-slot KV over-allocation, bump default ctx to 64K
committed
M3.1: vendor cpp-httplib + qxmx_serve skeleton
committed
M2.5: multi-slot token-interleaved scheduler (closes M2)
committed
M2.4: incremental UTF-8-safe detokenizer for streaming
committed
M2.3: stop conditions (EOS ids, stop strings, max_tokens, ctx exhaustion)
committed
M2.2: sampling (temperature/top-k/top-p/seed) on engine_i
committed
M2.1: slot-based engine API (open/close/reset/prefill/forward/sample)
committed
Add AGENTS.md (project guide), delete plan.md (M1 working spec, done)
committed
M1 closeout: refresh ROADMAP, trim plan.md to M1-done record (−708 lines)
committed
M1.7c: fused FlashAttention via sycl::joint_matrix -- 741 tok/s (+20%)
committed
FFN GEMM tile tune + naive DeltaNet default -- 613 tok/s (+22%)
committed
Update plan.md + ROADMAP.md: M1.7b done, 500 gate cleared, naive 598 tok/s
committed
ssm_timing_test: add naive (gpu_deltanet_batched) timing line
committed
M1.7b.13: naive DeltaNet barrier reduction -- 599 tok/s (+1.3% over M1.7b.12)
committed
Add runtime chunk size knob (QXMX_CHUNK) + chunk-size sweep
committed
M1.7b.12: naive register-recurrence DeltaNet -- 591 tok/s (+18% over WY/UT)
committed
Refactor: extract DeltaNet recurrence to per-impl files + meson toggle
committed
Dead code cleanup: remove 873 lines of evolutionary dead ends
committed
M1.7b.9+10: FFN gate+up fusion (gemm_gpu_v4_dual) + GEMM GPP=16 tile tune -- 500 tok/s
committed
M1.7b.8: WY/UT UT-solve opt -- precompute+fuse G into KK, 481 tok/s
committed
M1.7b.7: 3-kernel FA split (DPAS QK + softmax + DPAS PV) -- 439 tok/s
committed
M1.7b.6: plain-SYCL scalar FA kernel -- 230 tok/s (was 176 M1.7b.4)
committed
Reorganize source tree: src/ + tools/ + tests/
committed
Remove accidental build artifact + gitignore it
committed
M1.7b.4: ESIMD-vectorized FA kernel (no DPAS) -- 2.35x faster attn
committed
M1.7b.3: wire fused FlashAttention into attn_forward_b
committed
M1.7b.1: fused FlashAttention-2 kernel (correct, no DPAS)
committed
M1.7b.0: T13 rotated-domain V accumulator CPU harness (fa_t13_test)
committed
M1.7a: wire GEMM WY/UT into engine (ssm_forward_b), 2x faster DeltaNet
committed