Qwen3.8-27B-DSpine-b16

DSpine for Qwen3.8-27B

arXiv Project Page GitHub

DSpine generates draft tokens in parallel. Each layer passes predicted predecessor features to adjacent positions, carrying causal dependencies through network depth.

This checkpoint is a five-layer draft model for Qwen3.8-27B, with block size 16 and an 8,192-token sliding window.

DSpine architecture

Evaluation

• Runtime: SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8.
• Draft blocks: DSpine 16; official DFlash2 8.
• DSpine inference: dynamic verification referencing DSpark, with ReplaySSM enabled.
• General benchmarks: xhigh reasoning, greedy and temperature sampling, at most 4,096 new tokens.
• Agent benchmarks: We evaluate WorkBuddy Bench (Code/Web/Office) using CodeBuddy sandbox trajectories: low/xhigh reasoning, temperature sampling, 20 conversations per subset, all tool-call prefixes replayed in order, prefix caching, and up to 32,768 new tokens per request.
• Temperature sampling: temperature=1.0, top_p=0.95, top_k=20.

Results

General Benchmark Acceptance Length

Agent Acceptance Length

Benchmarklowxhigh

DFlash2DSpineDFlash2DSpine

WorkBuddy Code4.195.084.064.92

WorkBuddy Web3.954.913.955.01

WorkBuddy Office4.245.634.205.49

SWE-bench Pro4.194.994.004.77

τ²-bench4.705.534.294.95

Agent End-to-End Throughput

Throughput is total output tokens divided by end-to-end concurrent replay wall time (tokens/s). Parentheses show speedup over autoregressive decoding (AR).

Benchmarklowxhigh

ARDFlash2 DSpineARDFlash2DSpine

WorkBuddy Code253.35590.27 (2.33×)686.23 (2.71×)193.00550.40 (2.85×)613.55 (3.18×)

WorkBuddy Web247.40639.74 (2.59×)773.19 (3.13×)252.26734.04 (2.91×)840.14 (3.33×)

WorkBuddy Office210.03664.77 (3.17×)805.53 (3.84×)215.05662.34 (3.08×)763.83 (3.55×)

SWE-bench Pro241.86657.71 (2.72×)713.41 (2.95×)248.17711.60 (2.87×)791.30 (3.19×)

τ²-bench301.39891.37 (2.96×)964.30 (3.20×)340.39871.98 (2.56×)978.39 (2.87×)

Installation

git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .

Usage

sglang serve \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DSPINE \
  --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
  --speculative-num-draft-tokens 16 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --speculative-draft-attention-backend fa3 \
  --linear-attn-backend triton \
  --enable-linear-replayssm-spec \
  --mamba-ssm-dtype float32 \
  --mamba-radix-cache-strategy extra_buffer \
  --max-mamba-cache-size 64 \
  --mem-fraction-static 0.80 \
  --context-length 131072 \
  --max-running-requests 8 \
  --cuda-graph-max-bs-decode 8 \
  --min-free-slots-delay 1 \
  --sampling-backend flashinfer \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --trust-remote-code

Acknowledgements

We thank DFlash, SpecForge, SGLang and FlashInfer. Dynamic verification references DSpark.

Citation

@article{zhang2026dspine,
  title   = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
  author  = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
  journal = {arXiv preprint arXiv:2609.36173},
  year    = {2026}
}

License

Apache 2.0