Papers by TJ Bai
Accelerating Language Model Workflows with Prompt Choreography (2026.tacl-1)
Copied to clipboard
| Challenge: | Large language models are increasingly deployed in multi-agent workflows that require multiple agents to encode the same prompt from scratch. |
| Approach: | They propose a framework that maintains a dynamic, global KV cache that allows agents to attend to arbitrary, reordered subsets of previously encoded messages. |
| Outcome: | The proposed framework significantly reduces per-message latency (2.0–6.2 faster time-to-first-token) and achieves substantial speedups (>2.2) in some workflows dominated by redundant computation. |