Prefix caching·2 min
When does prefix reuse break?
0 / 1
Knowledge check
You saved a 200-token system-prompt cache. Which change makes reusing it incorrect for GPT-2?
Build your own KV cache
You saved a 200-token system-prompt cache. Which change makes reusing it incorrect for GPT-2?