TIThe Inference
Inference & Models·Advanced · 8–16h
Build your own quantizer
Shrink GPT-2 to int8/int4 and keep it coherent.
What you'll build
An int8/int4 GPT-2 within a perplexity budget.
Prerequisites
Shrink GPT-2 to int8/int4 and keep it coherent.
What you'll build
An int8/int4 GPT-2 within a perplexity budget.
Prerequisites