Series

Quantization

In this series, I will try to break down the quantization process and it's underpinnings as simple as possible. I feel it's high to focus on techniques to make LLM training and inference accessible even for a college student with low resources.

Understanding Quantization: Part I
Introduction Quantization in general can be defined as mapping values from a large set of real numbers i.e., FP32 or even FP16 to values in a small discrete set most likely Int8 or Int4. There are recent works trying to map to 1bit models. Typically ...
Sep 26, 20246 min read42

Command Palette