Theoretical noise analyses for quantization
Reviewing some literature I came across the following two papers: Fixed Point Quantization of Deep Convolutional Networks, and The Validity of the Additive Noise Model for Uniform Scalar Quantizers. Both tackle the noise model differently, focusing more on the SQNR of weights and activations, and finding an 'optimal bit' efficiency. They also rely on the gaussian model, even for activations. I've avoided SQNR so far - although it seems like a good metric of quantization error, the correlation between SQNR and model accuracy has always felt dicey at best in experiments I've run. Perhaps it's time to revisit this...