Abstract

This paper quantifies how far bf16 transformer large language model (LLM) weights—and Q4 K-typed tensors inside GGUF Q4 K M files—can be compressed with no loss of information. Every method is scored under one accounting discipline: a trained profile is billed to its method on each file, each run is checked for a byte-exact roundtrip, and the train/test partition is taken over source models rather than individual tensors. For bf16 weights the marginal-byte ceiling sits at ≈1.495× (model-level 95% confidence interval [1.487, 1.502]); the strongest coder that can be run end-to-end under the present accounting—the bf16 split coder introduced here—reaches 1.488×, just below that ceiling. For Q4 K-typed tensors the marginal-byte ceiling is ≈1.076× at the tensor-stream level, dropping to 1.041–1.045× once whole GGUF artifacts are scored, because Q4 K M files also carry Q6 K tensors that barely compress (near 1.01×); the mixture-CDF coder presented here attains 1.052×, whereas a dictionary-trained zstd ends up enlarging the stream once its dictionary is billed per file. Between adjacent same-role weight matrices at bf16 precision no exploitable linear redundancy is found (median Pearson +0.0004 over 250 layer pairs drawn from two Qwen2.5 models). Across 11,942 verified-roundtrip method-evaluations spanning 7,960 distinct benchmark rows, not a single roundtrip failed; the contribution is the comparison protocol itself rather than another compressor. Each ceiling is a compression-ratio bound—hardware independent and confirmed on weights up to 7B parameters (Qwen2.5-7B-Instruct); the single-core x86 (Sapphire Rapids) host constrains only the throughput figures, which are reported for comparability rather than as deployment numbers.

References

• Zhang, T., Sui, Y., Zhong, S., Chaudhary, V., Hu, X., and Shrivastava, A. 2025. 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float. In Advances in Neural Information Processing Systems (NeurIPS). arXiv:2504.11651.
• Nikulin, I. 2026. Unweight: Lossless MLP Weight Compression for LLM Inference. Tech. Rep. Cf-TR-2026.04.v1, Cloudflare Research.
• Hershcovitch, M., Choshen, L., Wood, A., Enriquez, I., Loaiza-Ganem, G., Peleg, T., Kim, H., Klein, T., and Harnik, D. 2024. ZipNN: Lossless Compression for AI Models. arXiv:2411.05239.
• Hao, Y., Cao, Y., and Mou, L. 2024. NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks. arXiv:2410.20650.
• Yang, Z., Zhang, T., Xie, Y., Li, B., Xu, Y., and Shrivastava, A. 2026. To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration. In International Conference on Learning Representations (ICLR). arXiv:2510.02676.
• Lindstrom, P. 2014. Fixed-Rate Compressed Floating-Point Arrays. IEEE Transactions on Visualization and Computer Graphics 20, 12, 2674–2683.
• Lindstrom, P., and Isenburg, M. 2006. Fast and Efficient Compression of Floating-Point Data. IEEE Transactions on Visualization and Computer Graphics 12, 5, 1245–1250.
• Liang, X., Zhao, K., Di, S., Li, S., Underwood, R., Gok, A. M., Tian, J., Deng, J., Calhoun, J. C., Tao, D., Chen, Z., and Cappello, F. 2023. SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors. IEEE Transactions on Big Data 9, 2, 485–498.
• Galicer, M., Nikulin, I., and Branch, C. 2026. Unweight: how we compressed an LLM 22% without sacrificing quality. Cloudflare Blog, Apr. 17, 2026.
• Shannon, C. E. 1948. A Mathematical Theory of Communication. Bell System Technical Journal 27, 3, 379–423.
• Hu, E. J., Shen, Y.,Wallis, P., Allen-Zhu, Z., Li, Y.,Wang, S., Wang, L., and Chen, W. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations (ICLR). arXiv:2106.09685.
• Kingma, D. P., and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR). arXiv:1412.6980.