Hy4-preview Compressed to ~200GiB GGUF with Mixed-Precision Quantization, Accuracy Nearly Unchanged

Tencent Hy ·

Core Info

AngelSlim released Hy4-preview GGUF, compressing the model from 1.5TB to ~200GiB using layer-wise mixed-precision quantization (STQ1_0 and IQ2_XXS), with minimal accuracy loss.

Key Points

  • The trick is not simply going low-bit but choosing per-layer bit-width from calibration data: some layers drop to 1.31-bit STQ1_0, others use 2.06-bit IQ2_XXS.
  • Accuracy barely moves vs BF16: MCP Atlas 83.7→83.2, SWE-Bench multi 82.9→81.3, MRCR 81.3→81.1, IFBench 73.5→72.5.
  • Weights and low-bit GGUFs are available on Hugging Face: AngelSlim/Hy4-preview-GGUF.
Loading...