Tencent Hy4-preview Shrinks from 1.5TB to ~200GiB with MIX-STQ1_0 Mixed-Precision Quantization

Tencent Hy ·

核心信息

Tencent's Hy4-preview, a 770B MoE with 256 routed experts per layer, has been compressed from 1.5TB to ~200GiB GGUF using MIX-STQ1_0, a calibration-driven mixed-precision quantization method. Instead of applying one low bit-width uniformly, the method allocates bit-width per layer, keeping accuracy nearly unchanged versus BF16.

要点

  • MIX-STQ1_0 treats bit-width as an allocation problem: calibration data picks each layer's precision, using 1.3125 bpw STQ1_0 (ternary {-d, 0, +d} with forced 3:4 sparsity) on 29 layers and 2.06 bpw IQ2_XXS on the rest.
  • Down experts get higher precision because they write directly into the residual stream; most of the model's mass sits in routed gate/up projections.
  • Accuracy barely moves vs BF16: MCP Atlas 83.7→83.2, SWE-Bench multi 82.9→81.3, MRCR 81.3→81.1, IFBench 73.5→72.5.
  • Weights and low-bit GGUFs are available on Hugging Face at AngelSlim/Hy4-preview-GGUF.
Loading...