On July 14, Tencent Hunyuan released the official IQ1_M and Q4_K_M GGUF quantized versions of Hy3. The most widely circulated figure is this: a 295B-parameter model compresses to roughly 85.5 GiB, small enough to load onto a single 96GB GPU.
That number is accurate, but it answers only whether the weights can fit. At runtime the model also needs a KV cache and compute buffers, and quantization can affect reliability on complex tasks. Add time-to-first-token and generation speed, and there are at least three separate bills to settle between “fits on one card” and “usable day to day.”
Hy3 is an open-source text-only model released by Tencent Hunyuan, with 295B total parameters and 21B active parameters, supporting 256K context under the Apache 2.0 license. It uses a MoE architecture, invoking only a subset of expert weights on each step. Total parameter count determines the model’s capacity, while the 21B active parameters govern the actual computation scale per step.
This design brings clear trade-offs. According to comprehensive tests by the third-party evaluator Artificial Analysis, Hy3 sits in a similar range as DeepSeek V4-Pro but below GLM-5.2. By task, web search, long-document retrieval, and tool orchestration are relative strengths; repository-level coding, terminal operations, and long-horizon software engineering are weaker. Tencent Hunyuan trades lower active compute for broader agent capability coverage, without simultaneously pursuing the strongest coding performance.
Quantization stores model weights at lower precision to reduce VRAM usage. IQ1_M, the most talked-about variant in AngelSlim/Hy3-GGUF, is commonly called the 1-bit version.
The “1-bit” here is the name of a quantization format. According to
Hugging
Face file metadata, the MTP-free Hy3-IQ1_M.gguf is
83.30 GiB, meaning each weight averages 2.425 bits for the full file,
i.e., 2.425 BPW; the version with MTP is 85.45 GiB and 2.488 BPW. The
MTP-free Q4_K_M version is 169.81 GiB.
The actual bit-width is higher than the name suggests because not all tensors are compressed to the same precision by the publisher. Critical paths such as attention, shared experts, embeddings, and the output layer preserve more information, and scale parameters plus file metadata also take up space. IQ1_M compresses the large set of routed expert weights most aggressively; the full file remains a mixed-precision recipe.
After the weights are loaded, the model must also store the context
it has read. The KV cache consists of these intermediate
attention results; the longer the input, the more VRAM it consumes. With
Hy3 using a q8 KV cache, a 64K context requires roughly 10.63 GiB.
Add that to the weights, and the single-card budget tightens. The MTP-free IQ1_M occupies 83.30 GiB; together with a 64K q8 KV cache the total reaches 93.93 GiB, leaving only 2.07 GiB. CUDA context, compute buffers, the computation graph, and VRAM fragmentation are not yet accounted for. The official release does provide a configuration for running 64K on a single H20 and labels it tight, but no peak-VRAM or speed logs for that configuration have been published. It shows the approach is plausible, not that production runs would be comfortable.
The weight file with MTP is 85.45 GiB. Adding the same main model KV cache pushes the arithmetic total to 96.08 GiB, not yet including the draft KV cache required by MTP itself or other runtime overhead. The official single-card recommendation therefore chooses MTP-free, 64K context, and q8 KV cache. Hy3 supports 256K — that is the architecture’s upper bound, not a claim that a single 96GB card can use a context that long.
At present, no complete comparison of BF16, FP8, Q4_K_M, and IQ1_M under the same evaluation pipeline has been released by Tencent. The public statement is: 4-bit approaches BF16 on agent, code, tool-calling, and long-text understanding tasks; for 1-bit, long-text understanding stays close to the original, while agent and coding capabilities see a modest decline.
The issue is that a modest decline, distributed across multi-step tasks, may not remain modest. A coding agent must consecutively select tools, fill in JSON parameters, modify code, and evaluate test results. A deviation at any step can alter all downstream actions. That ordinary Q&A shows no obvious difference does not prove the same reliability holds for long-horizon tasks.
This is a risk mechanism, not a measured failure rate for Hy3 IQ1_M. Hy3 has a large parameter count, and IQ1_M assigns different precision according to weight importance, so real-world performance may be better than naive 1-bit intuition suggests. The answer still requires A/B testing on real workflows, specifically tracking format errors, invalid tool calls, retry counts, and end-to-end task success rates.
Existing performance data come mainly from community-converted variants and should not be taken as uniform official measurements for IQ1_M or Q4_K_M. Community GGUFs on an M3 Max 128GB can generate roughly 23–25 tok/s; a community MLX version on an M3 Ultra reaches about 27–29 tok/s. These speeds are adequate for daily interaction. The MLX measurements and GGUF measurements used different quantizations, so the numbers should be treated only as a rough sense of responsiveness.
Nor is generation speed the whole story. The phase in which the model first reads the prompt is called prefill. Community testing on an M3 Max measured roughly 41 tok/s, meaning a cold read of an 8K prompt would take roughly close to 200 seconds. For a coding agent with a long context, what really governs perceived pace may be the time-to-first-token, not the tokens-per-second that follows.
A community IQ2_M test on two H200s clocked 72.2 tok/s, reaching 81.5 tok/s with MTP enabled. MTP predicts several subsequent tokens at once, which the main model then verifies. The official claim is a 50%–60% speedup, but community-auditable results depend heavily on confidence gating; this H200 test saw an improvement of roughly 13%, while Apple Silicon saw essentially no benefit. llama.cpp PR #25395 documents these discrepancies.
If you already have a 96GB GPU, you can start testing with MTP-free IQ1_M at 16K or 32K context. If you already have a 256GB M3 Ultra, you can also try the community 4-bit MLX version. Private production deployments are better served by evaluating Q4_K_M first; for low-usage, data-insensitive scenarios, APIs often save more maintenance effort.
Hy3 demonstrates how far large-model compression has come, while also exposing the costs: lower active compute, a smaller weight file, a shorter practical context, and complex-task reliability that remains to be verified. It suits those who already own the hardware and want to experiment; it is not yet a reason to buy a high-end Mac or H20 just for this.