--- title: Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored canonical_url: "https://www.modelscope.ai/models/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored" md_url: "https://www.modelscope.ai/models/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored.md" repository: RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored last_updated: 2026-09-07 license: apache-2.0 pipeline_tag: image-text-to-text tasks: - image-text-to-text model_type: - qwen3_5 architectures: - Qwen3_5ForConditionalGeneration base_model: - huihui-ai/Huihui-Qwen3.8-27B-abliterated base_model_relation: quantized library_name: - gguf downloads: 6 stars: 0 tags: - qwen3 - qwen3.8 - uncensored - abliterated - gsq - rco - gguf - quantized - speculative-decoding - multilingual - multimodal - vision - conversational - long-context - tool-use - 16gb-vram --- # Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored > Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored - An open-source model by RentedNoodle on ModelScope. Qwen3.8-27B-GSQ-RCO-IQ3XXS-Uncensored RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored is a image-text-to-text model on ModelScope. licensed under apache-2.0. derived from huihui-ai/Huihui-Qwen3.8-27B-abliterated. - **Repository**: RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored - **License**: apache-2.0 - **Tasks**: image-text-to-text - **Base model**: huihui-ai/Huihui-Qwen3.8-27B-abliterated - **Tags**: qwen3, qwen3.8, uncensored, abliterated, gsq, rco, gguf, quantized, speculative-decoding, multilingual, multimodal, vision, conversational, long-context, tool-use, 16gb-vram - **Downloads**: 6 - **Stars**: 0 - **Last updated**: 2026-09-07 Source: https://www.modelscope.ai/models/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored --- # Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored 9.75 GiB 混合精度 GSQ/RCO 量化版 Huihui 去限制 Qwen3.8-27B,保留 MTP 推测解码头,专为单张 16 GB 显卡上的 196K 长上下文验证。 这是 **`huihui-ai/Huihui-Qwen3.8-27B-abliterated`** 的量化版本,不是新的微调。底层模型是 Qwen3.8-27B;去限制来自 Huihui;量化方法遵循 ISTA-DASLab 的 GSQ/RCO。该检查点显著降低了拒绝行为,但不保证普遍服从——部署者需自行负责合规使用。 ## 快速规格 | | | |---|---| | 基座 | `Qwen/Qwen3.8-27B` 经 `huihui-ai/Huihui-Qwen3.8-27B-abliterated` | | 文件 | `Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf` (v1.1 发布版;v1 `-MTP.gguf` 已弃用) | | 大小 | 10,466,420,544 字节 (9.75 GiB) | | 张量 | 866 (65 blocks:64 层 + MTP 头) | | 位宽 | 3.058 (字节数×8 ÷ 27,320,697,856 参数) | | MTP 头 | 已保留 — 15 个 `blk.64.*` 权重(含 `nextn.eh_proj/enorm/hnorm/shared_head_norm`),已验证非空,实测 draft 接受率 0.46–0.95 | | 对话模板 | `froggeric-qwen3.8-tool-use.jinja` (本地文件头 v22.5) | | 上下文 | 16 GB 卡实测 196K 常驻 (`q4_0` KV);不承诺所有 16 GB 配置 | ## 来源说明(必读) 1. **基座:** Huihui 去限制版 Qwen3.8-27B safetensors。 2. **分配方案:** 逐字复用 ISTA-DASLab 公开的 Qwen3.8-27B-GSQ-RCO-IQ3_XXS per-tensor 位宽分配表。 3. **量化:** `llama.cpp` 的 `llama-quantize --tensor-type-file` 映射为原生 GGUF 类型。 4. **没有做的事:** 没有独立重跑多卡 RCO 预算搜索。公开版分配表 + 不同基座。 5. 自定义 importance-matrix 混合方案试过,needle 检索直接失败 (0/6 超时),已丢弃,保留官方 ISTA imatrix。好看但难用的数字不发布。 ## 实测性能 (RTX 5070 Ti 16 GB, build2 llama.cpp, 均为本文件实测) | 配置 | 解码速度 | 说明 | |---|---|---| | 串行 / 无推测解码 | ~39.6 tok/s | 400-token prose, thinking 开启 | | MTP `n-max 2` | ~59.3 tok/s | +50%。默认配置。接受率 0.46–0.95 (代码高 prose 低),平均长度 1.9–2.9 | | MTP `n-max 2` + `--fit off` + `--spec-draft-backend-sampling` + `-b 1024` | 69–82 tok/s | 发布配置。仅 `--fit off` 一项就带来 +38% | | MTP n-max 3 / 4 (+p-min) | 33–37 tok/s | 接受率更高但更慢——验证开销占优。不用 | | KV `iq4_nl` | 26.8 tok/s | 本构建无高速 kernel。不用 | | KV `q4_0/q4_0` | 发布配置 | 196K 时 15.8/16 GB 常驻 | 接受率高不等于吞吐高,验证开销同样重要——看 tok/s,不只看接受率。 | 测试集 | 得分 | 配置 | |---|---|---| | Needle 检索 | 6/6 | 15K 词 haystack,深度 0.1–0.9,temp 0.0,64K server ctx | | Toolcall v1 (Pi JSON 路径) | 8/8 | `tool_call_format=json` | | Toolcall v2 (参数 + 禁止调用 + 调用链,完整 14 例) | 9/14;negatives 4/4 全过 | 失败:tc-04 bash-vs-grep、tc-08 read-vs-edit、tc-07 noparse、tcc-01/02 chain-miss。Trunk 与 v1 逐位一致,不受头训练影响 | | Livebench-style | 12/12 | 推理/代码/抽取混合 | | Coherence | 4/4 | 多轮连贯 | | WikiText-2 困惑度 (test) | 6.63 | 单一 harness 在本文件上的测量;此处未测同 harness 的 BF16 基线,故不应读作“优于 BF16” | 温度 0.6 发布 (raw-template A/B 为 4/8 vs 3/8;agentic 循环的工具可靠性优先)。推理:默认 medium,短代码 low,仅可验证任务 xhigh。 本文件的学术基准 (AIME/GPQA/LiveCodeBench):待补。仅供参考,ISTA-DASLab 同 bpw 基座报告 AIME25 100.0 / GPQA-Diamond 88.89 / LiveCodeBench v6 84.57——那是他们的基座,不是本文件。 GPQA-Diamond (本文件):**60.6% no-thinking / 61.6% thinking-on** (pass@1,2048-token budget)对比 ISTA 的 88.89(生成式 xhigh harness,stock 基座)。差距源于 **harness + 去限制**,而非量化。(见 Ablerlitics,2026 年 9 月:Huihui 是可靠的经典去限制——遵循约 3/4 请求,推理保留,部分内容仍被拒。非“surgical 级”。) ## v1.1 MTP 头 (已训练 draft 头) 原生 MTP 头很强,但我用 4M token 自蒸馏语料微调了它以提升 draft 精度。这需要 `graph_mtp` 的数学精确 PyTorch 镜像——v1.1 线路在训练产生意义前解决了三个静默分歧 (gated-attention Q/gate 交错、NEOX RoPE 配对、float32 RMSNorm 累加) 外加两个 GGUF 管道 bug (F32 norm 保留 + `mul_mat` flat-layout 方向)。 | 指标 | v1 原生头 | v1.1 训练头 | 差值 | |---|---|---|---| | 离线 k1 (held-out,从未训练) | 0.4806 | **0.7660** | **+28.5pt** | | 离线 top5 | 0.763 | **0.925** | +16pt | | 服务 t/s (相同 ship flags,3 次中位数) | 70.68 | **75.15** | **+6.3%** | | Draft 接受率 | 0.636 | 0.620 | 持平 | 离线 k1 增益大且泛化 (在与训练分布不同处测量)。`n-max 2` 下服务吞吐被验证开销限制,头部略占优势表现为 +6.3%——并非离线增益的一对一映射。已用训练头 build 重新验证 needle 6/6。 ## 用法 Froggeric 模板是工具/推理行为所必需的 (`tool_call_format=json` 为认证路径;模板默认 xml)。 ```bash llama-server \ -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf \ --alias Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored \ --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \ --ctx-size 196608 -fa on -ctk q4_0 -ctv q4_0 \ -ngl 99 -b 1024 -ub 1024 -np 1 -t 8 --fit off \ --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2 \ --spec-draft-backend-sampling ``` 视觉 (mmproj 可选,纯文本不用): ```bash llama-server -m --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf --ctx-size 65536 [同上 flags] # 然后 POST /v1/chat/completions,content 块含 {"type":"image_url",...} ``` ## 诚实局限 激进 3-bit 量化——别指望 BF16 表现。知识回忆弱于高 bit 版本 (模型更依赖检索/工具)。MTP 输出在量化目标上可能与串行生成不同;固定 seed 和运行时配置以复现,需要逐 token 一致时关闭推测解码。视觉 + 长上下文在 16 GB 上紧张 (文本/视觉配置分离,或 offload projector)。imatrix 在 stock 基座上计算,未在 abliterated 权重上验证——影响未知,不假设安全。基座 (Huihui) 存在标准多层去限制常见的约 3pp TruthfulQA 退化 (Abliterlitics, 2026 年 9 月);长 agentic 链中应自行核实关键事实。 ## 仓库文件 | 文件 | 说明 | |---|---| | `Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf` | 量化本体 (SHA256 见 `SHA256SUMS.txt`) | | `mmproj/` | 视觉 projector:BF16 (0.87 GB) + Q8_0 (0.60 GB) | | `froggeric-qwen3.8-tool-use.jinja` | 必需的对话模板 | | `REF-IQ3_XXS-mtp.rco-allocation.txt` | 866 行 per-tensor 分配表 (比任何摘要权威) | | `imatrix.dat` | 实际使用的 ISTA-DASLab 公开 calibration 矩阵 | | `SHA256SUMS.txt` | 哈希 | | `LICENSE` | 上游 Apache-2.0 条款适用,见 Qwen/Huihui 仓库 | GGUF 内嵌 tokenizer 元数据——llama.cpp/Ollama/LM Studio/Pi 均无需单独 tokenizer。 ## 致谢 - **Qwen** — 架构 + 预训练权重:[Qwen3.8 repo](https://github.com/QwenLM/Qwen3.8)、[Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) - **Huihui** — 去限制 checkpoint ([model](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated)),经 [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) - **GSQ** — Dadgarnia, Tabesh, Nikdan, Helcig, Kurtic, Kleinegger, Alistarh (2026):[论文](https://arxiv.org/abs/2604.18556)、[代码](https://github.com/IST-DASLab/GSQ) - **RCO** — Helcig & Alistarh (2026):[论文](https://arxiv.org/abs/2605.00649)、[代码](https://github.com/IST-DASLab/RCO) - **ISTA-DASLab 量化发布** — [Qwen3.8-27B-GSQ-RCO-GGUF](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF)(分配表 + imatrix 来源) - **Froggeric** — [Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates) - **llama.cpp/GGML** — [运行时 + 格式](https://github.com/ggml-org/llama.cpp);IQ3_XXS 是其标准类型,无自定义内容 ## 可复现性 - 基座:`huihui-ai/Huihui-Qwen3.8-27B-abliterated` - 分配表:本仓库 `REF-IQ3_XXS-mtp.rco-allocation.txt`,经 `llama-quantize --tensor-type-file` 应用 - Imatrix:本仓库 `imatrix.dat`,ISTA-DASLab 公开矩阵 - 运行时:[den_llama.cpp](https://github.com/RentedNoodle/den_llama.cpp) @ `3231ee89`(build2;`-DGGML_CUDA_FA_ALL_QUANTS=ON`,CUDA 13.3) - GPU:RTX 5070 Ti 16 GB,Windows 11 - 评测:HF 本仓库 `evals/` 目录 (prompts/commands/raw) ## 版本 ### 更新日志 * **v1.1 (当前):**微调 MTP 头 (+28.5pt 离线 k1,+6.3% 服务 t/s)。新增 WikiText-2 困惑度验证 (6.63)。完整 14 例 toolcall 套件对账 (9/14)。Trunk 与 v1 逐位一致——无基座智能损失或变更。 * **v1.0:**初始发布。GSQ-RCO IQ3_XXS 分配应用于 Huihui 去限制基座。Needle 6/6,MTP 完整。 ### 路线图 * **v1.2 (Huihui 线)—— 暂停。** 我们转向 OrcaRouter 基座做 v2.0(外科式去限制、更干净的 TruthfulQA 保留、原生去除限制的 MTP 头)。不再计划 Huihui 线发布;v1.1 是 Huihui 最终版本。 * **v2.0 (计划中):**在 `orcarouter/Qwen3.8-27B-Uncensored` 上重建——为其 131 个编辑张量定制保护分配、重新量化、needle 门控,并在已去限制的 OrcaRouter 头上同步进行 MTP 头微调。发布时更新本卡。 社区量化,与 Qwen、Huihui、ISTA-DASLab 无隶属关系。Bug 反馈请附:模型版本、llama.cpp 构建、ctx/KV/spec 配置、采样器。"感觉变差"可以接受;可复现信息更有用。 --- [⬇️ 完整英文文档 / Full English Documentation Below ⬇️](#english) --- # Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored A 3-bit GGUF quant of the abliterated Qwen3.8-27B, built to hold full 196K context on a single 16 GB GPU. Sibling size class carries ISTA-DASLab's full-size AIME/GPQA numbers; this file trades recall for residency and says so below. A 9.75 GiB, mixed-precision GSQ/RCO quant of Huihui's refusal-reduced Qwen3.8-27B, preserving the MTP head and validated for 16 GB GPUs. This is a **quantization of `huihui-ai/Huihui-Qwen3.8-27B-abliterated`**, not a new fine-tune. The underlying model is Qwen3.8-27B; the refusal ablation comes from Huihui; the quantization work here follows the GSQ/RCO methodology developed by ISTA-DASLab. The checkpoint is designed to substantially reduce learned refusal behavior. This is not a guarantee of universal compliance — the deployer is responsible for compliant use. ## Quick specs | | | |---|---| | Base | `Qwen/Qwen3.8-27B` via `huihui-ai/Huihui-Qwen3.8-27B-abliterated` | | File | `Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf` (v1.1 release; v1 `-MTP.gguf` superseded) | | Size | 10,466,420,544 bytes (9.75 GiB) | | Tensors | 866 (65 blocks: 64 layers + MTP head) | | bpw | 3.058 (bytes×8 ÷ 27,320,697,856 params) | | MTP head | Present - 15 `blk.64.*` weights incl. `nextn.eh_proj/enorm/hnorm/shared_head_norm`, verified non-empty, draft-accept 0.46-0.95 by workload logged against this exact file | | Template | `froggeric-qwen3.8-tool-use.jinja` (local file header v22.5; upstream repo may still show v22.4) | | Context | 196K resident-fit on 16 GB (`q4_0` KV, verified loaded); quality gates ran at 32K server ctx + 64K needle ctx — **known gap:** full-196K retrieval accuracy is under active evaluation, do not treat as 196K-verified | ## What I changed Starting from the abliterated checkpoint, I produced this GGUF using a GSQ/RCO-based non-uniform allocation and IQ3_XXS packing. This isn't a generic `IQ3_XXS` conversion: sensitive tensors (SSM gates, embeddings, attention gates — 96 tensors) are held at BF16 while the bulk rides the GSQ-coded grid. The important honesty clause: I **reproduced ISTA-DASLab's published per-tensor allocation map** for Qwen3.8-27B and applied it to the abliterated weights. I did not independently re-run the multi-GPU RCO budget search. Same map, different base. I also tried custom importance-matrix blends during the process (fresh + syncretized). They looked promising and then failed needle retrieval outright (0/6 timeouts), so I threw them out and kept the official ISTA imatrix. If a change makes the benchmark prettier but the model worse to actually use, it doesn't ship. ## Tested performance (RTX 5070 Ti 16 GB, build2 llama.cpp, this exact file) | Configuration | Decode | Notes | |---|---|---| | Serial / no spec | ~39.6 t/s | 400-tok prose, thinking on | | MTP `n-max 2` | ~59.3 t/s | +50%. Ship default. Accept 0.46-0.95 by workload (code peaks, prose dips), mean len 1.9-2.9 | | MTP `n-max 2` + `--fit off` + `--spec-draft-backend-sampling` + `-b 1024` | 69–82 t/s | Ship flags. The `--fit off` alone was worth +38% | | MTP n-max 3 / 4 (+p-min) | 33–37 t/s | Higher acceptance yet slower — verification cost wins. Don't use | | KV `iq4_nl` | 26.8 t/s | No fast kernel in this build. Don't use | | KV `q4_0/q4_0` | ship | 15.8/16 GB resident | Higher acceptance does not automatically mean higher throughput. The verification cost matters — measure tok/s, not just accept rate. | Suite | Score | Setup | |---|---|---| | Needle retrieval | 6/6 | 15K-word haystack, depths 0.1–0.9, temp 0.0, 64K server ctx | | Toolcall v1 (Pi JSON path) | 8/8 | `tool_call_format=json` | | Toolcall v2 (args + must-not-fire + chains, full 14-case) | 9/14; negatives 4/4 clean | Fails: `tc-04` bash-vs-grep, `tc-08` read-vs-edit, `tc-07` noparse, `tcc-01/02` chain-miss. Trunk bit-identical to v1, so unchanged by the head work | | Livebench-style | 12/12 | reasoning/coding/extraction mix | | Coherence | 4/4 | multi-turn continuity | | WikiText-2 perplexity (test) | 6.63 | single-harness measurement on this file; no identical-harness BF16 baseline measured here, so do not read as 'beats BF16' | Temp ships at 0.6 (4/8 vs 3/8 raw-template A/B; favored for tool-use reliability in agentic loops). Reasoning: medium default, low for short code, xhigh for verifiable-correctness only. Academic benchmarks (AIME/GPQA/LiveCodeBench) for this specific file: pending, will append. For reference, ISTA-DASLab's same-bpw base reports AIME25 100.0 / GPQA-Diamond 88.89 / LiveCodeBench v6 84.57 — their base, not this file. GPQA-Diamond (this file): **60.6% no-thinking / 61.6% thinking-on** (pass@1, 2048-token budget) vs ISTA's 88.89 (generative xhigh harness, stock base). The gap is **harness + abliteration**, not quantization. (See Ablerlitics, Sept 2026: Huihui is the reliable classic ablation - follows ~3/4 requests, reasoning preserved, some content still refused. Not 'surgical tier'.) ## v1.1 MTP head (trained draft head) The native MTP head is strong, but I fine-tuned it on a 4M-token self-distilled corpus to improve draft accuracy. This required a mathematically exact PyTorch mirror of `graph_mtp` — the v1.1 line resolved three silent divergences (gated-attention Q/gate interleave, NEOX RoPE pairing, float32 RMSNorm accumulation) plus two GGUF-plumbing bugs (F32 norm preservation + `mul_mat` flat-layout orientation) before any training was meaningful. | Metric | v1 native head | v1.1 trained head | Delta | |---|---|---|---| | Offline k1 (held-out, never-trained-on) | 0.4806 | **0.7660** | **+28.5pt** | | Offline top5 | 0.763 | **0.925** | +16pt | | Serve t/s (identical ship flags, median-of-3) | 70.68 | **75.15** | **+6.3%** | | Draft acceptance | 0.636 | 0.620 | flat | The offline k1 gain is large and generalizes (measured on a different distribution than trained on). Serve-time throughput at `n-max 2` is bounded by verification cost, so the head's drafting advantage shows as a modest +6.3% — not a one-to-one map of the offline gain. Needle 6/6 re-verified on the trained-head build. ## Usage Froggeric template is required for the intended tool/reasoning behavior (`tool_call_format=json` for the certified path; template default is xml). ```bash llama-server \ -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf \ --alias Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored \ --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \ --ctx-size 196608 -fa on -ctk q4_0 -ctv q4_0 \ -ngl 99 -b 1024 -ub 1024 -np 1 -t 8 --fit off \ --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2 \ --spec-draft-backend-sampling ``` Vision: projector files live in this repo under `mmproj/` (BF16 or Q8_0). Start the server with one, then send a multimodal chat request (skip both flags for text-only): ```bash llama-server -m --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf --ctx-size 65536 [same flags as above] # then POST /v1/chat/completions with content blocks including {"type":"image_url",...} ``` Ollama (text generation supported; tool/reasoning behavior uncertified — use llama-server + Froggeric for that): ```dockerfile FROM ./Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf PARAMETER num_ctx 196608 PARAMETER num_gpu 999 ``` ## Honest limitations Aggressive 3-bit quant — don't expect BF16 behavior. Knowledge recall is weaker than higher-bit variants (the model leans on retrieval/tools instead). MTP output may differ from serial generation on quantized targets; fix the seed and runtime config for reproducibility, disable spec when exact serial behavior matters. Vision + long context gets tight on 16 GB (split text/vision profiles or offload the projector). The imatrix was computed on the stock base, not the abliterated weights — whether that shift matters is untested, not assumed safe. Base model (Huihui) exhibits a known ~3pp TruthfulQA degradation common to standard multi-layer ablations (Abliterlitics, Sept 2026); users should verify critical facts in long agentic chains. ## Files in this repo | File | What | |---|---| | `Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-v1.1.gguf` | The quant (SHA256 in `SHA256SUMS.txt`) | | Vision projector | `mmproj/mmproj-Qwen3.8-27B-BF16.gguf` (0.87 GB) + Q8_0 alt (0.60 GB, 27 tensors fallback-quantized) | | `froggeric-qwen3.8-tool-use.jinja` | Required chat template | | `REF-IQ3_XXS-mtp.rco-allocation.txt` | 866-row per-tensor allocation map (authoritative over any summary) | | `imatrix.dat` | ISTA-DASLab's published calibration matrix as used | | `SHA256SUMS.txt` | Hashes | | `LICENSE` | Upstream Apache-2.0 terms apply; see Qwen/Huihui repos | GGUF embeds tokenizer metadata — no separate tokenizer needed for llama.cpp/Ollama/LM Studio/Pi. ## Credits - **Qwen** — architecture + pretrained weights: [Qwen3.8 repo](https://github.com/QwenLM/Qwen3.8), [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) - **Huihui** — abliterated checkpoint ([model](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated)), via [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) - **GSQ** — Dadgarnia, Tabesh, Nikdan, Helcig, Kurtic, Kleinegger, Alistarh (2026): [paper](https://arxiv.org/abs/2604.18556), [code](https://github.com/IST-DASLab/GSQ) - **RCO** — Helcig & Alistarh (2026): [paper](https://arxiv.org/abs/2605.00649), [code](https://github.com/IST-DASLab/RCO) - **ISTA-DASLab quant release** — [Qwen3.8-27B-GSQ-RCO-GGUF](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF) (allocation + imatrix source) - **Froggeric** — [Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates) - **llama.cpp/GGML** — [runtime + format](https://github.com/ggml-org/llama.cpp); IQ3_XXS is their standard type, nothing custom here ## Reproducibility - Base: `huihui-ai/Huihui-Qwen3.8-27B-abliterated` - Allocation: `REF-IQ3_XXS-mtp.rco-allocation.txt` (in this repo), applied via `llama-quantize --tensor-type-file` - Imatrix: `imatrix.dat` (in this repo), ISTA-DASLab's published matrix as used - Runtime: [den_llama.cpp](https://github.com/RentedNoodle/den_llama.cpp) @ `3231ee89` (build2; `-DGGML_CUDA_FA_ALL_QUANTS=ON`, CUDA 13.3) - GPU: RTX 5070 Ti 16 GB, Windows 11, driver per `nvidia-smi` at run time - Evals: prompts in `evals/prompts/` (hashes in `evals/SHA256SUMS`), commands in `evals/commands.md`, raw outputs + timings + accept stats in `evals/raw/` ## Versions ### Changelog * **v1.1 (current):** fine-tuned MTP head (+28.5pt offline k1, +6.3% serve-time t/s). Added WikiText-2 perplexity validation (6.63). Reconciled 14-case toolcall suite (9/14). Trunk bit-identical to v1 — no base intelligence lost or altered. * **v1.0:** initial release. GSQ-RCO IQ3_XXS allocation applied to Huihui abliterated base. Needle 6/6, MTP intact. ### Roadmap * **v1.2 (Huihui line) — on hiatus.** We are pivoting to an OrcaRouter base for v2.0 (surgical abliteration, cleaner TruthfulQA retention, natively uncensored MTP head). No further Huihui-line releases planned; v1.1 is the final Huihui build. * **v2.0 (planned):** rebuild on `orcarouter/Qwen3.8-27B-Uncensored` — custom protected allocation for its 131 edited tensors, requant, needle-gated, plus concurrent MTP-head fine-tuning on the already-uncensored OrcaRouter head. This card updates on release. Community quant, not affiliated with Qwen, Huihui, or ISTA-DASLab. Bug reports: include model revision, llama.cpp build, ctx/KV/spec settings, sampler. "Feels worse" is fine; reproducible info makes it actionable.