tatianyi/Qwen3.8-Flash-Next-Uncensored-Halogen

中文文档

Qwen3.8-Flash-Next Uncensored · Halogen HGN Quant

This repository is a re-quantized .hgn (Halogen) build of orcarouter/Qwen3.8-Flash-Next-Uncensored, tuned for inference on unified-memory platforms such as the AMD Ryzen AI MAX+ 395 (Strix Halo) 128 GB developer kit.

Pipeline:

1. Quantize the upstream weights to GGUF using llama-quantize, guided by the importance matrix from mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF.
2. Convert the GGUF checkpoint to .hgn with the convert tool shipped in halogen-flash-server.
3. The tensor-type recipe is based on ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF and lifted where the Halogen runtime benefits from higher-precision layers.

---

⚠️ Disclaimer

The weights distributed from this repository are an uncensored model. They are derived from upstream Uncensored weights and have only been re-quantized and format-converted; most of the safety alignment built into the original Qwen releases has been removed, and the model will not reliably refuse content that an aligned model would.

By using these files you acknowledge and agree that:

1. Possible outputs. The model can produce inaccurate, fabricated, misleading, biased, offensive, or illegal content.
2. You are responsible. You are solely responsible for the prompts you send, the outputs you receive, and every downstream use of those outputs, and for assessing legal, ethical, and compliance implications before any production deployment.
3. Prohibited uses. You must not use this model for illegal purposes, harassment, fraud, violence, CSAM, discrimination, privacy violations, weapons or hazardous-substance synthesis, or any other use reasonably likely to cause harm. You are encouraged to add your own guardrails (content filtering, age restrictions, audit logging) and to avoid using raw, unreviewed outputs for high-stakes decisions in medicine, law, finance, or safety-critical domains.
4. No warranty; limitation of liability. The model is provided "as is", without any express or implied warranty of merchantability, fitness for a particular purpose, accuracy, or non-infringement. To the maximum extent permitted by applicable law, the publisher is not liable for any direct or indirect damages arising from your use of this model.

---

Files in this repository

File	Description
Q38FN.hgn	Halogen-format weights produced by the convert tool
tensor-types-halogen2.txt	Per-tensor quantization recipe consumed by llama-quantize

The vision tower (qwen38-flash-next-vision.hgn), MTP head (qwen38-flash-next-mtp.hgn), and tokenizer are not redistributed here. You can download those files from 'peonist-ai/halogen-qwen3.8-flash-next'. See Deployment for how to mount them at runtime.

---

Quantization pipeline

Stage	Tool / source	Notes
Base weights	orcarouter/Qwen3.8-Flash-Next-Uncensored	Upstream uncensored checkpoint
Importance matrix	mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF	Drives the imatrix-guided quant
GGUF quantizer	llama-quantize (llama.cpp)	Imatrix-aware quantization
Format conversion	halogen convert	Part of peonist-ai/halogen-flash-server
Recipe baseline	ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF	Lifted for Halogen

About tensor-types-halogen2.txt

• Built on top of the GSQ-RCO IQ3 recipe in ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF.
• A few tensor types are lifted to higher precision so the model stays coherent under Halogen's runtime on a 128 GB Strix Halo device.
• The legacy tensor-types-halogen.txt (no 2 suffix) is deprecated. It pushed llama-quantize's output past the 128 GB ceiling of the AI MAX+ 395 — out of memory. Use tensor-types-halogen2.txt.
• The file is published alongside the weights so anyone with the same imatrix can reproduce a bit-for-bit identical quant.

---

Reproduce

# 1. Pull the base weights and the imatrix
huggingface-cli download orcarouter/Qwen3.8-Flash-Next-Uncensored \
    --local-dir /models/bf-orig
huggingface-cli download mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF \
    --local-dir /models/qwen38-i1 --include "imatrix*"

# 2. llama-quantize (quant type placeholder — see tensor-types-halogen2.txt)
llama-quantize \
    --imatrix ./Qwen3.8-Flash-Next-Uncensored.imatrix.gguf \
    --tensor-type-file ./tensor-types-halogen2.txt \
    Q38FNBF16.gguf Q38FN.gguf Q5_K

# 3) halogen convert：GGUF -> .hgn
halogen convert \
    /models/bf-orig/Q38FN.gguf \
    /models/selfmade/Q38FN.hgn

---

Deployment · AMD Ryzen AI MAX+ 395 (128 GB)

docker run --rm \
  -p ${PORT}:8731 \
  --name halogen \
  --device /dev/kfd \
  --device /dev/dri \
  --ipc=host \
  --ulimit memlock=-1:-1 \
  -v /mnt/data/ai/models/:/models:ro \
  -e HALOGEN_WEIGHTS_LOCK=1 \
  -e HALOGEN_MAX_TOKENS_CAP=262144 \
  -e HALOGEN_MAX_TOKENS_DEFAULT=65536 \
  -e HALOGEN_API_PORT=8731 \
  -e HALOGEN_VERBOSE=1 \
  -e HALOGEN_CHECKPOINT=/models/selfmade/Q38FN.hgn \
  -e HALOGEN_VISION_TOWER=/models/halogen-models/qwen38-flash-next-vision.hgn \
  -e HALOGEN_MTP_HEAD=/models/halogen-models/qwen38-flash-next-mtp.hgn \
  -e HALOGEN_TOKENIZER=/models/halogen-models/tokenizer \
  -e HALOGEN_MODEL_ID=Qwen3.8-Flash-Next \
  ghcr.io/peonist-ai/halogen-flash-server:latest

After one full turn of conversation, the process holds around 117 GB of the device's unified memory — fits comfortably inside a 128 GB Strix Halo system without swapping.

About halogen-flash-server

The .hgn weights in this repository must be served by halogen-flash-server. It is a lightweight inference backend aimed at large-unified-memory AMD devices, with:

• Native ROCm paths and a WEIGHTS_LOCK mode for low-overhead long-running serving
• Context lengths default to 256k tokens and MTP (Multi-Token Prediction) support
• Pluggable vision tower for multimodal inference
• An OpenAI-compatible HTTP API
• A convert tool that takes GGUF and produces .hgn

For installation, tuning, and benchmarks, see: https://github.com/peonist-ai/halogen-flash-server

---

Acknowledgements

This release stands on the shoulders of a number of excellent open-source projects:

• orcarouter/Qwen3.8-Flash-Next-Uncensored — base weights
• mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF — importance matrix that keeps the quant coherent
• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF — the GSQ-RCO recipe this build extends
• ggml-org/llama.cpp — llama-quantize and the GGUF ecosystem
• peonist-ai/halogen-flash-server — convert tool and runtime that produce and serve .hgn
• The Qwen team for the underlying Qwen model family

---

License

Each component keeps its own license:

• Base weights follow the license of orcarouter/Qwen3.8-Flash-Next-Uncensored.
• The GGUF output and imatrix retain their respective upstream terms.
• The .hgn weights and the tensor-types-halogen2.txt recipe in this repository are released under the license in LICENSE.