---
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
library_name: transformers
model_name: semanticwiki-coder-14b-v3
tags:
- generated_from_trainer
- trl
- sft
- hf_jobs
- code-documentation
- technical-writing
- semanticwiki
licence: license
license: apache-2.0
---

# SemanticWiki Coder 14B v3

LoRA adapter fine-tuned from [Qwen/Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct)
to generate **DeepWiki-style architectural wiki pages** with **verifiable `path:line`
source citations**, Mermaid diagrams, and structured sections.

## What it does

Given a line-numbered code context and a query, it writes a wiki page whose factual
claims carry strict `path/file.ext:line` or `path:start-end` citations that resolve
against the real repository.

Input format (same as training):

```
<START_OF_CONTEXT>
# Repository: org/name (language: ...)
# File tree ...
### FILE: src/module.py (lines 1-N)
   1| <source with line numbers>
<END_OF_CONTEXT>

<query>
Generate a wiki page describing ...
</query>
```

## Training

| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen2.5-Coder-14B-Instruct |
| Method | LoRA SFT (r=64, alpha=128, dropout 0.05, all linear projections) |
| Data | [GhostScientist/semanticwiki-data-v3](https://huggingface.co/datasets/semanticwiki-data-v3) - 313 pages from 97 real GitHub repos, every citation deterministically verified |
| Loss | completion-only (prompt-completion format; the code context is not trained on) |
| Epochs / LR / effective batch | 2 / 1e-4 cosine / 16 |
| Max sequence length | 24576 |
| Final loss / token accuracy | 0.5101 / 0.823 |

## Paired eval (SemanticWiki-Eval v3, 13 held-out repos, 26 pages)

| Arm | Format | Citation validity | Fidelity (judge 1-5) | Citations/page |
|---|---:|---:|---:|---:|
| **this model** | **0.812** | **0.562** | 3.39 | 4.9 |
| Qwen2.5-Coder-14B-Instruct (base) | 0.755 | 0.308 | 3.15 | 2.0 |

Fine-tuning improved every axis; citation validity rose +83% relative. Full protocol
and per-page results: [GhostScientist/semanticwiki-eval-v3](https://huggingface.co/datasets/GhostScientist/semanticwiki-eval-v3).

## Limitations

Citation validity on unseen repos is 0.56 vs the teacher dataset's ~1.0 - hallucinated
line numbers remain the main failure mode. Fidelity is judge-based (judge shares the
Qwen3 family; see benchmark card caveats).

## Framework versions

TRL 1.14.1 - Transformers 5.18.0 - PyTorch 2.14.1 - PEFT 0.21.2 - Datasets 5.1.0

## Citation

```bibtex
@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallou{\'e}dec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}
```