Mariusz Kurman PRO

Great article!

One thing worth noting -> it opens and closes the reasoning tag due to the missing enable_thinking flag in your tokenizer's apply_chat_template during generation.

text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It might or might not learn anything special inside the reasoning part, but you forced it not to show it :P

Qwen3.5 0.8B chat_template:

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is true %}
        {{- '<think>\n' }}
    {%- else %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}