Qwen XML tool calls (Qwen3-Coder, Qwen3.5/3.6/3.8) on llamacpp a25c9865

fail 44% strict pass 24 pass · 23 soft pass · 7 fail

Run

Engine

llamacpp a25c9865 (a25c9865fe03c954c93fd755b5d79ae86ba99750)

Run

2026-09-26T05:36:17Z to 2026-09-26T05:37:20Z

Platform

linux-x86_64, Python 3.12.14, canitoolcall 0.1.0.dev0

Strategies

one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8 (normalization soft-v1); not counted for this engine: one, special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

Fixtures digest

a1e7b55367e195e8aa6e2ef746b2cc141b44f37caad0887ff8f30b104616f836 the fixture corpus used for this site matches the run

Engine details

{
  "build_info": "b1-a25c986",
  "detokenizer": "common_token_to_piece(vocab-only GGUF, special = id in preserved_tokens)",
  "harness": "harnesses/llamacpp/replay.cpp",
  "harness_protocol": 1,
  "pinned_version": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "reasoning_format": "deepseek",
  "template_source": "gguf",
  "tokens_per_step": "one"
}

Results file

data/llamacpp-a25c9865.json

Checks

Per fixture, the worst result of each check over the non-streaming parse and every realistic chunking strategy.
Check	Pass	Soft	Fail	Strict pass rate
`expected_match`	24	23	6	45%
`expected_error`	0	0	1	0%
`stream_equals_nonstream`	49	0	5	91%
`no_leakage`	50	0	4	93%
`arguments_json`	42	0	5	89%
`arguments_schema`	42	0	5	89%
`parallel_order`	7	0	0	100%

7 fixture(s) failed only a synthetic strategy: char:* (which can split special tokens), or a multi-token strategy this engine never produces because its server streams one token per event (one, special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8). These are listed below but never count toward the status.

Fixtures needing attention

fail qwen3-xml/bug-coder-number-outside-int64 expected_match, stream_equals_nonstream, no_leakage

Checks that did not pass for qwen3-xml/bug-coder-number-outside-int64
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected None, got '<tool_call><function=calculate>\n<parameter=x>\n1e20\n</parameter>\n</function></tool_call>'; tool_calls: expected ['calculate'], got []
`expected_match`	`token`	fail	tool_calls: expected ['calculate'], got []
`stream_equals_nonstream`	`token`	fail	content: nonstream '<tool_call><function=calculate>\n<parameter=x>\n1e20\n</parameter>\n</function></tool_call>', got None
`no_leakage`	`nonstream`	fail	content contains '<tool_call>'; content contains '</tool_call>'; content contains '<function='; content contains '</function>'; content contains '<parameter='; content contains '</parameter>'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/bug-coder-number-outside-int64 --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 20 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream

--- expected
+++ observed
@@ -1,12 +1,5 @@
 {
-  "content": null,
+  "content": "<tool_call><function=calculate>\n<parameter=x>\n1e20\n</parameter>\n</function></tool_call>",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "x": 1e+20
-      },
-      "name": "calculate"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -2,11 +2,4 @@
   "content": null,
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "x": 1e+20
-      },
-      "name": "calculate"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: bug_report, https://github.com/ollama/ollama/issues/18421. Tags: single-call, numeric-arguments, regression.

Raw output

<tool_call><function=calculate>
<parameter=x>
1e20
</parameter>
</function></tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "x": 1e+20
      },
      "name": "calculate"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/bug-coder-number-outside-int64", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "bug_report", "source_url": "https://github.com/ollama/ollama/issues/18421", "revision": "issue-18421", "license": "NOASSERTION", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py"}, "tools": [{"type": "function", "function": {"name": "calculate", "parameters": {"type": "object", "properties": {"x": {"type": "number"}}}}}], "raw_output": "<tool_call><function=calculate>\n<parameter=x>\n1e20\n</parameter>\n</function></tool_call>", "output_token_ids": [151657, 27, 1688, 28, 35597, 397, 27, 16181, 25387, 397, 16, 68, 17, 15, 198, 522, 16181, 397, 522, 1688, 29, 151658], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": null, "reasoning_content": null, "tool_calls": [{"name": "calculate", "arguments": {"x": 1e+20}}]}, "tags": ["single-call", "numeric-arguments", "regression"], "notes": "The issue's parser input: a number outside the int64 range must be preserved (expected 100000000000000000000, not 9223372036854775807)."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "ad7e2ead4bc7d2dc214195ab5ecfc9624f0e9ee2d6ce7daa0377c146e8cfd204",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

fail qwen3-xml/bug-coder-text-after-call expected_match

Checks that did not pass for qwen3-xml/bug-coder-text-after-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected 'beforeafter', got 'before'; tool_calls: expected ['get_weather'], got []
`expected_match`	`token`	fail	content: expected 'beforeafter', got 'before'; tool_calls: expected ['get_weather'], got []

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/bug-coder-text-after-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 19 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,5 @@
 {
-  "content": "beforeafter",
+  "content": "before",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Paris"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: bug_report, https://github.com/sgl-project/sglang/issues/40739. Tags: single-call, text-before-call, text-after-call, regression, x-no-newlines.

Raw output

before<tool_call><function=get_weather><parameter=city>Paris</parameter></function></tool_call>after

Expected parse

{
  "content": "beforeafter",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Paris"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/bug-coder-text-after-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "bug_report", "source_url": "https://github.com/sgl-project/sglang/issues/40739", "revision": "issue-40739", "license": "NOASSERTION", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}], "raw_output": "before<tool_call><function=get_weather><parameter=city>Paris</parameter></function></tool_call>after", "output_token_ids": [14801, 151657, 27, 1688, 28280, 69364, 1784, 16181, 28, 8926, 29, 59604, 522, 16181, 1472, 1688, 29, 151658, 10694], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": "beforeafter", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "Paris"}}]}, "tags": ["single-call", "text-before-call", "text-after-call", "regression", "x-no-newlines"], "notes": "The issue's input string (no newlines inside the call). Its expected behaviour: streaming and non-streaming both return content 'beforeafter' and the call."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "8c10375e158a7b0d068fcd16631958e02f9870ad7865ba050db0a659fda5ce5c",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

fail qwen3-xml/bug-missing-close-parameter-before-function expected_match, stream_equals_nonstream, no_leakage, arguments_json, arguments_schema

Checks that did not pass for qwen3-xml/bug-missing-close-parameter-before-function
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected None, got '<tool_call>\n<function=ThinQ_Connect>\n<parameter=body>\n{"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}\n</function>\n</t...; tool_calls: expected ['ThinQ_Connect'], got []
`expected_match`	`token`	fail	tool_calls[0].arguments: expected '{"body": {"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}}', got '{"body":{"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}'
`stream_equals_nonstream`	`token`	fail	content: nonstream '<tool_call>\n<function=ThinQ_Connect>\n<parameter=body>\n{"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}\n</function>\n</t..., got None; tool_calls: nonstream [], got ['ThinQ_Connect']
`no_leakage`	`nonstream`	fail	content contains '<tool_call>'; content contains '</tool_call>'; content contains '<function='; content contains '</function>'; content contains '<parameter='
`arguments_json`	`token`	fail	[0] ThinQ_Connect: arguments are not valid JSON text ('{"body":{"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}': Expecting ',' delimiter: line 1 column 90 (char 89))
`arguments_schema`	`token`	fail	[0] ThinQ_Connect: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@one fail, stream_equals_nonstream@special fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/bug-missing-close-parameter-before-function --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 17 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream

--- expected
+++ observed
@@ -1,16 +1,5 @@
 {
-  "content": null,
+  "content": "<tool_call>\n<function=ThinQ_Connect>\n<parameter=body>\n{\"airConJobMode\": \"AIR_CLEAN\", \"windStrength\": \"HIGH\", \"monitoringEnabled\": true}\n</function>\n</tool_call>",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "body": {
-          "airConJobMode": "AIR_CLEAN",
-          "monitoringEnabled": true,
-          "windStrength": "HIGH"
-        }
-      },
-      "name": "ThinQ_Connect"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: one, special

--- expected
+++ observed
@@ -2,15 +2,4 @@
   "content": null,
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "body": {
-          "airConJobMode": "AIR_CLEAN",
-          "monitoringEnabled": true,
-          "windStrength": "HIGH"
-        }
-      },
-      "name": "ThinQ_Connect"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: token, rand:1:8, rand:3:8, rand:4:8

--- expected
+++ observed
@@ -5,9 +5,5 @@
     {
       "arguments": {
-        "body": {
-          "airConJobMode": "AIR_CLEAN",
-          "monitoringEnabled": true,
-          "windStrength": "HIGH"
-        }
+        "<arguments_raw, not valid JSON>": "{\"body\":{\"airConJobMode\": \"AIR_CLEAN\", \"windStrength\": \"HIGH\", \"monitoringEnabled\": true}"
       },
       "name": "ThinQ_Connect"

Strategies: rand:2:8

--- expected
+++ observed
@@ -5,9 +5,5 @@
     {
       "arguments": {
-        "body": {
-          "airConJobMode": "AIR_CLEAN",
-          "monitoringEnabled": true,
-          "windStrength": "HIGH"
-        }
+        "<arguments_raw, not valid JSON>": "{\"body\":{\"airConJobMode\": \"AIR_CLEAN\", \"windStrength\": \"HIGH\", \"monitoringEnabled\":"
       },
       "name": "ThinQ_Connect"

Strategies: rand:5:8

--- expected
+++ observed
@@ -5,9 +5,5 @@
     {
       "arguments": {
-        "body": {
-          "airConJobMode": "AIR_CLEAN",
-          "monitoringEnabled": true,
-          "windStrength": "HIGH"
-        }
+        "<arguments_raw, not valid JSON>": "{\"body\":{\"airConJobMode\": \"AIR_CLEAN\", \"windStrength\": \"HIGH\", \"monitor"
       },
       "name": "ThinQ_Connect"

Fixture

Provenance: bug_report, https://github.com/vllm-project/vllm/issues/57699. Tags: single-call, malformed, nested-json, regression, x-missing-close-param.

Raw output

<tool_call>
<function=ThinQ_Connect>
<parameter=body>
{"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "body": {
          "airConJobMode": "AIR_CLEAN",
          "monitoringEnabled": true,
          "windStrength": "HIGH"
        }
      },
      "name": "ThinQ_Connect"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/bug-missing-close-parameter-before-function", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B"], "spec_version": "0.1", "provenance": {"kind": "bug_report", "source_url": "https://github.com/vllm-project/vllm/issues/57699", "revision": "issue-57699", "license": "NOASSERTION", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py"}, "tools": [{"type": "function", "function": {"name": "ThinQ_Connect", "parameters": {"type": "object", "properties": {"body": {"type": "object"}}, "required": ["body"]}}}], "raw_output": "<tool_call>\n<function=ThinQ_Connect>\n<parameter=body>\n{\"airConJobMode\": \"AIR_CLEAN\", \"windStrength\": \"HIGH\", \"monitoringEnabled\": true}\n</function>\n</tool_call>", "output_token_ids": [248058, 198, 27, 1628, 28, 89882, 48, 14659, 2245, 29, 198, 27, 15704, 28, 2513, 29, 198, 4754, 1264, 1073, 11895, 3513, 763, 328, 55247, 76916, 487, 328, 18574, 51079, 763, 328, 87116, 487, 328, 31162, 286, 5290, 763, 804, 92, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n\n</think>\n\n", "thinking": false, "expected": {"content": null, "reasoning_content": null, "tool_calls": [{"name": "ThinQ_Connect", "arguments": {"body": {"airConJobMode": "AIR_CLEAN", "windStrength": "HIGH", "monitoringEnabled": true}}}]}, "tags": ["single-call", "malformed", "nested-json", "regression", "x-missing-close-param"], "notes": "Raw text from the issue (Qwen3.8-27B, BFCL live_simple_40-17-0, temperature 0). The last parameter is closed by </function> without </parameter>; the call is complete, so the value must be kept (vLLM returns {} non-streaming). The issue's tool schema is not shown; 'body' is typed as an object here. The thinking mode of the request is not stated; the fixture assumes the no-think prompt."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": false,
    "thinking": false
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n\n</think>\n\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n\n</think>\n\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "818bd21399ffe5a2621cc8e504436ea7ea060d39e74f176fa577e2385d93be11",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

fail qwen3-xml/coder-python-literals expected_match, stream_equals_nonstream, no_leakage, arguments_json, arguments_schema

Checks that did not pass for qwen3-xml/coder-python-literals
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected None, got '<tool_call>\n<function=set_alarm>\n<parameter=hour>\n9\n</parameter>\n<parameter=minute>\n15\n</parameter>\n<parameter=volume>\n0.5\n</parameter>\n<paramete...; tool_calls: expected ['set_alarm'], got []
`expected_match`	`token`	fail	tool_calls[0].arguments: expected '{"hour": 9, "minute": 15, "volume": 0.5, "repeat": false}', got '{"hour":9,"minute":15,"volume":0.5,"repeat":'
`stream_equals_nonstream`	`token`	fail	content: nonstream '<tool_call>\n<function=set_alarm>\n<parameter=hour>\n9\n</parameter>\n<parameter=minute>\n15\n</parameter>\n<parameter=volume>\n0.5\n</parameter>\n<paramete..., got None; tool_calls: nonstream [], got ['set_alarm']
`no_leakage`	`nonstream`	fail	content contains '<tool_call>'; content contains '</tool_call>'; content contains '<function='; content contains '</function>'; content contains '<parameter='; content contains '</parameter>'
`arguments_json`	`token`	fail	[0] set_alarm: arguments are not valid JSON text ('{"hour":9,"minute":15,"volume":0.5,"repeat":': Expecting value: line 1 column 45 (char 44))
`arguments_schema`	`token`	fail	[0] set_alarm: arguments are not a JSON object; not validated

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/coder-python-literals --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 30 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream

--- expected
+++ observed
@@ -1,15 +1,5 @@
 {
-  "content": null,
+  "content": "<tool_call>\n<function=set_alarm>\n<parameter=hour>\n9\n</parameter>\n<parameter=minute>\n15\n</parameter>\n<parameter=volume>\n0.5\n</parameter>\n<parameter=repeat>\nFalse\n</parameter>\n</function>\n</tool_call>",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "hour": 9,
-        "minute": 15,
-        "repeat": false,
-        "volume": 0.5
-      },
-      "name": "set_alarm"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: one, special

--- expected
+++ observed
@@ -2,14 +2,4 @@
   "content": null,
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "hour": 9,
-        "minute": 15,
-        "repeat": false,
-        "volume": 0.5
-      },
-      "name": "set_alarm"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: token, rand:3:8

--- expected
+++ observed
@@ -5,8 +5,5 @@
     {
       "arguments": {
-        "hour": 9,
-        "minute": 15,
-        "repeat": false,
-        "volume": 0.5
+        "<arguments_raw, not valid JSON>": "{\"hour\":9,\"minute\":15,\"volume\":0.5,\"repeat\":"
       },
       "name": "set_alarm"

Strategies: rand:1:8, rand:2:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -5,8 +5,5 @@
     {
       "arguments": {
-        "hour": 9,
-        "minute": 15,
-        "repeat": false,
-        "volume": 0.5
+        "<arguments_raw, not valid JSON>": "{\"hour\":9,\"minute\":15,\"volume\":0.5"
       },
       "name": "set_alarm"

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/b2cff646eb4bb1d68355c01b18ae02e7cf42d120/chat_template.jinja. Tags: single-call, numeric-arguments, x-python-literals, x-schema-coercion.

Raw output

<tool_call>
<function=set_alarm>
<parameter=hour>
9
</parameter>
<parameter=minute>
15
</parameter>
<parameter=volume>
0.5
</parameter>
<parameter=repeat>
False
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "hour": 9,
        "minute": 15,
        "repeat": false,
        "volume": 0.5
      },
      "name": "set_alarm"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/coder-python-literals", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/b2cff646eb4bb1d68355c01b18ae02e7cf42d120/chat_template.jinja", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "<tool_call>\n<function=set_alarm>\n<parameter=hour>\n9\n</parameter>\n<parameter=minute>\n15\n</parameter>\n<parameter=volume>\n0.5\n</parameter>\n<parameter=repeat>\nFalse\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [151657, 198, 27, 1688, 68800, 66286, 397, 27, 16181, 28, 24677, 397, 24, 198, 522, 16181, 397, 27, 16181, 28, 42557, 397, 16, 20, 198, 522, 16181, 397, 27, 16181, 28, 25060, 397, 15, 13, 20, 198, 522, 16181, 397, 27, 16181, 28, 30624, 397, 4049, 198, 522, 16181, 397, 522, 1688, 397, 151658], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": null, "reasoning_content": null, "tool_calls": [{"name": "set_alarm", "arguments": {"hour": 9, "minute": 15, "volume": 0.5, "repeat": false}}]}, "tags": ["single-call", "numeric-arguments", "x-python-literals", "x-schema-coercion"], "notes": "History render of the official Qwen/Qwen3-Coder-30B-A3B-Instruct chat template. The Qwen3-Coder-30B template writes Python literals ('False'); with a boolean schema the value is false."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

fail qwen3-xml/q35-python-literals expected_match, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for qwen3-xml/q35-python-literals
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	reasoning_content (whitespace only): expected 'Set a repeating alarm.', got 'Set a repeating alarm.\n'; tool_calls: expected ['set_alarm'], got []
`expected_match`	`token`	fail	parser raised 'LlamaCppError: Invalid diff: now finding less tool calls!\n Previous (1):\n - name: \'set_alarm\', args: \'{"hour":6,"minute":0,"repeat":\'\n Current (...
`stream_equals_nonstream`	`token`	fail	exception: nonstream None, got 'LlamaCppError: Invalid diff: now finding less tool calls!\n Previous (1):\n - name: \'set_alarm\', args: \'{"hour":6,"minute":0,"repeat":\'\n Current (...; tool_calls: nonstream [], got ['set_alarm']
`arguments_json`	`token`	fail	[0] set_alarm: arguments are not valid JSON text ('{"hour":6,"minute":0,"repeat":': Expecting value: line 1 column 31 (char 30))
`arguments_schema`	`token`	fail	[0] set_alarm: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q35-python-literals --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 24 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special

--- expected
+++ observed
@@ -1,15 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Set a repeating alarm.",
-  "tool_calls": [
-    {
-      "arguments": {
-        "hour": 6,
-        "label": "gym",
-        "minute": 0,
-        "repeat": true
-      },
-      "name": "set_alarm"
-    }
-  ]
+  "reasoning_content": "Set a repeating alarm.\n",
+  "tool_calls": []
 }

Strategies: token, rand:2:8, rand:4:8

--- expected
+++ observed
@@ -1,12 +1,10 @@
 {
   "content": null,
-  "reasoning_content": "Set a repeating alarm.",
+  "exception": "LlamaCppError: Invalid diff: now finding less tool calls!\n  Previous (1):\n    - name: 'set_alarm', args: '{\"hour\":6,\"minute\":0,\"repeat\":'\n  Current (0):\n  Current msg text content:\n\n",
+  "reasoning_content": "Set a repeating alarm.\n",
   "tool_calls": [
     {
       "arguments": {
-        "hour": 6,
-        "label": "gym",
-        "minute": 0,
-        "repeat": true
+        "<arguments_raw, not valid JSON>": "{\"hour\":6,\"minute\":0,\"repeat\":"
       },
       "name": "set_alarm"

Strategies: rand:1:8, rand:3:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,10 @@
 {
   "content": null,
-  "reasoning_content": "Set a repeating alarm.",
+  "exception": "LlamaCppError: Invalid diff: now finding less tool calls!\n  Previous (1):\n    - name: 'set_alarm', args: '{\"hour\":6,\"minute\":0'\n  Current (0):\n  Current msg text content:\n\n",
+  "reasoning_content": "Set a repeating alarm.\n",
   "tool_calls": [
     {
       "arguments": {
-        "hour": 6,
-        "label": "gym",
-        "minute": 0,
-        "repeat": true
+        "<arguments_raw, not valid JSON>": "{\"hour\":6,\"minute\":0"
       },
       "name": "set_alarm"

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.5-9B/blob/c202236235762e1c871ad0ccb60c8ee5ba337b9a/chat_template.jinja. Tags: single-call, reasoning, reasoning-prefilled, numeric-arguments, x-python-literals.

Raw output

Set a repeating alarm.
</think>

<tool_call>
<function=set_alarm>
<parameter=hour>
6
</parameter>
<parameter=minute>
0
</parameter>
<parameter=repeat>
True
</parameter>
<parameter=label>
gym
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Set a repeating alarm.",
  "tool_calls": [
    {
      "arguments": {
        "hour": 6,
        "label": "gym",
        "minute": 0,
        "repeat": true
      },
      "name": "set_alarm"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q35-python-literals", "family": "qwen3-xml", "models": ["Qwen/Qwen3.5-9B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.5-9B/blob/c202236235762e1c871ad0ccb60c8ee5ba337b9a/chat_template.jinja", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Set a repeating alarm.\n</think>\n\n<tool_call>\n<function=set_alarm>\n<parameter=hour>\n6\n</parameter>\n<parameter=minute>\n0\n</parameter>\n<parameter=repeat>\nTrue\n</parameter>\n<parameter=label>\ngym\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [1591, 264, 38495, 16136, 13, 198, 248069, 271, 248058, 198, 27, 1628, 66473, 64049, 29, 198, 27, 15704, 28, 23899, 29, 198, 21, 198, 510, 15704, 29, 198, 27, 15704, 28, 41143, 29, 198, 15, 198, 510, 15704, 29, 198, 27, 15704, 28, 29625, 29, 198, 2434, 198, 510, 15704, 29, 198, 27, 15704, 95277, 29, 198, 70, 1544, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.5-9B", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Set a repeating alarm.", "tool_calls": [{"name": "set_alarm", "arguments": {"hour": 6, "minute": 0, "repeat": true, "label": "gym"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "numeric-arguments", "x-python-literals"], "notes": "History render of the official Qwen/Qwen3.5-9B chat template. The Qwen3.5-9B template writes Python literals ('True'); with a boolean schema the value is true."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.5-9B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.5-9B"
    }
  },
  "template_identical_to": "models/templates/Qwen3.5-4B.jinja",
  "template_path": "Qwen--Qwen3.5-9B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.5-9B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "ccfbc41329c3caba178c629bf95b67eab472cee6f1a271ba69ce059056edd896"
}

fail qwen3-xml/q38-truncated-in-parameter expected_error, arguments_json, arguments_schema

Checks that did not pass for qwen3-xml/q38-truncated-in-parameter
Check	Strategy	Result	Detail
`expected_error`	`nonstream`	fail	returned 1 tool call(s) ['write_file'] for Output stopped (max_tokens) inside a parameter value, before </parameter> and </function>. The call is incomplete; a truncated 'rm -rf /tmp/build' must not become a complete 'rm -rf /' (vLLM #57699).
`expected_error`	`token`	fail	returned 1 tool call(s) ['write_file'] for Output stopped (max_tokens) inside a parameter value, before </parameter> and </function>. The call is incomplete; a truncated 'rm -rf /tmp/build' must not become a complete 'rm -rf /' (vLLM #57699).
`arguments_json`	`nonstream`	fail	[0] write_file: arguments are not valid JSON text ('{"path":"/tmp/build.sh","content":"rm -rf /': Unterminated string starting at: line 1 column 35 (char 34))
`arguments_json`	`token`	fail	[0] write_file: arguments are not valid JSON text ('{"path":"/tmp/build.sh","content":"rm -rf /': Unterminated string starting at: line 1 column 35 (char 34))
`arguments_schema`	`nonstream`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`token`	fail	[0] write_file: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_error@one fail, expected_error@special fail, expected_error@rand:1:8 fail, expected_error@rand:2:8 fail, expected_error@rand:3:8 fail, expected_error@rand:4:8 fail, expected_error@rand:5:8 fail, arguments_json@one fail, arguments_json@special fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@one fail, arguments_schema@special fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-truncated-in-parameter --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 22 of fixtures/qwen3-xml/rendered.jsonl.

Observed

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

{
  "content": null,
  "reasoning_content": "Write the script.\n",
  "tool_calls": [
    {
      "arguments": {
        "<arguments_raw, not valid JSON>": "{\"path\":\"/tmp/build.sh\",\"content\":\"rm -rf /"
      },
      "name": "write_file"
    }
  ]
}

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: truncated, reasoning, reasoning-prefilled.

Expected graceful failure: Output stopped (max_tokens) inside a parameter value, before </parameter> and </function>. The call is incomplete; a truncated 'rm -rf /tmp/build' must not become a complete 'rm -rf /' (vLLM #57699). (accept: no_tool_calls, content_passthrough, exception).

Raw output

Write the script.
</think>

<tool_call>
<function=write_file>
<parameter=path>
/tmp/build.sh
</parameter>
<parameter=content>
rm -rf /

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-truncated-in-parameter", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Write the script.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\n/tmp/build.sh\n</parameter>\n<parameter=content>\nrm -rf /", "output_token_ids": [7734, 279, 5148, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 55101, 29594, 2318, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 8454, 471, 7799, 593], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected_error": {"reason": "Output stopped (max_tokens) inside a parameter value, before </parameter> and </function>. The call is incomplete; a truncated 'rm -rf /tmp/build' must not become a complete 'rm -rf /' (vLLM #57699).", "accept": ["no_tool_calls", "content_passthrough", "exception"]}, "tags": ["truncated", "reasoning", "reasoning-prefilled"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": null,
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

fail qwen3-xml/vllm-coder-missing-close-parameter expected_match, stream_equals_nonstream, no_leakage, arguments_json, arguments_schema

Checks that did not pass for qwen3-xml/vllm-coder-missing-close-parameter
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content (whitespace only): expected 'Let me check the weather for you:', got 'Let me check the weather for you:\n'; tool_calls: expected ['get_current_weather'], got []
`expected_match`	`token`	fail	parser raised 'LlamaCppError: Invalid diff: now finding less tool calls!\n Previous (1):\n - name: \'get_current_weather\', args: \'{"city":"Dallas\\n<parameter=state>...
`stream_equals_nonstream`	`token`	fail	exception: nonstream None, got 'LlamaCppError: Invalid diff: now finding less tool calls!\n Previous (1):\n - name: \'get_current_weather\', args: \'{"city":"Dallas\\n<parameter=state>...; tool_calls: nonstream [], got ['get_current_weather']
`no_leakage`	`token`	fail	tool_calls[0].arguments contains '<parameter='
`arguments_json`	`token`	fail	[0] get_current_weather: arguments are not valid JSON text ('{"city":"Dallas\\n<parameter=state>\\nTX"': Expecting ',' delimiter: line 1 column 40 (char 39))
`arguments_schema`	`token`	fail	[0] get_current_weather: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, no_leakage@rand:1:8 fail, no_leakage@rand:2:8 fail, no_leakage@rand:3:8 fail, no_leakage@rand:4:8 fail, no_leakage@rand:5:8 fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/vllm-coder-missing-close-parameter --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 16 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special

--- expected
+++ observed
@@ -1,14 +1,5 @@
 {
-  "content": "Let me check the weather for you:",
+  "content": "Let me check the weather for you:\n",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Dallas",
-        "state": "TX",
-        "unit": "fahrenheit"
-      },
-      "name": "get_current_weather"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: token, rand:1:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,11 +1,10 @@
 {
-  "content": "Let me check the weather for you:",
+  "content": "Let me check the weather for you:\n",
+  "exception": "LlamaCppError: Invalid diff: now finding less tool calls!\n  Previous (1):\n    - name: 'get_current_weather', args: '{\"city\":\"Dallas\\n<parameter=state>\\nTX\"'\n  Current (0):\n  Current msg text content:\nLet me check the weather for you:\n\n",
   "reasoning_content": null,
   "tool_calls": [
     {
       "arguments": {
-        "city": "Dallas",
-        "state": "TX",
-        "unit": "fahrenheit"
+        "<arguments_raw, not valid JSON>": "{\"city\":\"Dallas\\n<parameter=state>\\nTX\""
       },
       "name": "get_current_weather"

Strategies: rand:2:8, rand:3:8

--- expected
+++ observed
@@ -1,11 +1,10 @@
 {
-  "content": "Let me check the weather for you:",
+  "content": "Let me check the weather for you:\n",
+  "exception": "LlamaCppError: Invalid diff: now finding less tool calls!\n  Previous (1):\n    - name: 'get_current_weather', args: '{\"city\":\"Dallas\\n<parameter=state>\\nTX'\n  Current (0):\n  Current msg text content:\nLet me check the weather for you:\n\n",
   "reasoning_content": null,
   "tool_calls": [
     {
       "arguments": {
-        "city": "Dallas",
-        "state": "TX",
-        "unit": "fahrenheit"
+        "<arguments_raw, not valid JSON>": "{\"city\":\"Dallas\\n<parameter=state>\\nTX"
       },
       "name": "get_current_weather"

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_qwen3coder_tool_parser.py#L932-L971. Tags: single-call, text-before-call, malformed, x-missing-close-param.

Raw output

Let me check the weather for you:
<tool_call>
<function=get_current_weather>
<parameter=city>
Dallas
<parameter=state>
TX
</parameter>
<parameter=unit>
fahrenheit
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": "Let me check the weather for you:",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Dallas",
        "state": "TX",
        "unit": "fahrenheit"
      },
      "name": "get_current_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/vllm-coder-missing-close-parameter", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_qwen3coder_tool_parser.py#L932-L971", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)."}, "tools": [{"type": "function", "function": {"name": "get_current_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The city name"}, "state": {"type": "string", "description": "The state code"}, "unit": {"type": "string", "enum": ["fahrenheit", "celsius"]}}, "required": ["city", "state"]}, "description": "Get the current weather"}}, {"type": "function", "function": {"name": "calculate_area", "parameters": {"type": "object", "properties": {"shape": {"type": "string"}, "dimensions": {"type": "object"}, "precision": {"type": "integer"}}}, "description": "Calculate area of a shape"}}], "raw_output": "Let me check the weather for you:\n<tool_call>\n<function=get_current_weather>\n<parameter=city>\nDallas\n<parameter=state>\nTX\n</parameter>\n<parameter=unit>\nfahrenheit\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [10061, 752, 1779, 279, 9104, 369, 498, 510, 151657, 198, 27, 1688, 28280, 11080, 69364, 397, 27, 16181, 28, 8926, 397, 86514, 198, 27, 16181, 28, 2454, 397, 22867, 198, 522, 16181, 397, 27, 16181, 28, 3843, 397, 69, 47910, 198, 522, 16181, 397, 522, 1688, 397, 151658], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": "Let me check the weather for you:", "reasoning_content": null, "tool_calls": [{"name": "get_current_weather", "arguments": {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}}]}, "tags": ["single-call", "text-before-call", "malformed", "x-missing-close-param"], "notes": "test_extract_tool_calls_missing_closing_parameter_tag: the next <parameter= ends the unclosed value."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "ee015a6c7f32366149ce1177702ae0096ccc7e120ba5a57ccd5bbcdeb06a23a1",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

soft pass qwen3-xml/bug-coder-missing-tool-call-opener expected_match

Checks that did not pass for qwen3-xml/bug-coder-missing-tool-call-opener
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	content (whitespace only): expected 'I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.', got 'I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.\n\n'
`expected_match`	`token`	soft pass	content (whitespace only): expected 'I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.', got 'I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.\n\n'

Synthetic strategies (not counted): expected_match@one soft pass, expected_match@special soft pass, expected_match@rand:1:8 soft pass, expected_match@rand:2:8 soft pass, expected_match@rand:3:8 soft pass, expected_match@rand:4:8 soft pass, expected_match@rand:5:8 soft pass

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/bug-coder-missing-tool-call-opener --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 18 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,4 +1,4 @@
 {
-  "content": "I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.",
+  "content": "I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.\n\n",
   "reasoning_content": null,
   "tool_calls": [

Fixture

Provenance: bug_report, https://github.com/ollama/ollama/issues/18530. Tags: single-call, text-before-call, malformed, regression, x-bare-function.

Raw output

I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.

<function=glob>
<parameter=pattern>
**/*.md
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": "I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "pattern": "**/*.md"
      },
      "name": "glob"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/bug-coder-missing-tool-call-opener", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "bug_report", "source_url": "https://github.com/ollama/ollama/issues/18530", "revision": "issue-18530", "license": "NOASSERTION", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py"}, "tools": [{"type": "function", "function": {"name": "glob", "parameters": {"type": "object", "properties": {"pattern": {"type": "string"}}, "required": ["pattern"]}, "description": "Find files by glob pattern"}}], "raw_output": "I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.\n\n<function=glob>\n<parameter=pattern>\n**/*.md\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [40, 686, 1156, 1779, 369, 678, 73192, 3542, 304, 279, 1482, 6220, 323, 1181, 1186, 75701, 11, 323, 1221, 1140, 1105, 1667, 279, 13206, 5392, 382, 27, 1688, 37162, 1684, 397, 27, 16181, 28, 14339, 397, 334, 23540, 2277, 198, 522, 16181, 397, 522, 1688, 397, 151658], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": "I will first check for all Markdown files in the current directory and its subdirectories, and then list them using the glob tool.", "reasoning_content": null, "tool_calls": [{"name": "glob", "arguments": {"pattern": "**/*.md"}}]}, "tags": ["single-call", "text-before-call", "malformed", "regression", "x-bare-function"], "notes": "A real qwen3-coder:30b generation captured by the reporter with raw: true (temperature 0, seed 1): the model omitted the <tool_call> opener after a prose preamble."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "6f073c8c330018b211e4a3db4e0060c073b970cea8fbd0adc8bb4874740e8c9f",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

soft pass qwen3-xml/coder-text-before-call expected_match

Checks that did not pass for qwen3-xml/coder-text-before-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	content (whitespace only): expected "I'll create the file now.", got "I'll create the file now.\n\n"
`expected_match`	`token`	soft pass	content (whitespace only): expected "I'll create the file now.", got "I'll create the file now.\n\n"

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/coder-text-before-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 28 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,4 +1,4 @@
 {
-  "content": "I'll create the file now.",
+  "content": "I'll create the file now.\n\n",
   "reasoning_content": null,
   "tool_calls": [

Fixture

Raw output

I'll create the file now.

<tool_call>
<function=write_file>
<parameter=path>
hello.py
</parameter>
<parameter=content>
def greet(name: str) -> str:
    """Say hi."""
    return f"Hello, {name}!"


print(greet("Zoë"))
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": "I'll create the file now.",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "content": "def greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))",
        "path": "hello.py"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/coder-text-before-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3-Coder-30B-A3B-Instruct", "Qwen/Qwen3-Coder-480B-A35B-Instruct"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/b2cff646eb4bb1d68355c01b18ae02e7cf42d120/chat_template.jinja", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "I'll create the file now.\n\n<tool_call>\n<function=write_file>\n<parameter=path>\nhello.py\n</parameter>\n<parameter=content>\ndef greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [40, 3278, 1855, 279, 1034, 1431, 382, 151657, 198, 27, 1688, 28, 4934, 2458, 397, 27, 16181, 81940, 397, 14990, 7197, 198, 522, 16181, 397, 27, 16181, 28, 1796, 397, 750, 40786, 3153, 25, 607, 8, 1464, 607, 510, 262, 4210, 45764, 15588, 14442, 262, 470, 282, 1, 9707, 11, 314, 606, 92, 8958, 1406, 1350, 3268, 3744, 445, 57, 78, 12179, 5455, 522, 16181, 397, 522, 1688, 397, 151658], "tokenizer": {"repo": "Qwen/Qwen3-Coder-30B-A3B-Instruct", "revision": "b2cff646eb4bb1d68355c01b18ae02e7cf42d120", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n", "expected": {"content": "I'll create the file now.", "reasoning_content": null, "tool_calls": [{"name": "write_file", "arguments": {"path": "hello.py", "content": "def greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))"}}]}, "tags": ["single-call", "text-before-call", "x-multiline-string"], "notes": "History render of the official Qwen/Qwen3-Coder-30B-A3B-Instruct chat template. The Qwen3-Coder template writes '\\n' + content + '\\n' and then '\\n<tool_call>', so the content is followed by '\\n\\n' of markup."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n",
  "fixture_tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n",
  "model": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3-Coder-30B-A3B-Instruct"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "5a38bfa05833266240066aedc497decc9b00cc0d3e3b8cceea98cf530196ab06",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": false,
  "template_variant": "default",
  "thinking_end_tags": [],
  "thinking_start_tag": "",
  "tokenizer": "Qwen/Qwen3-Coder-30B-A3B-Instruct@b2cff646eb4bb1d68355c01b18ae02e7cf42d120",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3-Coder-30B-A3B-Instruct.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "835d0d2ad5a7f95016cdf13dffb330e8495936aac89fa4edf5e103c424ce9849"
}

soft pass qwen3-xml/llamacpp-q35-call-inside-think expected_match

Checks that did not pass for qwen3-xml/llamacpp-q35-call-inside-think
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Need to inspect the current directory.', got 'Need to inspect the current directory.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Need to inspect the current directory.', got 'Need to inspect the current directory.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/llamacpp-q35-call-inside-think --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 6 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Need to inspect the current directory.",
+  "reasoning_content": "Need to inspect the current directory.\n",
   "tool_calls": [
     {

Fixture

Provenance: engine_test, https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2323-L2340. Tags: single-call, reasoning, reasoning-prefilled, malformed, x-call-inside-think.

Raw output

Need to inspect the current directory.
<tool_call>
<function=run_in_terminal>
<parameter=command>
pwd
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Need to inspect the current directory.",
  "tool_calls": [
    {
      "arguments": {
        "command": "pwd"
      },
      "name": "run_in_terminal"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/llamacpp-q35-call-inside-think", "family": "qwen3-xml", "models": ["Qwen/Qwen3.5-9B"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2323-L2340", "revision": "7fe450e19305b828c199d602c23a8337aaa1f03b", "license": "MIT", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py", "attribution": "Copyright (c) 2023-2026 The ggml authors (MIT)."}, "tools": [{"type": "function", "function": {"name": "run_in_terminal", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "Shell command to run"}}, "required": ["command"]}, "description": "Run a shell command."}}], "raw_output": "Need to inspect the current directory.\n<tool_call>\n<function=run_in_terminal>\n<parameter=command>\npwd\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [22916, 310, 23331, 279, 1428, 6025, 13, 198, 248058, 198, 27, 1628, 28, 5917, 1201, 62138, 29, 198, 27, 15704, 28, 5454, 29, 198, 24751, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.5-9B", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Need to inspect the current directory.", "tool_calls": [{"name": "run_in_terminal", "arguments": {"command": "pwd"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "malformed", "x-call-inside-think"], "notes": "The model starts a tool call without closing the prefilled <think> block. llama.cpp's test ('a tool call ends the prefilled thinking block, with or without a closing </think>') treats <tool_call> as ending the reasoning. This is the most-reported failure for the family: https://github.com/ggml-org/llama.cpp/issues/20837, https://github.com/vllm-project/vllm/issues/39056."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.5-9B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.5-9B"
    }
  },
  "template_identical_to": "models/templates/Qwen3.5-4B.jinja",
  "template_path": "Qwen--Qwen3.5-9B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "bd287b2678df40687c4e3eb428a6030d96d519022ec29a850bb567c177ac3457",
  "vocab_gguf": "Qwen--Qwen3.5-9B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "ccfbc41329c3caba178c629bf95b67eab472cee6f1a271ba69ce059056edd896"
}

soft pass qwen3-xml/llamacpp-q35-reasoning-content-call expected_match

Checks that did not pass for qwen3-xml/llamacpp-q35-reasoning-content-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	content (whitespace only): expected 'Let me inspect it now.', got 'Let me inspect it now.\n'; reasoning_content (whitespace only): expected 'I should inspect the directory.', got 'I should inspect the directory.\n'
`expected_match`	`token`	soft pass	content (whitespace only): expected 'Let me inspect it now.', got 'Let me inspect it now.\n'; reasoning_content (whitespace only): expected 'I should inspect the directory.', got 'I should inspect the directory.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/llamacpp-q35-reasoning-content-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 8 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
-  "content": "Let me inspect it now.",
-  "reasoning_content": "I should inspect the directory.",
+  "content": "Let me inspect it now.\n",
+  "reasoning_content": "I should inspect the directory.\n",
   "tool_calls": [
     {

Fixture

Provenance: engine_test, https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2459-L2480. Tags: single-call, reasoning, reasoning-prefilled, text-before-call.

Raw output

I should inspect the directory.
</think>

Let me inspect it now.
<tool_call>
<function=run_in_terminal>
<parameter=command>
pwd
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": "Let me inspect it now.",
  "reasoning_content": "I should inspect the directory.",
  "tool_calls": [
    {
      "arguments": {
        "command": "pwd"
      },
      "name": "run_in_terminal"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/llamacpp-q35-reasoning-content-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.5-9B"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2459-L2480", "revision": "7fe450e19305b828c199d602c23a8337aaa1f03b", "license": "MIT", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py", "attribution": "Copyright (c) 2023-2026 The ggml authors (MIT)."}, "tools": [{"type": "function", "function": {"name": "run_in_terminal", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "Shell command to run"}}, "required": ["command"]}, "description": "Run a shell command."}}], "raw_output": "I should inspect the directory.\n</think>\n\nLet me inspect it now.\n<tool_call>\n<function=run_in_terminal>\n<parameter=command>\npwd\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [40, 1220, 23331, 279, 6025, 13, 198, 248069, 271, 9764, 728, 23331, 424, 1381, 13, 198, 248058, 198, 27, 1628, 28, 5917, 1201, 62138, 29, 198, 27, 15704, 28, 5454, 29, 198, 24751, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.5-9B", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": "Let me inspect it now.", "reasoning_content": "I should inspect the directory.", "tool_calls": [{"name": "run_in_terminal", "arguments": {"command": "pwd"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "text-before-call"]}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.5-9B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.5-9B"
    }
  },
  "template_identical_to": "models/templates/Qwen3.5-4B.jinja",
  "template_path": "Qwen--Qwen3.5-9B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "bd287b2678df40687c4e3eb428a6030d96d519022ec29a850bb567c177ac3457",
  "vocab_gguf": "Qwen--Qwen3.5-9B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "ccfbc41329c3caba178c629bf95b67eab472cee6f1a271ba69ce059056edd896"
}

soft pass qwen3-xml/llamacpp-q35-reasoning-then-call expected_match

Checks that did not pass for qwen3-xml/llamacpp-q35-reasoning-then-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected "I'm\nthinking", got "I'm\nthinking\n"
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected "I'm\nthinking", got "I'm\nthinking\n"

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/imported.jsonl --id qwen3-xml/llamacpp-q35-reasoning-then-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 1 of fixtures/qwen3-xml/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "I'm\nthinking",
+  "reasoning_content": "I'm\nthinking\n",
   "tool_calls": [
     {

Fixture

Provenance: engine_test, https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2164-L2174. Tags: single-call, reasoning, reasoning-prefilled, numeric-arguments.

Raw output

I'm
thinking
</think>

<tool_call>
<function=special_function>
<parameter=arg1>
1
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "I'm\nthinking",
  "tool_calls": [
    {
      "arguments": {
        "arg1": 1
      },
      "name": "special_function"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/llamacpp-q35-reasoning-then-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.5-9B"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/ggml-org/llama.cpp/blob/7fe450e19305b828c199d602c23a8337aaa1f03b/tests/test-chat.cpp#L2164-L2174", "revision": "7fe450e19305b828c199d602c23a8337aaa1f03b", "license": "MIT", "generator": "scripts/fixtures/qwen3-xml/import_qwen3_xml.py", "attribution": "Copyright (c) 2023-2026 The ggml authors (MIT)."}, "tools": [{"type": "function", "function": {"name": "special_function", "parameters": {"type": "object", "properties": {"arg1": {"type": "integer", "description": "The arg."}}, "required": ["arg1"]}, "description": "I'm special"}}], "raw_output": "I'm\nthinking\n</think>\n\n<tool_call>\n<function=special_function>\n<parameter=arg1>\n1\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [40, 2688, 198, 79420, 198, 248069, 271, 248058, 198, 27, 1628, 13744, 2869, 8901, 29, 198, 27, 15704, 28, 832, 16, 29, 198, 16, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.5-9B", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "I'm\nthinking", "tool_calls": [{"name": "special_function", "arguments": {"arg1": 1}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "numeric-arguments"]}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.5-9B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.5-9B"
    }
  },
  "template_identical_to": "models/templates/Qwen3.5-4B.jinja",
  "template_path": "Qwen--Qwen3.5-9B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "53278a826a08c3c0291496ff5dd3fb1560dce4e804a079722171783ccb2c40ed",
  "vocab_gguf": "Qwen--Qwen3.5-9B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "ccfbc41329c3caba178c629bf95b67eab472cee6f1a271ba69ce059056edd896"
}

soft pass qwen3-xml/q35-single-call expected_match

Checks that did not pass for qwen3-xml/q35-single-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Weather lookup.', got 'Weather lookup.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Weather lookup.', got 'Weather lookup.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q35-single-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 25 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Weather lookup.",
+  "reasoning_content": "Weather lookup.\n",
   "tool_calls": [
     {

Fixture

Raw output

Weather lookup.
</think>

<tool_call>
<function=get_weather>
<parameter=city>
Zürich
</parameter>
<parameter=unit>
c
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Weather lookup.",
  "tool_calls": [
    {
      "arguments": {
        "city": "Zürich",
        "unit": "c"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q35-single-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.5-9B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.5-9B/blob/c202236235762e1c871ad0ccb60c8ee5ba337b9a/chat_template.jinja", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Weather lookup.\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nZürich\n</parameter>\n<parameter=unit>\nc\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [28034, 18054, 13, 198, 248069, 271, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 57, 5021, 693, 198, 510, 15704, 29, 198, 27, 15704, 28, 3715, 29, 198, 66, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.5-9B", "revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Weather lookup.", "tool_calls": [{"name": "get_weather", "arguments": {"city": "Zürich", "unit": "c"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "unicode"], "notes": "History render of the official Qwen/Qwen3.5-9B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.5-9B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.5-9B"
    }
  },
  "template_identical_to": "models/templates/Qwen3.5-4B.jinja",
  "template_path": "Qwen--Qwen3.5-9B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.5-9B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "ccfbc41329c3caba178c629bf95b67eab472cee6f1a271ba69ce059056edd896"
}

soft pass qwen3-xml/q38-empty-arguments expected_match

Checks that did not pass for qwen3-xml/q38-empty-arguments
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Get the time.', got 'Get the time.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Get the time.', got 'Get the time.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-empty-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 6 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Get the time.",
+  "reasoning_content": "Get the time.\n",
   "tool_calls": [
     {

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: single-call, reasoning, reasoning-prefilled, empty-arguments.

Raw output

Get the time.
</think>

<tool_call>
<function=get_time>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Get the time.",
  "tool_calls": [
    {
      "arguments": {},
      "name": "get_time"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-empty-arguments", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Get the time.\n</think>\n\n<tool_call>\n<function=get_time>\n</function>\n</tool_call>", "output_token_ids": [1882, 279, 854, 13, 198, 248069, 271, 248058, 198, 27, 1628, 27362, 2914, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Get the time.", "tool_calls": [{"name": "get_time", "arguments": {}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "empty-arguments"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-long-arguments expected_match

Checks that did not pass for qwen3-xml/q38-long-arguments
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Write the module.', got 'Write the module.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Write the module.', got 'Write the module.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-long-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 16 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Write the module.",
+  "reasoning_content": "Write the module.\n",
   "tool_calls": [
     {

Fixture

Raw output

Write the module.
</think>

<tool_call>
<function=write_file>
<parameter=path>
steps.py
</parameter>
<parameter=content>
"""Generated module used as a long tool argument."""

from __future__ import annotations

def step_00(x: int) -> int:
    return x + 0

def step_01(x: int) -> int:
    return x + 1

def step_02(x: int) -> int:
    return x + 2

def step_03(x: int) -> int:
    return x + 3

def step_04(x: int) -> int:
    return x + 4

def step_05(x: int) -> int:
    return x + 5

def step_06(x: int) -> int:
    return x + 6

def step_07(x: int) -> int:
    return x + 7

def step_08(x: int) -> int:
    return x + 8

def step_09(x: int) -> int:
    return x + 9

def step_10(x: int) -> int:
    return x + 10

def step_11(x: int) -> int:
    return x + 11

def step_12(x: int) -> int:
    return x + 12

def step_13(x: int) -> int:
    return x + 13

def step_14(x: int) -> int:
    return x + 14

def step_15(x: int) -> int:
    return x + 15

def step_16(x: int) -> int:
    return x + 16

def step_17(x: int) -> int:
    return x + 17

def step_18(x: int) -> int:
    return x + 18

def step_19(x: int) -> int:
    return x + 19

def step_20(x: int) -> int:
    return x + 20

def step_21(x: int) -> int:
    return x + 21

def step_22(x: int) -> int:
    return x + 22

def step_23(x: int) -> int:
    return x + 23

def step_24(x: int) -> int:
    return x + 24

def step_25(x: int) -> int:
    return x + 25

def step_26(x: int) -> int:
    return x + 26

def step_27(x: int) -> int:
    return x + 27

def step_28(x: int) -> int:
    return x + 28

def step_29(x: int) -> int:
    return x + 29

def step_30(x: int) -> int:
    return x + 30

def step_31(x: int) -> int:
    return x + 31

def step_32(x: int) -> int:
    return x + 32

def step_33(x: int) -> int:
    return x + 33

def step_34(x: int) -> int:
    return x + 34

def step_35(x: int) -> int:
    return x + 35

def step_36(x: int) -> int:
    return x + 36

def step_37(x: int) -> int:
    return x + 37

def step_38(x: int) -> int:
    return x + 38

def step_39(x: int) -> int:
    return x + 39

def step_40(x: int) -> int:
    return x + 40

def step_41(x: int) -> int:
    return x + 41

def step_42(x: int) -> int:
    return x + 42

def step_43(x: int) -> int:
    return x + 43

def step_44(x: int) -> int:
    return x + 44

def step_45(x: int) -> int:
    return x + 45

def step_46(x: int) -> int:
    return x + 46

def step_47(x: int) -> int:
    return x + 47

def step_48(x: int) -> int:
    return x + 48

def step_49(x: int) -> int:
    return x + 49

def step_50(x: int) -> int:
    return x + 50

def step_51(x: int) -> int:
    return x + 51

def step_52(x: int) -> int:
    return x + 52

def step_53(x: int) -> int:
    return x + 53

def step_54(x: int) -> int:
    return x + 54

def step_55(x: int) -> int:
    return x + 55

def step_56(x: int) -> int:
    return x + 56

def step_57(x: int) -> int:
    return x + 57

def step_58(x: int) -> int:
    return x + 58

def step_59(x: int) -> int:
    return x + 59

def step_60(x: int) -> int:
    return x + 60

def step_61(x: int) -> int:
    return x + 61

def step_62(x: int) -> int:
    return x + 62

def step_63(x: int) -> int:
    return x + 63

def step_64(x: int) -> int:
    return x + 64

def step_65(x: int) -> int:
    return x + 65

def step_66(x: int) -> int:
    return x + 66

def step_67(x: int) -> int:
    return x + 67

def step_68(x: int) -> int:
    return x + 68

def step_69(x: int) -> int:
    return x + 69

def step_70(x: int) -> int:
    return x + 70

def step_71(x: int) -> int:
    return x + 71

def step_72(x: int) -> int:
    return x + 72

def step_73(x: int) -> int:
    return x + 73

def step_74(x: int) -> int:
    return x + 74

def step_75(x: int) -> int:
    return x + 75

def step_76(x: int) -> int:
    return x + 76

def step_77(x: int) -> int:
    return x + 77

def step_78(x: int) -> int:
    return x + 78

def step_79(x: int) -> int:
    return x + 79

</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the module.",
  "tool_calls": [
    {
      "arguments": {
        "content": "\"\"\"Generated module used as a long tool argument.\"\"\"\n\nfrom __future__ import annotations\n\ndef step_00(x: int) -> int:\n    return x + 0\n\ndef step_01(x: int) -> int:\n    return x + 1\n\ndef step_02(x: int) -> int:\n    return x + 2\n\ndef step_03(x: int) -> int:\n    return x + 3\n\ndef step_04(x: int) -> int:\n    return x + 4\n\ndef step_05(x: int) -> int:\n    return x + 5\n\ndef step_06(x: int) -> int:\n    return x + 6\n\ndef step_07(x: int) -> int:\n    return x + 7\n\ndef step_08(x: int) -> int:\n    return x + 8\n\ndef step_09(x: int) -> int:\n    return x + 9\n\ndef step_10(x: int) -> int:\n    return x + 10\n\ndef step_11(x: int) -> int:\n    return x + 11\n\ndef step_12(x: int) -> int:\n    return x + 12\n\ndef step_13(x: int) -> int:\n    return x + 13\n\ndef step_14(x: int) -> int:\n    return x + 14\n\ndef step_15(x: int) -> int:\n    return x + 15\n\ndef step_16(x: int) -> int:\n    return x + 16\n\ndef step_17(x: int) -> int:\n    return x + 17\n\ndef step_18(x: int) -> int:\n    return x + 18\n\ndef step_19(x: int) -> int:\n    return x + 19\n\ndef step_20(x: int) -> int:\n    return x + 20\n\ndef step_21(x: int) -> int:\n    return x + 21\n\ndef step_22(x: int) -> int:\n    return x + 22\n\ndef step_23(x: int) -> int:\n    return x + 23\n\ndef step_24(x: int) -> int:\n    return x + 24\n\ndef step_25(x: int) -> int:\n    return x + 25\n\ndef step_26(x: int) -> int:\n    return x + 26\n\ndef step_27(x: int) -> int:\n    return x + 27\n\ndef step_28(x: int) -> int:\n    return x + 28\n\ndef step_29(x: int) -> int:\n    return x + 29\n\ndef step_30(x: int) -> int:\n    return x + 30\n\ndef step_31(x: int) -> int:\n    return x + 31\n\ndef step_32(x: int) -> int:\n    return x + 32\n\ndef step_33(x: int) -> int:\n    return x + 33\n\ndef step_34(x: int) -> int:\n    return x + 34\n\ndef step_35(x: int) -> int:\n    return x + 35\n\ndef step_36(x: int) -> int:\n    return x + 36\n\ndef step_37(x: int) -> int:\n    return x + 37\n\ndef step_38(x: int) -> int:\n    return x + 38\n\ndef step_39(x: int) -> int:\n    return x + 39\n\ndef step_40(x: int) -> int:\n    return x + 40\n\ndef step_41(x: int) -> int:\n    return x + 41\n\ndef step_42(x: int) -> int:\n    return x + 42\n\ndef step_43(x: int) -> int:\n    return x + 43\n\ndef step_44(x: int) -> int:\n    return x + 44\n\ndef step_45(x: int) -> int:\n    return x + 45\n\ndef step_46(x: int) -> int:\n    return x + 46\n\ndef step_47(x: int) -> int:\n    return x + 47\n\ndef step_48(x: int) -> int:\n    return x + 48\n\ndef step_49(x: int) -> int:\n    return x + 49\n\ndef step_50(x: int) -> int:\n    return x + 50\n\ndef step_51(x: int) -> int:\n    return x + 51\n\ndef step_52(x: int) -> int:\n    return x + 52\n\ndef step_53(x: int) -> int:\n    return x + 53\n\ndef step_54(x: int) -> int:\n    return x + 54\n\ndef step_55(x: int) -> int:\n    return x + 55\n\ndef step_56(x: int) -> int:\n    return x + 56\n\ndef step_57(x: int) -> int:\n    return x + 57\n\ndef step_58(x: int) -> int:\n    return x + 58\n\ndef step_59(x: int) -> int:\n    return x + 59\n\ndef step_60(x: int) -> int:\n    return x + 60\n\ndef step_61(x: int) -> int:\n    return x + 61\n\ndef step_62(x: int) -> int:\n    return x + 62\n\ndef step_63(x: int) -> int:\n    return x + 63\n\ndef step_64(x: int) -> int:\n    return x + 64\n\ndef step_65(x: int) -> int:\n    return x + 65\n\ndef step_66(x: int) -> int:\n    return x + 66\n\ndef step_67(x: int) -> int:\n    return x + 67\n\ndef step_68(x: int) -> int:\n    return x + 68\n\ndef step_69(x: int) -> int:\n    return x + 69\n\ndef step_70(x: int) -> int:\n    return x + 70\n\ndef step_71(x: int) -> int:\n    return x + 71\n\ndef step_72(x: int) -> int:\n    return x + 72\n\ndef step_73(x: int) -> int:\n    return x + 73\n\ndef step_74(x: int) -> int:\n    return x + 74\n\ndef step_75(x: int) -> int:\n    return x + 75\n\ndef step_76(x: int) -> int:\n    return x + 76\n\ndef step_77(x: int) -> int:\n    return x + 77\n\ndef step_78(x: int) -> int:\n    return x + 78\n\ndef step_79(x: int) -> int:\n    return x + 79\n",
        "path": "steps.py"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-long-arguments", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Write the module.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\nsteps.py\n</parameter>\n<parameter=content>\n\"\"\"Generated module used as a long tool argument.\"\"\"\n\nfrom __future__ import annotations\n\ndef step_00(x: int) -> int:\n    return x + 0\n\ndef step_01(x: int) -> int:\n    return x + 1\n\ndef step_02(x: int) -> int:\n    return x + 2\n\ndef step_03(x: int) -> int:\n    return x + 3\n\ndef step_04(x: int) -> int:\n    return x + 4\n\ndef step_05(x: int) -> int:\n    return x + 5\n\ndef step_06(x: int) -> int:\n    return x + 6\n\ndef step_07(x: int) -> int:\n    return x + 7\n\ndef step_08(x: int) -> int:\n    return x + 8\n\ndef step_09(x: int) -> int:\n    return x + 9\n\ndef step_10(x: int) -> int:\n    return x + 10\n\ndef step_11(x: int) -> int:\n    return x + 11\n\ndef step_12(x: int) -> int:\n    return x + 12\n\ndef step_13(x: int) -> int:\n    return x + 13\n\ndef step_14(x: int) -> int:\n    return x + 14\n\ndef step_15(x: int) -> int:\n    return x + 15\n\ndef step_16(x: int) -> int:\n    return x + 16\n\ndef step_17(x: int) -> int:\n    return x + 17\n\ndef step_18(x: int) -> int:\n    return x + 18\n\ndef step_19(x: int) -> int:\n    return x + 19\n\ndef step_20(x: int) -> int:\n    return x + 20\n\ndef step_21(x: int) -> int:\n    return x + 21\n\ndef step_22(x: int) -> int:\n    return x + 22\n\ndef step_23(x: int) -> int:\n    return x + 23\n\ndef step_24(x: int) -> int:\n    return x + 24\n\ndef step_25(x: int) -> int:\n    return x + 25\n\ndef step_26(x: int) -> int:\n    return x + 26\n\ndef step_27(x: int) -> int:\n    return x + 27\n\ndef step_28(x: int) -> int:\n    return x + 28\n\ndef step_29(x: int) -> int:\n    return x + 29\n\ndef step_30(x: int) -> int:\n    return x + 30\n\ndef step_31(x: int) -> int:\n    return x + 31\n\ndef step_32(x: int) -> int:\n    return x + 32\n\ndef step_33(x: int) -> int:\n    return x + 33\n\ndef step_34(x: int) -> int:\n    return x + 34\n\ndef step_35(x: int) -> int:\n    return x + 35\n\ndef step_36(x: int) -> int:\n    return x + 36\n\ndef step_37(x: int) -> int:\n    return x + 37\n\ndef step_38(x: int) -> int:\n    return x + 38\n\ndef step_39(x: int) -> int:\n    return x + 39\n\ndef step_40(x: int) -> int:\n    return x + 40\n\ndef step_41(x: int) -> int:\n    return x + 41\n\ndef step_42(x: int) -> int:\n    return x + 42\n\ndef step_43(x: int) -> int:\n    return x + 43\n\ndef step_44(x: int) -> int:\n    return x + 44\n\ndef step_45(x: int) -> int:\n    return x + 45\n\ndef step_46(x: int) -> int:\n    return x + 46\n\ndef step_47(x: int) -> int:\n    return x + 47\n\ndef step_48(x: int) -> int:\n    return x + 48\n\ndef step_49(x: int) -> int:\n    return x + 49\n\ndef step_50(x: int) -> int:\n    return x + 50\n\ndef step_51(x: int) -> int:\n    return x + 51\n\ndef step_52(x: int) -> int:\n    return x + 52\n\ndef step_53(x: int) -> int:\n    return x + 53\n\ndef step_54(x: int) -> int:\n    return x + 54\n\ndef step_55(x: int) -> int:\n    return x + 55\n\ndef step_56(x: int) -> int:\n    return x + 56\n\ndef step_57(x: int) -> int:\n    return x + 57\n\ndef step_58(x: int) -> int:\n    return x + 58\n\ndef step_59(x: int) -> int:\n    return x + 59\n\ndef step_60(x: int) -> int:\n    return x + 60\n\ndef step_61(x: int) -> int:\n    return x + 61\n\ndef step_62(x: int) -> int:\n    return x + 62\n\ndef step_63(x: int) -> int:\n    return x + 63\n\ndef step_64(x: int) -> int:\n    return x + 64\n\ndef step_65(x: int) -> int:\n    return x + 65\n\ndef step_66(x: int) -> int:\n    return x + 66\n\ndef step_67(x: int) -> int:\n    return x + 67\n\ndef step_68(x: int) -> int:\n    return x + 68\n\ndef step_69(x: int) -> int:\n    return x + 69\n\ndef step_70(x: int) -> int:\n    return x + 70\n\ndef step_71(x: int) -> int:\n    return x + 71\n\ndef step_72(x: int) -> int:\n    return x + 72\n\ndef step_73(x: int) -> int:\n    return x + 73\n\ndef step_74(x: int) -> int:\n    return x + 74\n\ndef step_75(x: int) -> int:\n    return x + 75\n\ndef step_76(x: int) -> int:\n    return x + 76\n\ndef step_77(x: int) -> int:\n    return x + 77\n\ndef step_78(x: int) -> int:\n    return x + 78\n\ndef step_79(x: int) -> int:\n    return x + 79\n\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [7734, 279, 4536, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 23326, 6971, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 12237, 15280, 4536, 1429, 430, 264, 1248, 5224, 5515, 71483, 271, 1445, 1259, 20409, 548, 1120, 31144, 271, 727, 2923, 62, 15, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 15, 271, 727, 2923, 62, 15, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 271, 727, 2923, 62, 15, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 271, 727, 2923, 62, 15, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 271, 727, 2923, 62, 15, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 271, 727, 2923, 62, 15, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 271, 727, 2923, 62, 15, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 271, 727, 2923, 62, 15, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 271, 727, 2923, 62, 15, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 23, 271, 727, 2923, 62, 15, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 24, 271, 727, 2923, 62, 16, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 15, 271, 727, 2923, 62, 16, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 16, 271, 727, 2923, 62, 16, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 17, 271, 727, 2923, 62, 16, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 18, 271, 727, 2923, 62, 16, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 19, 271, 727, 2923, 62, 16, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 20, 271, 727, 2923, 62, 16, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 21, 271, 727, 2923, 62, 16, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 22, 271, 727, 2923, 62, 16, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 23, 271, 727, 2923, 62, 16, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 16, 24, 271, 727, 2923, 62, 17, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 15, 271, 727, 2923, 62, 17, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 16, 271, 727, 2923, 62, 17, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 17, 271, 727, 2923, 62, 17, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 18, 271, 727, 2923, 62, 17, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 19, 271, 727, 2923, 62, 17, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 20, 271, 727, 2923, 62, 17, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 21, 271, 727, 2923, 62, 17, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 22, 271, 727, 2923, 62, 17, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 23, 271, 727, 2923, 62, 17, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 17, 24, 271, 727, 2923, 62, 18, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 15, 271, 727, 2923, 62, 18, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 16, 271, 727, 2923, 62, 18, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 17, 271, 727, 2923, 62, 18, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 18, 271, 727, 2923, 62, 18, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 19, 271, 727, 2923, 62, 18, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 20, 271, 727, 2923, 62, 18, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 21, 271, 727, 2923, 62, 18, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 22, 271, 727, 2923, 62, 18, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 23, 271, 727, 2923, 62, 18, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 18, 24, 271, 727, 2923, 62, 19, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 15, 271, 727, 2923, 62, 19, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 16, 271, 727, 2923, 62, 19, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 17, 271, 727, 2923, 62, 19, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 18, 271, 727, 2923, 62, 19, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 19, 271, 727, 2923, 62, 19, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 20, 271, 727, 2923, 62, 19, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 21, 271, 727, 2923, 62, 19, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 22, 271, 727, 2923, 62, 19, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 23, 271, 727, 2923, 62, 19, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 19, 24, 271, 727, 2923, 62, 20, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 15, 271, 727, 2923, 62, 20, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 16, 271, 727, 2923, 62, 20, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 17, 271, 727, 2923, 62, 20, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 18, 271, 727, 2923, 62, 20, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 19, 271, 727, 2923, 62, 20, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 20, 271, 727, 2923, 62, 20, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 21, 271, 727, 2923, 62, 20, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 22, 271, 727, 2923, 62, 20, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 23, 271, 727, 2923, 62, 20, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 20, 24, 271, 727, 2923, 62, 21, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 15, 271, 727, 2923, 62, 21, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 16, 271, 727, 2923, 62, 21, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 17, 271, 727, 2923, 62, 21, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 18, 271, 727, 2923, 62, 21, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 19, 271, 727, 2923, 62, 21, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 20, 271, 727, 2923, 62, 21, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 21, 271, 727, 2923, 62, 21, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 22, 271, 727, 2923, 62, 21, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 23, 271, 727, 2923, 62, 21, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 21, 24, 271, 727, 2923, 62, 22, 15, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 15, 271, 727, 2923, 62, 22, 16, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 16, 271, 727, 2923, 62, 22, 17, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 17, 271, 727, 2923, 62, 22, 18, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 18, 271, 727, 2923, 62, 22, 19, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 19, 271, 727, 2923, 62, 22, 20, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 20, 271, 727, 2923, 62, 22, 21, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 21, 271, 727, 2923, 62, 22, 22, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 22, 271, 727, 2923, 62, 22, 23, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 23, 271, 727, 2923, 62, 22, 24, 2007, 25, 514, 8, 1411, 514, 25, 198, 262, 460, 830, 478, 220, 22, 24, 271, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Write the module.", "tool_calls": [{"name": "write_file", "arguments": {"path": "steps.py", "content": "\"\"\"Generated module used as a long tool argument.\"\"\"\n\nfrom __future__ import annotations\n\ndef step_00(x: int) -> int:\n    return x + 0\n\ndef step_01(x: int) -> int:\n    return x + 1\n\ndef step_02(x: int) -> int:\n    return x + 2\n\ndef step_03(x: int) -> int:\n    return x + 3\n\ndef step_04(x: int) -> int:\n    return x + 4\n\ndef step_05(x: int) -> int:\n    return x + 5\n\ndef step_06(x: int) -> int:\n    return x + 6\n\ndef step_07(x: int) -> int:\n    return x + 7\n\ndef step_08(x: int) -> int:\n    return x + 8\n\ndef step_09(x: int) -> int:\n    return x + 9\n\ndef step_10(x: int) -> int:\n    return x + 10\n\ndef step_11(x: int) -> int:\n    return x + 11\n\ndef step_12(x: int) -> int:\n    return x + 12\n\ndef step_13(x: int) -> int:\n    return x + 13\n\ndef step_14(x: int) -> int:\n    return x + 14\n\ndef step_15(x: int) -> int:\n    return x + 15\n\ndef step_16(x: int) -> int:\n    return x + 16\n\ndef step_17(x: int) -> int:\n    return x + 17\n\ndef step_18(x: int) -> int:\n    return x + 18\n\ndef step_19(x: int) -> int:\n    return x + 19\n\ndef step_20(x: int) -> int:\n    return x + 20\n\ndef step_21(x: int) -> int:\n    return x + 21\n\ndef step_22(x: int) -> int:\n    return x + 22\n\ndef step_23(x: int) -> int:\n    return x + 23\n\ndef step_24(x: int) -> int:\n    return x + 24\n\ndef step_25(x: int) -> int:\n    return x + 25\n\ndef step_26(x: int) -> int:\n    return x + 26\n\ndef step_27(x: int) -> int:\n    return x + 27\n\ndef step_28(x: int) -> int:\n    return x + 28\n\ndef step_29(x: int) -> int:\n    return x + 29\n\ndef step_30(x: int) -> int:\n    return x + 30\n\ndef step_31(x: int) -> int:\n    return x + 31\n\ndef step_32(x: int) -> int:\n    return x + 32\n\ndef step_33(x: int) -> int:\n    return x + 33\n\ndef step_34(x: int) -> int:\n    return x + 34\n\ndef step_35(x: int) -> int:\n    return x + 35\n\ndef step_36(x: int) -> int:\n    return x + 36\n\ndef step_37(x: int) -> int:\n    return x + 37\n\ndef step_38(x: int) -> int:\n    return x + 38\n\ndef step_39(x: int) -> int:\n    return x + 39\n\ndef step_40(x: int) -> int:\n    return x + 40\n\ndef step_41(x: int) -> int:\n    return x + 41\n\ndef step_42(x: int) -> int:\n    return x + 42\n\ndef step_43(x: int) -> int:\n    return x + 43\n\ndef step_44(x: int) -> int:\n    return x + 44\n\ndef step_45(x: int) -> int:\n    return x + 45\n\ndef step_46(x: int) -> int:\n    return x + 46\n\ndef step_47(x: int) -> int:\n    return x + 47\n\ndef step_48(x: int) -> int:\n    return x + 48\n\ndef step_49(x: int) -> int:\n    return x + 49\n\ndef step_50(x: int) -> int:\n    return x + 50\n\ndef step_51(x: int) -> int:\n    return x + 51\n\ndef step_52(x: int) -> int:\n    return x + 52\n\ndef step_53(x: int) -> int:\n    return x + 53\n\ndef step_54(x: int) -> int:\n    return x + 54\n\ndef step_55(x: int) -> int:\n    return x + 55\n\ndef step_56(x: int) -> int:\n    return x + 56\n\ndef step_57(x: int) -> int:\n    return x + 57\n\ndef step_58(x: int) -> int:\n    return x + 58\n\ndef step_59(x: int) -> int:\n    return x + 59\n\ndef step_60(x: int) -> int:\n    return x + 60\n\ndef step_61(x: int) -> int:\n    return x + 61\n\ndef step_62(x: int) -> int:\n    return x + 62\n\ndef step_63(x: int) -> int:\n    return x + 63\n\ndef step_64(x: int) -> int:\n    return x + 64\n\ndef step_65(x: int) -> int:\n    return x + 65\n\ndef step_66(x: int) -> int:\n    return x + 66\n\ndef step_67(x: int) -> int:\n    return x + 67\n\ndef step_68(x: int) -> int:\n    return x + 68\n\ndef step_69(x: int) -> int:\n    return x + 69\n\ndef step_70(x: int) -> int:\n    return x + 70\n\ndef step_71(x: int) -> int:\n    return x + 71\n\ndef step_72(x: int) -> int:\n    return x + 72\n\ndef step_73(x: int) -> int:\n    return x + 73\n\ndef step_74(x: int) -> int:\n    return x + 74\n\ndef step_75(x: int) -> int:\n    return x + 75\n\ndef step_76(x: int) -> int:\n    return x + 76\n\ndef step_77(x: int) -> int:\n    return x + 77\n\ndef step_78(x: int) -> int:\n    return x + 78\n\ndef step_79(x: int) -> int:\n    return x + 79\n"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "long-arguments", "x-multiline-string"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-marker-in-arguments expected_match

Checks that did not pass for qwen3-xml/q38-marker-in-arguments
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Write the note.', got 'Write the note.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Write the note.', got 'Write the note.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-marker-in-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 7 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Write the note.",
+  "reasoning_content": "Write the note.\n",
   "tool_calls": [
     {

Fixture

Raw output

Write the note.
</think>

<tool_call>
<function=write_file>
<parameter=path>
notes.md
</parameter>
<parameter=content>
Close tags inline: </parameter> and </function> and <parameter=path> are text here.
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the note.",
  "tool_calls": [
    {
      "arguments": {
        "content": "Close tags inline: </parameter> and </function> and <parameter=path> are text here.",
        "path": "notes.md"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-marker-in-arguments", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Write the note.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\nnotes.md\n</parameter>\n<parameter=content>\nClose tags inline: </parameter> and </function> and <parameter=path> are text here.\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [7734, 279, 5020, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 17742, 20668, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 7675, 9212, 7147, 25, 672, 15704, 29, 321, 672, 1628, 29, 321, 361, 15704, 79114, 29, 513, 1414, 1532, 13, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Write the note.", "tool_calls": [{"name": "write_file", "arguments": {"path": "notes.md", "content": "Close tags inline: </parameter> and </function> and <parameter=path> are text here."}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "marker-in-arguments"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. The value contains XML-looking markers that are not at a line start. The structural delimiter is '\\n</parameter>\\n', so a parser that splits on a bare '</parameter>' cuts the value short."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-multiline-string expected_match

Checks that did not pass for qwen3-xml/q38-multiline-string
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Write the script.', got 'Write the script.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Write the script.', got 'Write the script.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-multiline-string --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 11 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Write the script.",
+  "reasoning_content": "Write the script.\n",
   "tool_calls": [
     {

Fixture

Raw output

Write the script.
</think>

<tool_call>
<function=write_file>
<parameter=path>
greet.py
</parameter>
<parameter=content>
def greet(name: str) -> str:
    """Say hi."""
    return f"Hello, {name}!"


print(greet("Zoë"))
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the script.",
  "tool_calls": [
    {
      "arguments": {
        "content": "def greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))",
        "path": "greet.py"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-multiline-string", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Write the script.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\ngreet.py\n</parameter>\n<parameter=content>\ndef greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [7734, 279, 5148, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 70, 3617, 6971, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 727, 39440, 3052, 25, 592, 8, 1411, 592, 25, 198, 262, 4071, 44240, 15131, 71483, 198, 262, 460, 281, 1, 9419, 11, 313, 591, 92, 8687, 1358, 1302, 3163, 3617, 437, 230123, 11831, 2673, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Write the script.", "tool_calls": [{"name": "write_file", "arguments": {"path": "greet.py", "content": "def greet(name: str) -> str:\n    \"\"\"Say hi.\"\"\"\n    return f\"Hello, {name}!\"\n\n\nprint(greet(\"Zoë\"))"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "string-escapes", "x-multiline-string"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-nested-json expected_match

Checks that did not pass for qwen3-xml/q38-nested-json
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Search with structured filters.', got 'Search with structured filters.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Search with structured filters.', got 'Search with structured filters.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-nested-json --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 5 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Search with structured filters.",
+  "reasoning_content": "Search with structured filters.\n",
   "tool_calls": [
     {

Fixture

Raw output

Search with structured filters.
</think>

<tool_call>
<function=search>
<parameter=query>
hotels
</parameter>
<parameter=filters>
{"price": {"min": 50, "max": 120.5}, "tags": ["pool", "wifi"], "open": null}
</parameter>
<parameter=sites>
["a.example", "b.example"]
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Search with structured filters.",
  "tool_calls": [
    {
      "arguments": {
        "filters": {
          "open": null,
          "price": {
            "max": 120.5,
            "min": 50
          },
          "tags": [
            "pool",
            "wifi"
          ]
        },
        "query": "hotels",
        "sites": [
          "a.example",
          "b.example"
        ]
      },
      "name": "search"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-nested-json", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Search with structured filters.\n</think>\n\n<tool_call>\n<function=search>\n<parameter=query>\nhotels\n</parameter>\n<parameter=filters>\n{\"price\": {\"min\": 50, \"max\": 120.5}, \"tags\": [\"pool\", \"wifi\"], \"open\": null}\n</parameter>\n<parameter=sites>\n[\"a.example\", \"b.example\"]\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [5708, 440, 31838, 13019, 13, 198, 248069, 271, 248058, 198, 27, 1628, 93260, 29, 198, 27, 15704, 76478, 29, 198, 10307, 1943, 198, 510, 15704, 29, 198, 27, 15704, 17572, 8349, 29, 198, 4754, 6352, 763, 5046, 1030, 763, 220, 20, 15, 11, 328, 2721, 763, 220, 16, 17, 15, 13, 20, 2069, 328, 13679, 763, 4241, 9979, 487, 328, 65011, 7664, 328, 2428, 763, 819, 92, 198, 510, 15704, 29, 198, 27, 15704, 13744, 3490, 29, 198, 1143, 64, 7479, 487, 328, 65, 7479, 1293, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Search with structured filters.", "tool_calls": [{"name": "search", "arguments": {"query": "hotels", "filters": {"price": {"min": 50, "max": 120.5}, "tags": ["pool", "wifi"], "open": null}, "sites": ["a.example", "b.example"]}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "nested-json"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Object and array values are rendered with tojson; the parser must decode them as JSON."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-no-call expected_match

Checks that did not pass for qwen3-xml/q38-no-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'A simple greeting; no tool is needed.', got 'A simple greeting; no tool is needed.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'A simple greeting; no tool is needed.', got 'A simple greeting; no tool is needed.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-no-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 10 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": "Hello! How can I help you today?",
-  "reasoning_content": "A simple greeting; no tool is needed.",
+  "reasoning_content": "A simple greeting; no tool is needed.\n",
   "tool_calls": []
 }

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: no-call, reasoning, reasoning-prefilled.

Raw output

A simple greeting; no tool is needed.
</think>

Hello! How can I help you today?

Expected parse

{
  "content": "Hello! How can I help you today?",
  "reasoning_content": "A simple greeting; no tool is needed.",
  "tool_calls": []
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-no-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "A simple greeting; no tool is needed.\n</think>\n\nHello! How can I help you today?", "output_token_ids": [32, 4145, 40719, 26, 874, 5224, 369, 4221, 13, 198, 248069, 271, 9419, 0, 2500, 628, 353, 1438, 488, 3242, 30], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": "Hello! How can I help you today?", "reasoning_content": "A simple greeting; no tool is needed.", "tool_calls": []}, "tags": ["no-call", "reasoning", "reasoning-prefilled"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-numeric-and-literal-arguments expected_match

Checks that did not pass for qwen3-xml/q38-numeric-and-literal-arguments
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Set the alarm.', got 'Set the alarm.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Set the alarm.', got 'Set the alarm.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-numeric-and-literal-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 13 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Set the alarm.",
+  "reasoning_content": "Set the alarm.\n",
   "tool_calls": [
     {

Fixture

Raw output

Set the alarm.
</think>

<tool_call>
<function=set_alarm>
<parameter=hour>
7
</parameter>
<parameter=minute>
30
</parameter>
<parameter=volume>
0.75
</parameter>
<parameter=repeat>
true
</parameter>
<parameter=label>
null
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Set the alarm.",
  "tool_calls": [
    {
      "arguments": {
        "hour": 7,
        "label": null,
        "minute": 30,
        "repeat": true,
        "volume": 0.75
      },
      "name": "set_alarm"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-numeric-and-literal-arguments", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Set the alarm.\n</think>\n\n<tool_call>\n<function=set_alarm>\n<parameter=hour>\n7\n</parameter>\n<parameter=minute>\n30\n</parameter>\n<parameter=volume>\n0.75\n</parameter>\n<parameter=repeat>\ntrue\n</parameter>\n<parameter=label>\nnull\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [1591, 279, 16136, 13, 198, 248069, 271, 248058, 198, 27, 1628, 66473, 64049, 29, 198, 27, 15704, 28, 23899, 29, 198, 22, 198, 510, 15704, 29, 198, 27, 15704, 28, 41143, 29, 198, 18, 15, 198, 510, 15704, 29, 198, 27, 15704, 28, 24266, 29, 198, 15, 13, 22, 20, 198, 510, 15704, 29, 198, 27, 15704, 28, 29625, 29, 198, 1802, 198, 510, 15704, 29, 198, 27, 15704, 95277, 29, 198, 2827, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Set the alarm.", "tool_calls": [{"name": "set_alarm", "arguments": {"hour": 7, "minute": 30, "volume": 0.75, "repeat": true, "label": null}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "numeric-arguments", "x-schema-coercion"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Scalars are raw text; integer/number/boolean/null come back only through the tool's JSON schema."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-numeric-looking-strings expected_match

Checks that did not pass for qwen3-xml/q38-numeric-looking-strings
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Look it up.', got 'Look it up.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Look it up.', got 'Look it up.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-numeric-looking-strings --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 14 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Look it up.",
+  "reasoning_content": "Look it up.\n",
   "tool_calls": [
     {

Fixture

Raw output

Look it up.
</think>

<tool_call>
<function=lookup>
<parameter=zip>
02139
</parameter>
<parameter=account_id>
123456789012345678901
</parameter>
<parameter=flag>
true
</parameter>
<parameter=payload>
{"a": 1}
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Look it up.",
  "tool_calls": [
    {
      "arguments": {
        "account_id": "123456789012345678901",
        "flag": "true",
        "payload": "{\"a\": 1}",
        "zip": "02139"
      },
      "name": "lookup"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-numeric-looking-strings", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Look it up.\n</think>\n\n<tool_call>\n<function=lookup>\n<parameter=zip>\n02139\n</parameter>\n<parameter=account_id>\n123456789012345678901\n</parameter>\n<parameter=flag>\ntrue\n</parameter>\n<parameter=payload>\n{\"a\": 1}\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [10071, 424, 685, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 20377, 29, 198, 27, 15704, 28, 9669, 29, 198, 15, 17, 16, 18, 24, 198, 510, 15704, 29, 198, 27, 15704, 28, 4459, 816, 29, 198, 16, 17, 18, 19, 20, 21, 22, 23, 24, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 15, 16, 198, 510, 15704, 29, 198, 27, 15704, 28, 9610, 29, 198, 1802, 198, 510, 15704, 29, 198, 27, 15704, 16874, 6771, 29, 198, 4754, 64, 763, 220, 16, 92, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Look it up.", "tool_calls": [{"name": "lookup", "arguments": {"zip": "02139", "account_id": "123456789012345678901", "flag": "true", "payload": "{\"a\": 1}"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "x-schema-coercion"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Every parameter is typed 'string' in the schema, so '02139', 'true' and '{\"a\": 1}' must stay strings."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-parallel-calls expected_match

Checks that did not pass for qwen3-xml/q38-parallel-calls
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'I should call the tools.', got 'I should call the tools.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'I should call the tools.', got 'I should call the tools.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-parallel-calls --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 2 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "I should call the tools.",
+  "reasoning_content": "I should call the tools.\n",
   "tool_calls": [
     {

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: parallel-calls, reasoning, reasoning-prefilled, unicode, nested-json, string-escapes.

Raw output

I should call the tools.
</think>

<tool_call>
<function=get_weather>
<parameter=city>
Zürich
</parameter>
<parameter=unit>
c
</parameter>
</function>
</tool_call>
<tool_call>
<function=search>
<parameter=query>
café "best"
</parameter>
<parameter=filters>
{"tags": ["a", "b"], "max": 3}
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "I should call the tools.",
  "tool_calls": [
    {
      "arguments": {
        "city": "Zürich",
        "unit": "c"
      },
      "name": "get_weather"
    },
    {
      "arguments": {
        "filters": {
          "max": 3,
          "tags": [
            "a",
            "b"
          ]
        },
        "query": "café \"best\""
      },
      "name": "search"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-parallel-calls", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "I should call the tools.\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nZürich\n</parameter>\n<parameter=unit>\nc\n</parameter>\n</function>\n</tool_call>\n<tool_call>\n<function=search>\n<parameter=query>\ncafé \"best\"\n</parameter>\n<parameter=filters>\n{\"tags\": [\"a\", \"b\"], \"max\": 3}\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [40, 1220, 1562, 279, 7141, 13, 198, 248069, 271, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 57, 5021, 693, 198, 510, 15704, 29, 198, 27, 15704, 28, 3715, 29, 198, 66, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059, 198, 248058, 198, 27, 1628, 93260, 29, 198, 27, 15704, 76478, 29, 198, 895, 56868, 328, 15395, 1, 198, 510, 15704, 29, 198, 27, 15704, 17572, 8349, 29, 198, 4754, 13679, 763, 4241, 64, 487, 328, 65, 7664, 328, 2721, 763, 220, 18, 92, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "I should call the tools.", "tool_calls": [{"name": "get_weather", "arguments": {"city": "Zürich", "unit": "c"}}, {"name": "search", "arguments": {"query": "café \"best\"", "filters": {"tags": ["a", "b"], "max": 3}}}]}, "tags": ["parallel-calls", "reasoning", "reasoning-prefilled", "unicode", "nested-json", "string-escapes"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-single-call expected_match

Checks that did not pass for qwen3-xml/q38-single-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-single-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 1 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
+  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.\n",
   "tool_calls": [
     {

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: single-call, reasoning, reasoning-prefilled.

Raw output

The user wants the weather in Paris. I should call get_weather.
</think>

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
<parameter=unit>
c
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
  "tool_calls": [
    {
      "arguments": {
        "city": "Paris",
        "unit": "c"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-single-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "The user wants the weather in Paris. I should call get_weather.\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n<parameter=unit>\nc\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [760, 1156, 6587, 279, 8831, 303, 11751, 13, 353, 1220, 1562, 615, 67017, 13, 198, 248069, 271, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 57590, 198, 510, 15704, 29, 198, 27, 15704, 28, 3715, 29, 198, 66, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "The user wants the weather in Paris. I should call get_weather.", "tool_calls": [{"name": "get_weather", "arguments": {"city": "Paris", "unit": "c"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-string-escapes expected_match

Checks that did not pass for qwen3-xml/q38-string-escapes
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Write it.', got 'Write it.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Write it.', got 'Write it.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-string-escapes --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 15 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Write it.",
+  "reasoning_content": "Write it.\n",
   "tool_calls": [
     {

Fixture

Raw output

Write it.
</think>

<tool_call>
<function=write_file>
<parameter=path>
C:\tmp\x.txt
</parameter>
<parameter=content>
a "quote", a \backslash, <tag> & ampersand	tab
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Write it.",
  "tool_calls": [
    {
      "arguments": {
        "content": "a \"quote\", a \\backslash, <tag> & ampersand\ttab",
        "path": "C:\\tmp\\x.txt"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-string-escapes", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Write it.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\nC:\\tmp\\x.txt\n</parameter>\n<parameter=content>\na \"quote\", a \\backslash, <tag> & ampersand\ttab\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [7734, 424, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 34, 6964, 5008, 3351, 3778, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 64, 328, 2855, 487, 264, 1088, 1371, 48569, 11, 361, 4429, 29, 594, 22050, 382, 429, 56196, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Write it.", "tool_calls": [{"name": "write_file", "arguments": {"path": "C:\\tmp\\x.txt", "content": "a \"quote\", a \\backslash, <tag> & ampersand\ttab"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "string-escapes"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. String values are inserted verbatim with no escaping."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-text-before-call expected_match

Checks that did not pass for qwen3-xml/q38-text-before-call
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	content (whitespace only): expected 'Let me check the weather for you.', got 'Let me check the weather for you.\n\n'; reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'
`expected_match`	`token`	soft pass	content (whitespace only): expected 'Let me check the weather for you.', got 'Let me check the weather for you.\n\n'; reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-text-before-call --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 9 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
-  "content": "Let me check the weather for you.",
-  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
+  "content": "Let me check the weather for you.\n\n",
+  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.\n",
   "tool_calls": [
     {

Fixture

Raw output

The user wants the weather in Paris. I should call get_weather.
</think>

Let me check the weather for you.

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": "Let me check the weather for you.",
  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
  "tool_calls": [
    {
      "arguments": {
        "city": "Paris"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-text-before-call", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "The user wants the weather in Paris. I should call get_weather.\n</think>\n\nLet me check the weather for you.\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [760, 1156, 6587, 279, 8831, 303, 11751, 13, 353, 1220, 1562, 615, 67017, 13, 198, 248069, 271, 9764, 728, 1716, 279, 8831, 364, 488, 13, 271, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 57590, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": "Let me check the weather for you.", "reasoning_content": "The user wants the weather in Paris. I should call get_weather.", "tool_calls": [{"name": "get_weather", "arguments": {"city": "Paris"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "text-before-call"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Qwen3.5+ templates join content and the first <tool_call> with '\\n\\n', which is markup."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-three-parallel-calls expected_match

Checks that did not pass for qwen3-xml/q38-three-parallel-calls
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Three cities, three calls.', got 'Three cities, three calls.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Three cities, three calls.', got 'Three cities, three calls.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-three-parallel-calls --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 3 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Three cities, three calls.",
+  "reasoning_content": "Three cities, three calls.\n",
   "tool_calls": [
     {

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: parallel-calls, reasoning, reasoning-prefilled.

Raw output

Three cities, three calls.
</think>

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>
</tool_call>
<tool_call>
<function=get_weather>
<parameter=city>
Berlin
</parameter>
</function>
</tool_call>
<tool_call>
<function=get_weather>
<parameter=city>
Tokyo
</parameter>
<parameter=unit>
f
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Three cities, three calls.",
  "tool_calls": [
    {
      "arguments": {
        "city": "Paris"
      },
      "name": "get_weather"
    },
    {
      "arguments": {
        "city": "Berlin"
      },
      "name": "get_weather"
    },
    {
      "arguments": {
        "city": "Tokyo",
        "unit": "f"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-three-parallel-calls", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Three cities, three calls.\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call>\n<tool_call>\n<function=get_weather>\n<parameter=city>\nBerlin\n</parameter>\n</function>\n</tool_call>\n<tool_call>\n<function=get_weather>\n<parameter=city>\nTokyo\n</parameter>\n<parameter=unit>\nf\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [19041, 9432, 11, 2250, 6526, 13, 198, 248069, 271, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 57590, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059, 198, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 91149, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059, 198, 248058, 198, 27, 1628, 27362, 67017, 29, 198, 27, 15704, 28, 8656, 29, 198, 51076, 15560, 198, 510, 15704, 29, 198, 27, 15704, 28, 3715, 29, 198, 69, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Three cities, three calls.", "tool_calls": [{"name": "get_weather", "arguments": {"city": "Paris"}}, {"name": "get_weather", "arguments": {"city": "Berlin"}}, {"name": "get_weather", "arguments": {"city": "Tokyo", "unit": "f"}}]}, "tags": ["parallel-calls", "reasoning", "reasoning-prefilled"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-tool-call-token-in-arguments expected_match

Checks that did not pass for qwen3-xml/q38-tool-call-token-in-arguments
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Document the format.', got 'Document the format.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Document the format.', got 'Document the format.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-tool-call-token-in-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 8 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Document the format.",
+  "reasoning_content": "Document the format.\n",
   "tool_calls": [
     {

Fixture

Raw output

Document the format.
</think>

<tool_call>
<function=write_file>
<parameter=path>
fmt.md
</parameter>
<parameter=content>
Qwen wraps calls in <tool_call> ... </tool_call> blocks.
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Document the format.",
  "tool_calls": [
    {
      "arguments": {
        "content": "Qwen wraps calls in <tool_call> ... </tool_call> blocks.",
        "path": "fmt.md"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-tool-call-token-in-arguments", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Document the format.\n</think>\n\n<tool_call>\n<function=write_file>\n<parameter=path>\nfmt.md\n</parameter>\n<parameter=content>\nQwen wraps calls in <tool_call> ... </tool_call> blocks.\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [7288, 279, 3443, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 4775, 2378, 29, 198, 27, 15704, 79114, 29, 198, 12143, 20668, 198, 510, 15704, 29, 198, 27, 15704, 28, 1733, 29, 198, 48, 16451, 38394, 6526, 303, 220, 248058, 2423, 220, 248059, 9714, 13, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Document the format.", "tool_calls": [{"name": "write_file", "arguments": {"path": "fmt.md", "content": "Qwen wraps calls in <tool_call> ... </tool_call> blocks."}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "marker-in-arguments"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. '<tool_call>' and '</tool_call>' are added tokens, so the tokenizer maps them to their marker ids even inside the argument value (see output_token_ids). Only the XML structure tells them apart."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-truncated-at-tool-call-open expected_match

Checks that did not pass for qwen3-xml/q38-truncated-at-tool-call-open
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'The user wants the weather in Paris. I should call get_weather.', got 'The user wants the weather in Paris. I should call get_weather.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-truncated-at-tool-call-open --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 23 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
+  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.\n",
   "tool_calls": []
 }

Fixture

Provenance: template_render, https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja. Tags: truncated, reasoning, reasoning-prefilled, x-stop-at-open-marker.

Raw output

The user wants the weather in Paris. I should call get_weather.
</think>

<tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "The user wants the weather in Paris. I should call get_weather.",
  "tool_calls": []
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-truncated-at-tool-call-open", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "The user wants the weather in Paris. I should call get_weather.\n</think>\n\n<tool_call>", "output_token_ids": [760, 1156, 6587, 279, 8831, 303, 11751, 13, 353, 1220, 1562, 615, 67017, 13, 198, 248069, 271, 248058], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "The user wants the weather in Paris. I should call get_weather.", "tool_calls": []}, "tags": ["truncated", "reasoning", "reasoning-prefilled", "x-stop-at-open-marker"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Output stopped right after the <tool_call> opener. The reasoning is complete and must be kept; there is no call yet and the marker must not leak into content (SGLang #35565: https://github.com/sgl-project/sglang/issues/35565)."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": null,
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-unicode-emoji expected_match

Checks that did not pass for qwen3-xml/q38-unicode-emoji
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected '用户想知道东京的天气。🌦️', got '用户想知道东京的天气。🌦️\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected '用户想知道东京的天气。🌦️', got '用户想知道东京的天气。🌦️\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-unicode-emoji --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 4 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "用户想知道东京的天气。🌦️",
+  "reasoning_content": "用户想知道东京的天气。🌦️\n",
   "tool_calls": [
     {

Fixture

Raw output

用户想知道东京的天气。🌦️
</think>

<tool_call>
<function=search>
<parameter=query>
東京の天気 ☀️🌧️ — «prévisions» für Zürich 🇨🇭
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "用户想知道东京的天气。🌦️",
  "tool_calls": [
    {
      "arguments": {
        "query": "東京の天気 ☀️🌧️ — «prévisions» für Zürich 🇨🇭"
      },
      "name": "search"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-unicode-emoji", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "用户想知道东京的天气。🌦️\n</think>\n\n<tool_call>\n<function=search>\n<parameter=query>\n東京の天気 ☀️🌧️ — «prévisions» für Zürich 🇨🇭\n</parameter>\n</function>\n</tool_call>", "output_token_ids": [97237, 102487, 110799, 131802, 1710, 9008, 234, 99, 29545, 198, 248069, 271, 248058, 198, 27, 1628, 93260, 29, 198, 27, 15704, 76478, 29, 198, 115197, 15303, 247570, 24329, 222, 29545, 9008, 234, 100, 29545, 1892, 12129, 631, 933, 38685, 12620, 6954, 194323, 10838, 229, 101, 9008, 229, 255, 198, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "用户想知道东京的天气。🌦️", "tool_calls": [{"name": "search", "arguments": {"query": "東京の天気 ☀️🌧️ — «prévisions» für Zürich 🇨🇭"}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "unicode"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

soft pass qwen3-xml/q38-whitespace-significant expected_match

Checks that did not pass for qwen3-xml/q38-whitespace-significant
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	soft pass	reasoning_content (whitespace only): expected 'Replace the block exactly.', got 'Replace the block exactly.\n'
`expected_match`	`token`	soft pass	reasoning_content (whitespace only): expected 'Replace the block exactly.', got 'Replace the block exactly.\n'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/qwen3-xml/rendered.jsonl --id qwen3-xml/q38-whitespace-significant --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 12 of fixtures/qwen3-xml/rendered.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,5 +1,5 @@
 {
   "content": null,
-  "reasoning_content": "Replace the block exactly.",
+  "reasoning_content": "Replace the block exactly.\n",
   "tool_calls": [
     {

Fixture

Raw output

Replace the block exactly.
</think>

<tool_call>
<function=edit_file>
<parameter=path>
a.py
</parameter>
<parameter=old_string>
    if x:
        return 1

</parameter>
<parameter=new_string>
  two  spaces  
</parameter>
</function>
</tool_call>

Expected parse

{
  "content": null,
  "reasoning_content": "Replace the block exactly.",
  "tool_calls": [
    {
      "arguments": {
        "new_string": "  two  spaces  ",
        "old_string": "    if x:\n        return 1\n",
        "path": "a.py"
      },
      "name": "edit_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "qwen3-xml/q38-whitespace-significant", "family": "qwen3-xml", "models": ["Qwen/Qwen3.8-27B", "Qwen/Qwen3.6-35B-A3B"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "license": "Apache-2.0", "generator": "scripts/fixtures/qwen3-xml/render_qwen3_xml.py", "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object"}, "sites": {"type": "array", "items": {"type": "string"}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}, "required": []}}}, {"type": "function", "function": {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "edit_file", "description": "Replace an exact string in a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old_string": {"type": "string"}, "new_string": {"type": "string"}}, "required": ["path", "old_string", "new_string"]}}}, {"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm.", "parameters": {"type": "object", "properties": {"hour": {"type": "integer"}, "minute": {"type": "integer"}, "volume": {"type": "number"}, "repeat": {"type": "boolean"}, "label": {"type": ["string", "null"]}}, "required": ["hour", "minute"]}}}, {"type": "function", "function": {"name": "lookup", "description": "Look up a record by its identifiers.", "parameters": {"type": "object", "properties": {"zip": {"type": "string"}, "account_id": {"type": "string"}, "flag": {"type": "string"}, "payload": {"type": "string"}}, "required": ["zip"]}}}], "raw_output": "Replace the block exactly.\n</think>\n\n<tool_call>\n<function=edit_file>\n<parameter=path>\na.py\n</parameter>\n<parameter=old_string>\n    if x:\n        return 1\n\n</parameter>\n<parameter=new_string>\n  two  spaces  \n</parameter>\n</function>\n</tool_call>", "output_token_ids": [22386, 279, 2424, 6681, 13, 198, 248069, 271, 248058, 198, 27, 1628, 28, 3468, 2378, 29, 198, 27, 15704, 79114, 29, 198, 64, 6971, 198, 510, 15704, 29, 198, 27, 15704, 28, 787, 3773, 29, 198, 262, 413, 830, 25, 198, 285, 460, 220, 16, 271, 510, 15704, 29, 198, 27, 15704, 8083, 3773, 29, 198, 220, 1330, 220, 12258, 2228, 510, 15704, 29, 198, 510, 1628, 29, 198, 248059], "tokenizer": {"repo": "Qwen/Qwen3.8-27B", "revision": "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0", "mode": "hf"}, "generation_prompt": "<|im_start|>assistant\n<think>\n", "expected": {"content": null, "reasoning_content": "Replace the block exactly.", "tool_calls": [{"name": "edit_file", "arguments": {"path": "a.py", "old_string": "    if x:\n        return 1\n", "new_string": "  two  spaces  "}}]}, "tags": ["single-call", "reasoning", "reasoning-prefilled", "x-whitespace-significant-string"], "notes": "History render of the official Qwen/Qwen3.8-27B chat template. Leading indentation and a trailing newline are part of the values. Exactly one '\\n' on each side of a value is markup (vLLM #48753: https://github.com/vllm-project/vllm/issues/48753)."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handler 'Qwen3-Coder'",
  "fixture_generation_prompt": "<|im_start|>assistant\n<think>\n",
  "fixture_tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "format": "peg-native",
  "generation_prompt": "<|im_start|>assistant\n<think>\n",
  "model": "Qwen/Qwen3.8-27B",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<tool_call>",
    "</tool_call>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after Qwen/Qwen3.8-27B"
    }
  },
  "template_identical_to": null,
  "template_path": "Qwen--Qwen3.8-27B.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "\n</think>",
    "</think>",
    "<tool_call>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "33e20799515a5fd3964787ec5ae7c6b604e9e6dc0d825fce90acb0f5f3e4462a",
  "vocab_gguf": "Qwen--Qwen3.8-27B.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "7b5dcc22c704d56bf35ebc9cc40ce3a2f766890c968f5c8b6d34a3808680749f"
}

pass strict match on every realistic strategy
soft pass only whitespace differs (normalization soft-v1)
fail a check failed
error the harness failed, not the engine's parser
unsupported the engine has no parser for this family or model