Kimi (K2.x section tokens, K3 XTML) on llamacpp a25c9865

fail 71% strict pass 35 pass · 14 fail

Run

Engine

llamacpp a25c9865 (a25c9865fe03c954c93fd755b5d79ae86ba99750)

Run

2026-09-26T05:36:17Z to 2026-09-26T05:37:20Z

Platform

linux-x86_64, Python 3.12.14, canitoolcall 0.1.0.dev0

Strategies

one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8 (normalization soft-v1); not counted for this engine: one, special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

Fixtures digest

a1e7b55367e195e8aa6e2ef746b2cc141b44f37caad0887ff8f30b104616f836 the fixture corpus used for this site matches the run

Engine details

{
  "build_info": "b1-a25c986",
  "detokenizer": "common_token_to_piece(vocab-only GGUF, special = id in preserved_tokens)",
  "harness": "harnesses/llamacpp/replay.cpp",
  "harness_protocol": 1,
  "pinned_version": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "reasoning_format": "deepseek",
  "template_source": "gguf",
  "tokens_per_step": "one"
}

Results file

data/llamacpp-a25c9865.json

Checks

Per fixture, the worst result of each check over the non-streaming parse and every realistic chunking strategy.
Check	Pass	Fail	Strict pass rate
`expected_match`	35	11	76%
`expected_error`	1	2	33%
`stream_equals_nonstream`	39	10	80%
`no_leakage`	48	1	98%
`arguments_json`	26	5	84%
`arguments_schema`	25	6	81%
`parallel_order`	3	1	75%

14 fixture(s) failed only a synthetic strategy: char:* (which can split special tokens), or a multi-token strategy this engine never produces because its server streams one token per event (one, special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8). These are listed below but never count toward the status.

Fixtures needing attention

fail kimi/k26-truncated-inside-arguments expected_error, arguments_json, arguments_schema

Checks that did not pass for kimi/k26-truncated-inside-arguments
Check	Strategy	Result	Detail
`expected_error`	`nonstream`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`token`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`arguments_json`	`nonstream`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`token`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_schema`	`nonstream`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`token`	fail	[0] get_weather: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_error@one fail, expected_error@special fail, expected_error@rand:1:8 fail, expected_error@rand:2:8 fail, expected_error@rand:3:8 fail, expected_error@rand:4:8 fail, expected_error@rand:5:8 fail, arguments_json@one fail, arguments_json@special fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@one fail, arguments_schema@special fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/truncated.jsonl --id kimi/k26-truncated-inside-arguments --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 1 of fixtures/kimi/truncated.jsonl.

Observed

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "<arguments_raw, not valid JSON>": "{\"city\": \"Madrid\", \"unit\":"
      },
      "name": "get_weather"
    }
  ]
}

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja. Tags: truncated, x-kimi-k2.

Expected graceful failure: max_tokens hit inside the argument JSON; the call never closes. (accept: no_tool_calls, content_passthrough, exception).

Raw output

<|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0<|tool_call_argument_begin|>{"city": "Madrid", "unit":

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k26-truncated-inside-arguments", "family": "kimi", "models": ["moonshotai/Kimi-K2.6"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "license": "LicenseRef-modified-mit", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}], "raw_output": "<|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0<|tool_call_argument_begin|>{\"city\": \"Madrid\", \"unit\":", "output_token_ids": [163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 163598, 8264, 37666, 1289, 414, 44, 12708, 338, 665, 414, 8175, 1289], "tokenizer": {"repo": "moonshotai/Kimi-K2.6", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "mode": "hf"}, "generation_prompt": "<|im_assistant|>assistant<|im_middle|><think></think>", "thinking": false, "expected_error": {"reason": "max_tokens hit inside the argument JSON; the call never closes.", "accept": ["no_tool_calls", "content_passthrough", "exception"]}, "tags": ["truncated", "x-kimi-k2"], "notes": "Token prefix of kimi/k26-thinking-disabled-call."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": false,
    "thinking": false
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": null,
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|><think></think>",
  "fixture_tokenizer": "moonshotai/Kimi-K2.6@7eb5002f6aadc958aed6a9177b7ed26bb94011bb",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|><think></think>",
  "model": "moonshotai/Kimi-K2.6",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2.6"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2.6.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2.6@7eb5002f6aadc958aed6a9177b7ed26bb94011bb",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "a27267d507584ea678786a65346e27e363b3bc753dcaeed9dbc4826f761f908a",
  "vocab_gguf": "moonshotai--Kimi-K2.6.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "b6b10e1d15ea0db1828e64d081edfaa743cb757c0b5a809e456c6b07928e838e"
}

fail kimi/k2i-vllm-content-after-tool-section expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-content-after-tool-section
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got 'Before. '

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-content-after-tool-section --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 7 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
-  "content": "Before.  After tools.",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Tokyo"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
-  "content": "Before.  After tools.",
+  "content": "Before. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Tokyo"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L435-L458. Tags: single-call, text-before-call, text-after-call, x-policy, x-kimi-k2.

Raw output

Before. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{"city": "Tokyo"} <|tool_call_end|><|tool_calls_section_end|> After tools.

Expected parse

{
  "content": "Before.  After tools.",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Tokyo"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-content-after-tool-section", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L435-L458", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}], "raw_output": "Before. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{\"city\": \"Tokyo\"} <|tool_call_end|><|tool_calls_section_end|> After tools.", "output_token_ids": [13295, 13, 220, 163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 220, 163598, 8264, 37666, 1289, 414, 39818, 18009, 16934, 220, 163599, 163596, 6671, 7697, 13], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Before.  After tools.", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "Tokyo"}}]}, "tags": ["single-call", "text-before-call", "text-after-call", "x-policy", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. The deltas of the streaming test, concatenated. vLLM's test asserts the trailing text is DROPPED; this fixture expects it kept, because text outside the tool-calls section is ordinary content and dropping it loses model output. Content is the text before and after the section, concatenated verbatim (spec/README.md, 'Content around tool calls'). Tagged x-policy: the expected value encodes that spec rule, which neither the Kimi format nor the cited test fixes, so the matrix can show it apart from format conformance."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "09331a2e2df9a56a2f3acb61624999968c53eea45d0a0be1bb17cd57f265c61f",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k2i-vllm-id-without-functions-prefix expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-id-without-functions-prefix
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got 'No prefix. '

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-id-without-functions-prefix --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 4 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
-  "content": "No prefix. ",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Tokyo"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
   "content": "No prefix. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Tokyo"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L120-L126. Tags: single-call, text-before-call, x-non-kimi-call-id, x-kimi-k2.

Raw output

No prefix. <|tool_calls_section_begin|><|tool_call_begin|>get_weather:0 <|tool_call_argument_begin|>{"city": "Tokyo"}<|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "No prefix. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Tokyo"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-id-without-functions-prefix", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L120-L126", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}], "raw_output": "No prefix. <|tool_calls_section_begin|><|tool_call_begin|>get_weather:0 <|tool_call_argument_begin|>{\"city\": \"Tokyo\"}<|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [4729, 14280, 13, 220, 163595, 163597, 618, 21055, 2800, 25, 15, 220, 163598, 8264, 37666, 1289, 414, 39818, 18009, 16934, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "No prefix. ", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "Tokyo"}}]}, "tags": ["single-call", "text-before-call", "x-non-kimi-call-id", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. The id lacks the 'functions.' prefix; the name is still recoverable from '{name}:{idx}'."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "09331a2e2df9a56a2f3acb61624999968c53eea45d0a0be1bb17cd57f265c61f",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k2i-vllm-multiline-json expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-multiline-json
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got 'Formatted. '

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-multiline-json --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 3 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,13 +1,6 @@
 {
-  "content": "Formatted. ",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "name": "test",
-        "value": 123
-      },
-      "name": "process_data"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,13 +1,6 @@
 {
   "content": "Formatted. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "name": "test",
-        "value": 123
-      },
-      "name": "process_data"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L107-L119. Tags: single-call, text-before-call, numeric-arguments, x-kimi-k2.

Raw output

Formatted. <|tool_calls_section_begin|><|tool_call_begin|>functions.process_data:0 <|tool_call_argument_begin|>{
  "name": "test",
  "value": 123
}<|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "Formatted. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "name": "test",
        "value": 123
      },
      "name": "process_data"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-multiline-json", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L107-L119", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "process_data", "parameters": {"type": "object", "properties": {"name": {"type": "string"}, "value": {"type": "integer"}}}}}], "raw_output": "Formatted. <|tool_calls_section_begin|><|tool_call_begin|>functions.process_data:0 <|tool_call_argument_begin|>{\n  \"name\": \"test\",\n  \"value\": 123\n}<|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [68615, 13, 220, 163595, 163597, 41937, 28866, 3419, 25, 15, 220, 163598, 1028, 220, 414, 1152, 1289, 414, 3078, 1137, 220, 414, 2035, 1289, 220, 6694, 198, 92, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Formatted. ", "reasoning_content": null, "tool_calls": [{"name": "process_data", "arguments": {"name": "test", "value": 123}}]}, "tags": ["single-call", "text-before-call", "numeric-arguments", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "50c0b07a87b6b5a24edb5c308f09342fcb12f7956d52fb55afba9d220d38d2de",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k2i-vllm-noise-between-markers expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-noise-between-markers
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got 'Reasoning. '

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-noise-between-markers --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 5 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
-  "content": "Reasoning. ",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "k": "v"
-      },
-      "name": "test"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
   "content": "Reasoning. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "k": "v"
-      },
-      "name": "test"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L370-L386. Tags: single-call, text-before-call, malformed, x-kimi-k2.

Raw output

Reasoning. <|tool_calls_section_begin|> spurious noise <|tool_call_begin|>functions.test:0 <|tool_call_argument_begin|>{"k": "v"} <|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "Reasoning. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "k": "v"
      },
      "name": "test"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-noise-between-markers", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L370-L386", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "test", "parameters": {"type": "object", "properties": {"k": {"type": "string"}}}}}], "raw_output": "Reasoning. <|tool_calls_section_begin|> spurious noise <|tool_call_begin|>functions.test:0 <|tool_call_argument_begin|>{\"k\": \"v\"} <|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [24977, 288, 13, 220, 163595, 1284, 26163, 18226, 220, 163597, 41937, 9478, 25, 15, 220, 163598, 8264, 74, 1289, 414, 85, 16934, 220, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Reasoning. ", "reasoning_content": null, "tool_calls": [{"name": "test", "arguments": {"k": "v"}}]}, "tags": ["single-call", "text-before-call", "malformed", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. The deltas of the streaming test, concatenated. Text inside the section but outside a call is not content (vLLM asserts it does not leak)."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "a7520a48960356756fbafdc88eb9c4ec7ad513da01ee33c5fd96af6a96cb1654",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k2i-vllm-single-call-content-before expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-single-call-content-before
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got "I'll check. "

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-single-call-content-before --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 1 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
-  "content": "I'll check. ",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Beijing"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
   "content": "I'll check. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "Beijing"
-      },
-      "name": "get_weather"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L59-L66. Tags: single-call, text-before-call, x-id-encodes-name, x-kimi-k2.

Raw output

I'll check. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{"city": "Beijing"}<|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "I'll check. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Beijing"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-single-call-content-before", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L59-L66", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}], "raw_output": "I'll check. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{\"city\": \"Beijing\"}<|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [65447, 2598, 13, 220, 163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 220, 163598, 8264, 37666, 1289, 414, 4396, 39556, 16934, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "I'll check. ", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "Beijing"}}]}, "tags": ["single-call", "text-before-call", "x-id-encodes-name", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. vLLM's test helper puts a space between the call id and <|tool_call_argument_begin|>."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "09331a2e2df9a56a2f3acb61624999968c53eea45d0a0be1bb17cd57f265c61f",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k2i-vllm-three-tool-calls expected_match, stream_equals_nonstream, parallel_order

Checks that did not pass for kimi/k2i-vllm-three-tool-calls
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got 'Multiple tasks. '
`parallel_order`	`nonstream`	fail	expected 3 calls ['get_weather', 'get_news', 'send_email'], got 0 []
`parallel_order`	`token`	fail	expected 3 calls ['get_weather', 'get_news', 'send_email'], got 0 []

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@special fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, parallel_order@one fail, parallel_order@special fail, parallel_order@rand:1:8 fail, parallel_order@rand:2:8 fail, parallel_order@rand:3:8 fail, parallel_order@rand:4:8 fail, parallel_order@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-three-tool-calls --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 2 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,25 +1,6 @@
 {
-  "content": "Multiple tasks. ",
+  "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "New York"
-      },
-      "name": "get_weather"
-    },
-    {
-      "arguments": {
-        "topic": "technology"
-      },
-      "name": "get_news"
-    },
-    {
-      "arguments": {
-        "subject": "Daily Update",
-        "to": "user@example.com"
-      },
-      "name": "send_email"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,25 +1,6 @@
 {
   "content": "Multiple tasks. ",
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "city": "New York"
-      },
-      "name": "get_weather"
-    },
-    {
-      "arguments": {
-        "topic": "technology"
-      },
-      "name": "get_news"
-    },
-    {
-      "arguments": {
-        "subject": "Daily Update",
-        "to": "user@example.com"
-      },
-      "name": "send_email"
-    }
-  ]
+  "tool_calls": []
 }

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L78-L96. Tags: parallel-calls, text-before-call, x-global-idx, x-kimi-k2.

Raw output

Multiple tasks. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{"city": "New York"}<|tool_call_end|><|tool_call_begin|>functions.get_news:1 <|tool_call_argument_begin|>{"topic": "technology"}<|tool_call_end|><|tool_call_begin|>functions.send_email:2 <|tool_call_argument_begin|>{"to": "user@example.com", "subject": "Daily Update"}<|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "Multiple tasks. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "New York"
      },
      "name": "get_weather"
    },
    {
      "arguments": {
        "topic": "technology"
      },
      "name": "get_news"
    },
    {
      "arguments": {
        "subject": "Daily Update",
        "to": "user@example.com"
      },
      "name": "send_email"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-three-tool-calls", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L78-L96", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_news", "parameters": {"type": "object", "properties": {"topic": {"type": "string"}}}}}, {"type": "function", "function": {"name": "send_email", "parameters": {"type": "object", "properties": {"to": {"type": "string"}, "subject": {"type": "string"}}}}}], "raw_output": "Multiple tasks. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{\"city\": \"New York\"}<|tool_call_end|><|tool_call_begin|>functions.get_news:1 <|tool_call_argument_begin|>{\"topic\": \"technology\"}<|tool_call_end|><|tool_call_begin|>functions.send_email:2 <|tool_call_argument_begin|>{\"to\": \"user@example.com\", \"subject\": \"Daily Update\"}<|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [33251, 12757, 13, 220, 163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 220, 163598, 8264, 37666, 1289, 414, 5632, 7282, 16934, 163599, 163597, 41937, 1150, 126818, 25, 16, 220, 163598, 8264, 30547, 1289, 414, 65785, 16934, 163599, 163597, 41937, 14131, 28599, 25, 17, 220, 163598, 8264, 1753, 1289, 414, 2482, 31, 17480, 1304, 665, 414, 37407, 1289, 414, 62156, 10829, 16934, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Multiple tasks. ", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "New York"}}, {"name": "get_news", "arguments": {"topic": "technology"}}, {"name": "send_email", "arguments": {"to": "user@example.com", "subject": "Daily Update"}}]}, "tags": ["parallel-calls", "text-before-call", "x-global-idx", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {},
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|im_end|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "fixture_tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "format": "peg-native",
  "generation_prompt": "<|im_assistant|>assistant<|im_middle|>",
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|tool_calls_section_begin|>",
    "<|tool_calls_section_end|>",
    "<|tool_call_begin|>",
    "<|tool_call_argument_begin|>",
    "<|tool_call_end|>",
    "<think>",
    "</think>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": false,
      "reason": "llama.cpp models/templates has no copy named after moonshotai/Kimi-K2-Instruct-0905"
    }
  },
  "template_identical_to": null,
  "template_path": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "</think>"
  ],
  "thinking_start_tag": "<think>",
  "tokenizer": "moonshotai/Kimi-K2-Instruct-0905@ac6c49f04883bd0a0598b790693a72061c676629",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "cdb88deae96d8c3c8a8a04d39300e17cc8f0127c64a386415fffcf6133c8479d",
  "vocab_gguf": "moonshotai--Kimi-K2-Instruct-0905.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "4ee284a56c0855e2512f2a78ebd0c0ae2ad12a18b6268ab2512e973bd244e6d0"
}

fail kimi/k3-control-marker-text-in-value expected_match, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for kimi/k3-control-marker-text-in-value
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	reasoning_content: nonstream None, got 'Write the literal XTML marker text.'; tool_calls: nonstream [], got ['write_file']
`arguments_json`	`token`	fail	[0] write_file: arguments are not valid JSON text ('{"path":"x.txt","content":""': Expecting ',' delimiter: line 1 column 29 (char 28))
`arguments_schema`	`token`	fail	[0] write_file: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@special fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, arguments_json@special fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@special fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/k3-render.jsonl --id kimi/k3-control-marker-text-in-value --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 9 of fixtures/kimi/k3-render.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,13 +1,6 @@
 {
   "content": null,
-  "reasoning_content": "Write the literal XTML marker text.",
-  "tool_calls": [
-    {
-      "arguments": {
-        "content": "<|close|>argument<|sep|> is how args end",
-        "path": "x.txt"
-      },
-      "name": "write_file"
-    }
-  ]
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
+  "reasoning_content": null,
+  "tool_calls": []
 }

Strategies: special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,10 +1,10 @@
 {
   "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": "Write the literal XTML marker text.",
   "tool_calls": [
     {
       "arguments": {
-        "content": "<|close|>argument<|sep|> is how args end",
-        "path": "x.txt"
+        "<arguments_raw, not valid JSON>": "{\"path\":\"x.txt\",\"content\":\""
       },
       "name": "write_file"

Strategies: token

--- expected
+++ observed
@@ -1,10 +1,10 @@
 {
   "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": "Write the literal XTML marker text.",
   "tool_calls": [
     {
       "arguments": {
-        "content": "<|close|>argument<|sep|> is how args end",
-        "path": "x.txt"
+        "<arguments_raw, not valid JSON>": "{\"path\":\"x.txt\",\"content\":\"\""
       },
       "name": "write_file"

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py. Tags: single-call, marker-in-arguments, reasoning, x-control-token-vs-text, x-think-no-open-tag, x-kimi-k3.

Raw output

Write the literal XTML marker text.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="write_file" index="1"<|sep|><|open|>argument key="path" type="string"<|sep|>x.txt<|close|>argument<|sep|><|open|>argument key="content" type="string"<|sep|><|close|>argument<|sep|> is how args end<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the literal XTML marker text.",
  "tool_calls": [
    {
      "arguments": {
        "content": "<|close|>argument<|sep|> is how args end",
        "path": "x.txt"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-control-marker-text-in-value", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "license": "LicenseRef-kimi-k3", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "49ff03305fdc4be26867972788d36150b67f8a9e852e62bb7959d87482223676"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}, {"type": "function", "function": {"name": "save_answer", "description": "Store an answer under odd-looking keys.", "parameters": {"type": "object", "properties": {"q&a": {"type": "string"}, "say \"hi\"": {"type": "string"}}}}}], "raw_output": "Write the literal XTML marker text.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"write_file\" index=\"1\"<|sep|><|open|>argument key=\"path\" type=\"string\"<|sep|>x.txt<|close|>argument<|sep|><|open|>argument key=\"content\" type=\"string\"<|sep|><|close|>argument<|sep|> is how args end<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>", "output_token_ids": [9570, 276, 34459, 115921, 4920, 24894, 2913, 13, 163588, 39964, 163589, 163587, 12092, 163589, 163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 8124, 6101, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 4953, 1, 1798, 878, 2033, 1, 163589, 87, 8842, 163588, 47185, 163589, 163587, 47185, 2355, 878, 4204, 1, 1798, 878, 2033, 1, 163589, 27, 91, 10794, 91, 29, 47185, 27, 91, 40433, 91, 29, 387, 1632, 5924, 1565, 163588, 47185, 163589, 163588, 10257, 163589, 163588, 25385, 163589, 163588, 2778, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "thinking": true, "expected": {"content": null, "reasoning_content": "Write the literal XTML marker text.", "tool_calls": [{"name": "write_file", "arguments": {"path": "x.txt", "content": "<|close|>argument<|sep|> is how args end"}}]}, "tags": ["single-call", "marker-in-arguments", "reasoning", "x-control-token-vs-text", "x-think-no-open-tag", "x-kimi-k3"], "notes": "encoding_k3 encodes argument text with allow_special=False, so the value's <|close|>/<|sep|> are ORDINARY tokens while the real markers are control tokens. The text is identical; only output_token_ids tell them apart."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": true,
    "thinking": true
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|end_of_msg|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": null,
        "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
        "reasoning_content": null,
        "tool_calls": []
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "6511c730913452bec647054683f51493cc594f0cdccc95a78cc85d4a61952767",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-truncated-inside-argument expected_error, arguments_json, arguments_schema

Checks that did not pass for kimi/k3-truncated-inside-argument
Check	Strategy	Result	Detail
`expected_error`	`nonstream`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the first <argument> element.
`expected_error`	`token`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the first <argument> element.
`arguments_json`	`nonstream`	fail	[0] get_weather: arguments are not valid JSON text ('{"city":"': Unterminated string starting at: line 1 column 9 (char 8))
`arguments_json`	`token`	fail	[0] get_weather: arguments are not valid JSON text ('{"city":"': Unterminated string starting at: line 1 column 9 (char 8))
`arguments_schema`	`nonstream`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`token`	fail	[0] get_weather: arguments are not a JSON object; not validated

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/truncated.jsonl --id kimi/k3-truncated-inside-argument --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 2 of fixtures/kimi/truncated.jsonl.

Observed

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "<arguments_raw, not valid JSON>": "{\"city\":\""
      },
      "name": "get_weather"
    }
  ]
}

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py. Tags: truncated, x-kimi-k3.

Expected graceful failure: max_tokens hit inside the first <argument> element. (accept: no_tool_calls, content_passthrough, exception).

Raw output

<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="get_weather" index="1"<|sep|><|open|>argument key="city" type="string"<|sep|>

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-truncated-inside-argument", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "license": "LicenseRef-kimi-k3", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "49ff03305fdc4be26867972788d36150b67f8a9e852e62bb7959d87482223676"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}, {"type": "function", "function": {"name": "save_answer", "description": "Store an answer under odd-looking keys.", "parameters": {"type": "object", "properties": {"q&a": {"type": "string"}, "say \"hi\"": {"type": "string"}}}}}], "raw_output": "<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"get_weather\" index=\"1\"<|sep|><|open|>argument key=\"city\" type=\"string\"<|sep|>", "output_token_ids": [163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 618, 21055, 2800, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 37666, 1, 1798, 878, 2033, 1, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>", "thinking": false, "expected_error": {"reason": "max_tokens hit inside the first <argument> element.", "accept": ["no_tool_calls", "content_passthrough", "exception"]}, "tags": ["truncated", "x-kimi-k3"], "notes": "Token prefix of kimi/k3-thinking-disabled-call."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": false,
    "thinking": false
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": null,
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": "",
        "exception": null,
        "reasoning_content": null,
        "tool_calls": [
          {
            "arguments_raw": "{\"city\":\"",
            "name": "get_weather"
          }
        ]
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "6511c730913452bec647054683f51493cc594f0cdccc95a78cc85d4a61952767",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-typed-values expected_match, arguments_schema

Checks that did not pass for kimi/k3-typed-values
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	tool_calls[0].arguments: expected '{"title": "Standup", "duration_minutes": 15, "all_day": false, "reminder_minutes": null}', got '{"title":"Standup","duration_minutes":15,"all_day":false,"reminder_minutes":"null"}'
`expected_match`	`token`	fail	tool_calls[0].arguments: expected '{"title": "Standup", "duration_minutes": 15, "all_day": false, "reminder_minutes": null}', got '{"title":"Standup","duration_minutes":15,"all_day":false,"reminder_minutes":"null"}'
`arguments_schema`	`nonstream`	fail	[0] create_event: $.reminder_minutes: 'null' is not of type 'integer', 'null'
`arguments_schema`	`token`	fail	[0] create_event: $.reminder_minutes: 'null' is not of type 'integer', 'null'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/k3-render.jsonl --id kimi/k3-typed-values --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 6 of fixtures/kimi/k3-render.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -7,5 +7,5 @@
         "all_day": false,
         "duration_minutes": 15,
-        "reminder_minutes": null,
+        "reminder_minutes": "null",
         "title": "Standup"
       },

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py. Tags: single-call, numeric-arguments, reasoning, x-typed-args, x-think-no-open-tag, x-kimi-k3.

Raw output

Standup with no reminder.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="create_event" index="1"<|sep|><|open|>argument key="title" type="string"<|sep|>Standup<|close|>argument<|sep|><|open|>argument key="duration_minutes" type="number"<|sep|>15<|close|>argument<|sep|><|open|>argument key="all_day" type="boolean"<|sep|>false<|close|>argument<|sep|><|open|>argument key="reminder_minutes" type="null"<|sep|>null<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>

Expected parse

{
  "content": null,
  "reasoning_content": "Standup with no reminder.",
  "tool_calls": [
    {
      "arguments": {
        "all_day": false,
        "duration_minutes": 15,
        "reminder_minutes": null,
        "title": "Standup"
      },
      "name": "create_event"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-typed-values", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "license": "LicenseRef-kimi-k3", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "49ff03305fdc4be26867972788d36150b67f8a9e852e62bb7959d87482223676"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}, {"type": "function", "function": {"name": "save_answer", "description": "Store an answer under odd-looking keys.", "parameters": {"type": "object", "properties": {"q&a": {"type": "string"}, "say \"hi\"": {"type": "string"}}}}}], "raw_output": "Standup with no reminder.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"create_event\" index=\"1\"<|sep|><|open|>argument key=\"title\" type=\"string\"<|sep|>Standup<|close|>argument<|sep|><|open|>argument key=\"duration_minutes\" type=\"number\"<|sep|>15<|close|>argument<|sep|><|open|>argument key=\"all_day\" type=\"boolean\"<|sep|>false<|close|>argument<|sep|><|open|>argument key=\"reminder_minutes\" type=\"null\"<|sep|>null<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>", "output_token_ids": [51532, 771, 472, 1495, 35114, 13, 163588, 39964, 163589, 163587, 12092, 163589, 163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 7788, 12411, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 6621, 1, 1798, 878, 2033, 1, 163589, 51532, 771, 163588, 47185, 163589, 163587, 47185, 2355, 878, 35548, 13030, 4840, 1, 1798, 878, 10006, 1, 163589, 1308, 163588, 47185, 163589, 163587, 47185, 2355, 878, 639, 43167, 1, 1798, 878, 16849, 1, 163589, 8092, 163588, 47185, 163589, 163587, 47185, 2355, 878, 2636, 8195, 13030, 4840, 1, 1798, 878, 6562, 1, 163589, 6562, 163588, 47185, 163589, 163588, 10257, 163589, 163588, 25385, 163589, 163588, 2778, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "thinking": true, "expected": {"content": null, "reasoning_content": "Standup with no reminder.", "tool_calls": [{"name": "create_event", "arguments": {"title": "Standup", "duration_minutes": 15, "all_day": false, "reminder_minutes": null}}]}, "tags": ["single-call", "numeric-arguments", "reasoning", "x-typed-args", "x-think-no-open-tag", "x-kimi-k3"], "notes": "Kimi-K3 renders each argument with an XTML type attribute (string/number/boolean/null)."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": true,
    "thinking": true
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|end_of_msg|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": "",
        "exception": null,
        "reasoning_content": "Standup with no reminder.",
        "tool_calls": [
          {
            "arguments_raw": "{\"title\":\"Standup\",\"duration_minutes\":15,\"all_day\":false,\"reminder_minutes\":\"null\"}",
            "name": "create_event"
          }
        ]
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "6511c730913452bec647054683f51493cc594f0cdccc95a78cc85d4a61952767",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-vllm-call-thinking-disabled expected_match, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for kimi/k3-vllm-call-thinking-disabled
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	tool_calls: nonstream [], got ['calc']
`arguments_json`	`token`	fail	[0] calc: arguments are not valid JSON text ('{': Expecting property name enclosed in double quotes: line 1 column 2 (char 1))
`arguments_schema`	`token`	fail	[0] calc: arguments are not a JSON object; not validated

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k3-vllm-call-thinking-disabled --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 9 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,12 +1,6 @@
 {
   "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
-  "tool_calls": [
-    {
-      "arguments": {
-        "x": 1
-      },
-      "name": "calc"
-    }
-  ]
+  "tool_calls": []
 }

Strategies: special, token

--- expected
+++ observed
@@ -1,9 +1,10 @@
 {
   "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": null,
   "tool_calls": [
     {
       "arguments": {
-        "x": 1
+        "<arguments_raw, not valid JSON>": "{"
       },
       "name": "calc"

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_use/test_kimi_k3_tool_parser.py#L238-L261. Tags: single-call, numeric-arguments, x-typed-args, x-kimi-k3.

Raw output

<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="calc" index="1"<|sep|><|open|>argument key="x" type="number"<|sep|>1<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|>

Expected parse

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "x": 1
      },
      "name": "calc"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-vllm-call-thinking-disabled", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_use/test_kimi_k3_tool_parser.py#L238-L261", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "calc", "parameters": {"type": "object", "properties": {}}}}], "raw_output": "<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"calc\" index=\"1\"<|sep|><|open|>argument key=\"x\" type=\"number\"<|sep|>1<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|>", "output_token_ids": [163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 48556, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 87, 1, 1798, 878, 10006, 1, 163589, 16, 163588, 47185, 163589, 163588, 10257, 163589, 163588, 25385, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>", "thinking": false, "expected": {"content": null, "reasoning_content": null, "tool_calls": [{"name": "calc", "arguments": {"x": 1}}]}, "tags": ["single-call", "numeric-arguments", "x-typed-args", "x-kimi-k3"]}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": false,
    "thinking": false
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": "<|end_of_msg|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": null,
        "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
        "reasoning_content": null,
        "tool_calls": []
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "024f30b66716a0fe7806557bff6cb869b48560febc6e26a7d0d816dd1558f253",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-vllm-reasoning-prefix-consumed expected_match, no_leakage

Checks that did not pass for kimi/k3-vllm-reasoning-prefix-consumed
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected 'answer', got 'answer<\|end_of_msg\|>'
`expected_match`	`token`	fail	content: expected 'answer', got 'answer<\|end_of_msg\|>'
`no_leakage`	`nonstream`	fail	content contains '<\|end_of_msg\|>'
`no_leakage`	`token`	fail	content contains '<\|end_of_msg\|>'

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k3-vllm-reasoning-prefix-consumed --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 11 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,4 +1,4 @@
 {
-  "content": "answer",
+  "content": "answer<|end_of_msg|>",
   "reasoning_content": "step",
   "tool_calls": []

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/reasoning/test_kimi_k3_reasoning_parser.py#L70-L80. Tags: no-call, reasoning, x-think-no-open-tag, x-kimi-k3.

Raw output

step<|close|>think<|sep|><|open|>response<|sep|>answer

Expected parse

{
  "content": "answer",
  "reasoning_content": "step",
  "tool_calls": []
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-vllm-reasoning-prefix-consumed", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/reasoning/test_kimi_k3_reasoning_parser.py#L70-L80", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "calc", "parameters": {"type": "object", "properties": {}}}}], "raw_output": "step<|close|>think<|sep|><|open|>response<|sep|>answer", "output_token_ids": [16901, 163588, 39964, 163589, 163587, 12092, 163589, 26921], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "thinking": true, "expected": {"content": "answer", "reasoning_content": "step", "tool_calls": []}, "tags": ["no-call", "reasoning", "x-think-no-open-tag", "x-kimi-k3"], "notes": "The generation prompt opened the think channel; the response is not closed in this test string."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": true,
    "thinking": true
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|end_of_msg|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": "answer<|end_of_msg|>",
        "exception": null,
        "reasoning_content": "step",
        "tool_calls": []
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "024f30b66716a0fe7806557bff6cb869b48560febc6e26a7d0d816dd1558f253",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-vllm-truncated-tools stream_equals_nonstream

Checks that did not pass for kimi/k3-vllm-truncated-tools
Check	Strategy	Result	Detail
`stream_equals_nonstream`	`token`	fail	content: nonstream '<\|close\|>response<\|sep\|><\|open\|>tools<\|sep\|><\|open\|>call tool="calc" index="1"', got None

Synthetic strategies (not counted): stream_equals_nonstream@one fail, stream_equals_nonstream@special fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/imported.jsonl --id kimi/k3-vllm-truncated-tools --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 10 of fixtures/kimi/imported.jsonl.

Observed

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": []
}

Strategies: nonstream

{
  "content": "<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"calc\" index=\"1\"",
  "reasoning_content": null,
  "tool_calls": []
}

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_use/test_kimi_k3_tool_parser.py#L264-L283. Tags: truncated, x-kimi-k3.

Expected graceful failure: The tools element is cut off inside the first call's open tag. (accept: no_tool_calls).

Raw output

<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="calc" index="1"

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-vllm-truncated-tools", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_use/test_kimi_k3_tool_parser.py#L264-L283", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "calc", "parameters": {"type": "object", "properties": {}}}}], "raw_output": "<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"calc\" index=\"1\"", "output_token_ids": [163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 48556, 1, 4002, 878, 16, 1], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>", "thinking": false, "expected_error": {"reason": "The tools element is cut off inside the first call's open tag.", "accept": ["no_tool_calls"]}, "tags": ["truncated", "x-kimi-k3"], "notes": "vLLM asserts no reasoning, no content and no calls: the XTML must not leak into content."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": false,
    "thinking": false
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": false,
  "end_token": null,
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>response<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": "<|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"calc\" index=\"1\"",
        "exception": null,
        "reasoning_content": null,
        "tool_calls": []
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "024f30b66716a0fe7806557bff6cb869b48560febc6e26a7d0d816dd1558f253",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

fail kimi/k3-xtml-attr-escaping expected_match, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for kimi/k3-xtml-attr-escaping
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`expected_match`	`token`	fail	parser raised 'LlamaCppError: The model produced output that does not match the expected peg-native format'
`stream_equals_nonstream`	`token`	fail	reasoning_content: nonstream None, got 'Keys need escaping.'; tool_calls: nonstream [], got ['save_answer']
`arguments_json`	`token`	fail	[0] save_answer: arguments are not valid JSON text ('{': Expecting property name enclosed in double quotes: line 1 column 2 (char 1))
`arguments_schema`	`token`	fail	[0] save_answer: arguments are not a JSON object; not validated

Synthetic strategies (not counted): expected_match@one fail, expected_match@special fail, expected_match@rand:1:8 fail, expected_match@rand:2:8 fail, expected_match@rand:3:8 fail, expected_match@rand:4:8 fail, expected_match@rand:5:8 fail, stream_equals_nonstream@special fail, stream_equals_nonstream@rand:1:8 fail, stream_equals_nonstream@rand:2:8 fail, stream_equals_nonstream@rand:3:8 fail, stream_equals_nonstream@rand:4:8 fail, stream_equals_nonstream@rand:5:8 fail, arguments_json@special fail, arguments_json@rand:1:8 fail, arguments_json@rand:2:8 fail, arguments_json@rand:3:8 fail, arguments_json@rand:4:8 fail, arguments_json@rand:5:8 fail, arguments_schema@special fail, arguments_schema@rand:1:8 fail, arguments_schema@rand:2:8 fail, arguments_schema@rand:3:8 fail, arguments_schema@rand:4:8 fail, arguments_schema@rand:5:8 fail

Minimal repro

uv run canitoolcall run --engine llamacpp --fixtures fixtures/kimi/k3-render.jsonl --id kimi/k3-xtml-attr-escaping --strategy token --observed all

Set up the engine first with scripts/engines/llamacpp.sh; this run used llamacpp a25c9865. The fixture is line 10 of fixtures/kimi/k3-render.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one

--- expected
+++ observed
@@ -1,13 +1,6 @@
 {
   "content": null,
-  "reasoning_content": "Keys need escaping.",
-  "tool_calls": [
-    {
-      "arguments": {
-        "q&a": "fish & chips",
-        "say \"hi\"": "she said \"hi\""
-      },
-      "name": "save_answer"
-    }
-  ]
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
+  "reasoning_content": null,
+  "tool_calls": []
 }

Strategies: special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,10 +1,10 @@
 {
   "content": null,
+  "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
   "reasoning_content": "Keys need escaping.",
   "tool_calls": [
     {
       "arguments": {
-        "q&a": "fish & chips",
-        "say \"hi\"": "she said \"hi\""
+        "<arguments_raw, not valid JSON>": "{"
       },
       "name": "save_answer"

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py. Tags: single-call, reasoning, x-xtml-attr-escaping, x-think-no-open-tag, x-kimi-k3.

Raw output

Keys need escaping.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="save_answer" index="1"<|sep|><|open|>argument key="q&amp;a" type="string"<|sep|>fish & chips<|close|>argument<|sep|><|open|>argument key="say &quot;hi&quot;" type="string"<|sep|>she said "hi"<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>

Expected parse

{
  "content": null,
  "reasoning_content": "Keys need escaping.",
  "tool_calls": [
    {
      "arguments": {
        "q&a": "fish & chips",
        "say \"hi\"": "she said \"hi\""
      },
      "name": "save_answer"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-xtml-attr-escaping", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "license": "LicenseRef-kimi-k3", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "49ff03305fdc4be26867972788d36150b67f8a9e852e62bb7959d87482223676"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}, {"type": "function", "function": {"name": "save_answer", "description": "Store an answer under odd-looking keys.", "parameters": {"type": "object", "properties": {"q&a": {"type": "string"}, "say \"hi\"": {"type": "string"}}}}}], "raw_output": "Keys need escaping.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"save_answer\" index=\"1\"<|sep|><|open|>argument key=\"q&amp;a\" type=\"string\"<|sep|>fish & chips<|close|>argument<|sep|><|open|>argument key=\"say &quot;hi&quot;\" type=\"string\"<|sep|>she said \"hi\"<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>", "output_token_ids": [13115, 1362, 89625, 13, 163588, 39964, 163589, 163587, 12092, 163589, 163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 15450, 91494, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 80, 42055, 77360, 1, 1798, 878, 2033, 1, 163589, 25048, 841, 28510, 163588, 47185, 163589, 163587, 47185, 2355, 878, 69586, 841, 13847, 26, 9663, 39877, 26, 1, 1798, 878, 2033, 1, 163589, 47941, 2531, 414, 9663, 1, 163588, 47185, 163589, 163588, 10257, 163589, 163588, 25385, 163589, 163588, 2778, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "thinking": true, "expected": {"content": null, "reasoning_content": "Keys need escaping.", "tool_calls": [{"name": "save_answer", "arguments": {"q&a": "fish & chips", "say \"hi\"": "she said \"hi\""}}]}, "tags": ["single-call", "reasoning", "x-xtml-attr-escaping", "x-think-no-open-tag", "x-kimi-k3"], "notes": "Attribute values escape & and \" (&amp; &quot;); argument bodies stay raw."}

Parser configuration

{
  "additional_stops": [],
  "chat_parser": "common_chat_templates_apply -> common_chat_parse (PEG)",
  "chat_template_kwargs": {
    "enable_thinking": true,
    "thinking": true
  },
  "detokenized_matches_raw_output": true,
  "enable_thinking": true,
  "end_token": "<|end_of_msg|>",
  "engine_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "eog_positions": [],
  "family_handler": "specialized handlers 'Kimi K2 Thinking'/'Kimi K3', autoparser otherwise",
  "fixture_generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "fixture_tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "format": "peg-native",
  "generation_prompt": "<|open|>message role=\"assistant\"<|sep|><|open|>think<|sep|>",
  "model": "moonshotai/Kimi-K3",
  "parallel_tool_calls": true,
  "preserved_tokens": [
    "<|open|>",
    "<|close|>",
    "<|sep|>",
    "<|end_of_msg|>"
  ],
  "reasoning_format": "deepseek",
  "template_alternatives": {
    "llamacpp": {
      "available": true,
      "format": "peg-native",
      "nonstream": {
        "content": null,
        "exception": "LlamaCppError: The model produced output that does not match the expected peg-native format",
        "reasoning_content": null,
        "tool_calls": []
      },
      "path": "models/templates/Kimi-K3.jinja",
      "sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3"
    }
  },
  "template_identical_to": "models/templates/Kimi-K3.jinja",
  "template_path": "moonshotai--Kimi-K3.vocab.gguf: tokenizer.chat_template",
  "template_sha256": "4578103bf225f58d5b7ceb613ccc2ac8fe82b6707b7289211b3d4825e87806b3",
  "template_source": "gguf",
  "template_source_reason": null,
  "template_source_requested": "gguf",
  "template_supports_thinking": true,
  "template_variant": "default",
  "thinking_end_tags": [
    "<|close|>think<|sep|>"
  ],
  "thinking_start_tag": "<|open|>think<|sep|>",
  "tokenizer": "moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06",
  "tokenizer_mode": "gguf-vocab-only",
  "tools_sha256": "6511c730913452bec647054683f51493cc594f0cdccc95a78cc85d4a61952767",
  "vocab_gguf": "moonshotai--Kimi-K3.vocab.gguf",
  "vocab_gguf_converter_env": "llama.cpp requirements-convert_hf_to_gguf.txt",
  "vocab_gguf_converter_patches": null,
  "vocab_gguf_llama_cpp_commit": "a25c9865fe03c954c93fd755b5d79ae86ba99750",
  "vocab_gguf_sha256": "e165cb1b4ab44a4d40939631bc7a5b351b17560e97b62b8a1448b46c24ecdb72"
}

pass strict match on every realistic strategy
soft pass only whitespace differs (normalization soft-v1)
fail a check failed
error the harness failed, not the engine's parser
unsupported the engine has no parser for this family or model