Kimi (K2.x section tokens, K3 XTML) on sglang 0.5.20

fail 90% strict pass 44 pass · 5 fail

Run

Engine

sglang 0.5.20

Run

2026-09-26T05:19:05Z to 2026-09-26T05:22:38Z

Platform

linux-x86_64, Python 3.12.14, canitoolcall 0.1.0.dev0

Strategies

one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8 (normalization soft-v1)

Fixtures digest

a1e7b55367e195e8aa6e2ef746b2cc141b44f37caad0887ff8f30b104616f836 the fixture corpus used for this site matches the run

Engine details

{
  "deps": {
    "orjson": "3.12.0",
    "partial-json-parser": "0.2.1.1.post7",
    "pydantic": "2.13.5",
    "tokenizers": "0.22.2",
    "torch": "2.13.0",
    "transformers": "5.12.1",
    "xgrammar": "0.2.1"
  },
  "dist_info": "sglang-0.5.20.dist-info",
  "install": "cp312 manylinux wheel unpacked and added via .pth (scripts/engines/sglang.sh)",
  "pinned_version": "0.5.20",
  "reasoning_parsers": 31,
  "tokens_per_step": "many",
  "tool_call_parsers": 39
}

Results file

data/sglang-0.5.20.json

Checks

Per fixture, the worst result of each check over the non-streaming parse and every realistic chunking strategy.
Check	Pass	Fail	Strict pass rate
`expected_match`	42	4	91%
`expected_error`	2	1	67%
`stream_equals_nonstream`	45	4	92%
`split_invariance`	49	0	100%
`no_leakage`	49	0	100%
`arguments_json`	34	2	94%
`arguments_schema`	34	2	94%
`parallel_order`	4	0	100%

Fixtures needing attention

fail kimi/k26-marker-terminator-in-string expected_match, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for kimi/k26-marker-terminator-in-string
Check	Strategy	Result	Detail
`expected_match`	`one`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`special`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`token`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`rand:1:8`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`rand:2:8`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`rand:3:8`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`rand:4:8`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`expected_match`	`rand:5:8`	fail	content: expected None, got ' please"}'; tool_calls[0].arguments: expected '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`one`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`special`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`token`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`rand:1:8`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`rand:2:8`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`rand:3:8`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`rand:4:8`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`stream_equals_nonstream`	`rand:5:8`	fail	content: nonstream None, got ' please"}'; tool_calls[0].arguments: nonstream '{"path": "end.txt", "content": "stop at <\|tool_call_end\|> please"}', got '{"path": "end.txt", "content": "stop at '
`arguments_json`	`one`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`special`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`token`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`rand:1:8`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`rand:2:8`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`rand:3:8`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`rand:4:8`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_json`	`rand:5:8`	fail	[0] write_file: arguments are not valid JSON text ('{"path": "end.txt", "content": "stop at ': Unterminated string starting at: line 1 column 32 (char 31))
`arguments_schema`	`one`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`special`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`token`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`rand:1:8`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`rand:2:8`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`rand:3:8`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`rand:4:8`	fail	[0] write_file: arguments are not a JSON object; not validated
`arguments_schema`	`rand:5:8`	fail	[0] write_file: arguments are not a JSON object; not validated

Minimal repro

uv run canitoolcall run --engine sglang --fixtures fixtures/kimi/k26-render.jsonl --id kimi/k26-marker-terminator-in-string --strategy one --strategy rand:1:8 --strategy rand:2:8 --strategy rand:3:8 --strategy rand:4:8 --strategy rand:5:8 --strategy special --strategy token --observed all

Set up the engine first with scripts/engines/sglang.sh; this run used sglang 0.5.20. The fixture is line 11 of fixtures/kimi/k26-render.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,10 +1,9 @@
 {
-  "content": null,
+  "content": " please\"}",
   "reasoning_content": "Write the literal end marker into the file.",
   "tool_calls": [
     {
       "arguments": {
-        "content": "stop at <|tool_call_end|> please",
-        "path": "end.txt"
+        "<arguments_raw, not valid JSON>": "{\"path\": \"end.txt\", \"content\": \"stop at "
       },
       "name": "write_file"

Strategies: nonstream

Matches the expected parse.

{
  "content": null,
  "reasoning_content": "Write the literal end marker into the file.",
  "tool_calls": [
    {
      "arguments": {
        "content": "stop at <|tool_call_end|> please",
        "path": "end.txt"
      },
      "name": "write_file"
    }
  ]
}

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja. Tags: single-call, marker-in-arguments, reasoning, x-marker-terminator-in-string, x-think-no-open-tag, x-kimi-k2.

Raw output

Write the literal end marker into the file.</think><|tool_calls_section_begin|><|tool_call_begin|>functions.write_file:0<|tool_call_argument_begin|>{"path": "end.txt", "content": "stop at <|tool_call_end|> please"}<|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the literal end marker into the file.",
  "tool_calls": [
    {
      "arguments": {
        "content": "stop at <|tool_call_end|> please",
        "path": "end.txt"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k26-marker-terminator-in-string", "family": "kimi", "models": ["moonshotai/Kimi-K2.6"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "license": "LicenseRef-modified-mit", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}], "raw_output": "Write the literal end marker into the file.</think><|tool_calls_section_begin|><|tool_call_begin|>functions.write_file:0<|tool_call_argument_begin|>{\"path\": \"end.txt\", \"content\": \"stop at <|tool_call_end|> please\"}<|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [9570, 276, 34459, 1565, 24894, 1576, 276, 1650, 13, 163607, 163595, 163597, 41937, 9189, 6101, 25, 15, 163598, 8264, 4953, 1289, 414, 517, 8842, 665, 414, 4204, 1289, 414, 18434, 623, 220, 163599, 5462, 16934, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2.6", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "mode": "hf"}, "thinking": true, "expected": {"content": null, "reasoning_content": "Write the literal end marker into the file.", "tool_calls": [{"name": "write_file", "arguments": {"path": "end.txt", "content": "stop at <|tool_call_end|> please"}}]}, "tags": ["single-call", "marker-in-arguments", "reasoning", "x-marker-terminator-in-string", "x-think-no-open-tag", "x-kimi-k2"], "notes": "The JSON string contains <|tool_call_end|>, tokenized as the marker token by the template. Only a JSON-aware parser recovers the full argument; the call's real end is the second <|tool_call_end|>."}

Parser configuration

{
  "auto_detected": {
    "reasoning_parser": "kimi_k2",
    "tool_call_parser": "kimi_k2"
  },
  "chat_encoding_spec": null,
  "chat_template_kwargs": {
    "thinking": true
  },
  "chat_template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f",
  "detokenizer": "DetokenizerManager._decode_batch_token_id_output",
  "engine": "sglang",
  "hf_config": {
    "architectures": [
      "KimiK25ForConditionalGeneration"
    ],
    "model_type": "kimi_k25"
  },
  "hf_config_error": null,
  "model": "moonshotai/Kimi-K2.6",
  "no_stop_trim": false,
  "notes": [],
  "prompt_tail": {
    "ids": [
      163588,
      69702,
      163601,
      163606
    ],
    "source": "generation_prompt"
  },
  "reasoning_detector": "KimiK2Detector",
  "reasoning_effort": null,
  "reasoning_enabled": true,
  "reasoning_parser": "kimi_k2",
  "separate_reasoning": true,
  "skip_special_tokens": false,
  "spaces_between_special_tokens": true,
  "stop": {
    "appended": true,
    "finish_reason": "stop",
    "id": 163586,
    "kept_by_engine": false,
    "rule": "first stop token of the reference model",
    "token": "<|im_end|>"
  },
  "stream_reasoning": true,
  "template_force_reasoning": false,
  "template_reasoning_config": "ReasoningToggleConfig(toggle_param='thinking', default_enabled=True, special_case=None, effort_kwarg=None)",
  "thinking": true,
  "tokenizer": {
    "class": "TikTokenTokenizer",
    "loader": "sglang.srt.utils.hf_transformers_utils.get_tokenizer(revision=, tokenizer_revision=)",
    "repo": "moonshotai/Kimi-K2.6",
    "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb",
    "trust_remote_code": true
  },
  "tokenizer_mode": "hf",
  "tool_call_detector": "KimiK2Detector",
  "tool_call_parser": "kimi_k2",
  "tool_choice": "auto",
  "tools_offered": 6,
  "version": "0.5.20"
}

fail kimi/k26-truncated-inside-arguments expected_error, stream_equals_nonstream, arguments_json, arguments_schema

Checks that did not pass for kimi/k26-truncated-inside-arguments
Check	Strategy	Result	Detail
`expected_error`	`one`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`special`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`token`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`rand:1:8`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`rand:2:8`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`rand:3:8`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`rand:4:8`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`expected_error`	`rand:5:8`	fail	returned 1 tool call(s) ['get_weather'] for max_tokens hit inside the argument JSON; the call never closes.
`stream_equals_nonstream`	`one`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`special`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`token`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`rand:1:8`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`rand:2:8`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`rand:3:8`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`rand:4:8`	fail	tool_calls: nonstream [], got ['get_weather']
`stream_equals_nonstream`	`rand:5:8`	fail	tool_calls: nonstream [], got ['get_weather']
`arguments_json`	`one`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`special`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`token`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`rand:1:8`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`rand:2:8`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`rand:3:8`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`rand:4:8`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_json`	`rand:5:8`	fail	[0] get_weather: arguments are not valid JSON text ('{"city": "Madrid", "unit":': Expecting value: line 1 column 27 (char 26))
`arguments_schema`	`one`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`special`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`token`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`rand:1:8`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`rand:2:8`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`rand:3:8`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`rand:4:8`	fail	[0] get_weather: arguments are not a JSON object; not validated
`arguments_schema`	`rand:5:8`	fail	[0] get_weather: arguments are not a JSON object; not validated

Minimal repro

uv run canitoolcall run --engine sglang --fixtures fixtures/kimi/truncated.jsonl --id kimi/k26-truncated-inside-arguments --strategy one --strategy rand:1:8 --strategy rand:2:8 --strategy rand:3:8 --strategy rand:4:8 --strategy rand:5:8 --strategy special --strategy token --observed all

Set up the engine first with scripts/engines/sglang.sh; this run used sglang 0.5.20. The fixture is line 1 of fixtures/kimi/truncated.jsonl.

Observed

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "<arguments_raw, not valid JSON>": "{\"city\": \"Madrid\", \"unit\":"
      },
      "name": "get_weather"
    }
  ]
}

Strategies: nonstream

{
  "content": null,
  "reasoning_content": null,
  "tool_calls": []
}

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja. Tags: truncated, x-kimi-k2.

Expected graceful failure: max_tokens hit inside the argument JSON; the call never closes. (accept: no_tool_calls, content_passthrough, exception).

Raw output

<|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0<|tool_call_argument_begin|>{"city": "Madrid", "unit":

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k26-truncated-inside-arguments", "family": "kimi", "models": ["moonshotai/Kimi-K2.6"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K2.6/blob/7eb5002f6aadc958aed6a9177b7ed26bb94011bb/chat_template.jinja", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "license": "LicenseRef-modified-mit", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}], "raw_output": "<|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0<|tool_call_argument_begin|>{\"city\": \"Madrid\", \"unit\":", "output_token_ids": [163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 163598, 8264, 37666, 1289, 414, 44, 12708, 338, 665, 414, 8175, 1289], "tokenizer": {"repo": "moonshotai/Kimi-K2.6", "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb", "mode": "hf"}, "generation_prompt": "<|im_assistant|>assistant<|im_middle|><think></think>", "thinking": false, "expected_error": {"reason": "max_tokens hit inside the argument JSON; the call never closes.", "accept": ["no_tool_calls", "content_passthrough", "exception"]}, "tags": ["truncated", "x-kimi-k2"], "notes": "Token prefix of kimi/k26-thinking-disabled-call."}

Parser configuration

{
  "auto_detected": {
    "reasoning_parser": "kimi_k2",
    "tool_call_parser": "kimi_k2"
  },
  "chat_encoding_spec": null,
  "chat_template_kwargs": {
    "thinking": false
  },
  "chat_template_sha256": "8bf859698fd4781c0e1e1c63ce74422aab27e53ccc5f47116317c64cda06132f",
  "detokenizer": "DetokenizerManager._decode_batch_token_id_output",
  "engine": "sglang",
  "hf_config": {
    "architectures": [
      "KimiK25ForConditionalGeneration"
    ],
    "model_type": "kimi_k25"
  },
  "hf_config_error": null,
  "model": "moonshotai/Kimi-K2.6",
  "no_stop_trim": false,
  "notes": [],
  "prompt_tail": {
    "ids": [
      163588,
      69702,
      163601,
      163606,
      163607
    ],
    "source": "generation_prompt"
  },
  "reasoning_detector": "KimiK2Detector",
  "reasoning_effort": null,
  "reasoning_enabled": false,
  "reasoning_parser": "kimi_k2",
  "separate_reasoning": true,
  "skip_special_tokens": false,
  "spaces_between_special_tokens": true,
  "stop": {
    "appended": false,
    "finish_reason": "length",
    "id": null,
    "kept_by_engine": false,
    "rule": "truncated fixture: finish_reason length",
    "token": null
  },
  "stream_reasoning": true,
  "template_force_reasoning": false,
  "template_reasoning_config": "ReasoningToggleConfig(toggle_param='thinking', default_enabled=True, special_case=None, effort_kwarg=None)",
  "thinking": false,
  "tokenizer": {
    "class": "TikTokenTokenizer",
    "loader": "sglang.srt.utils.hf_transformers_utils.get_tokenizer(revision=, tokenizer_revision=)",
    "repo": "moonshotai/Kimi-K2.6",
    "revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb",
    "trust_remote_code": true
  },
  "tokenizer_mode": "hf",
  "tool_call_detector": "KimiK2Detector",
  "tool_call_parser": "kimi_k2",
  "tool_choice": "auto",
  "tools_offered": 6,
  "version": "0.5.20"
}

fail kimi/k2i-vllm-content-after-tool-section expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-content-after-tool-section
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	content: expected 'Before. After tools.', got 'Before. '
`stream_equals_nonstream`	`one`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`special`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`token`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`rand:1:8`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`rand:2:8`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`rand:3:8`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`rand:4:8`	fail	content: nonstream 'Before. ', got 'Before. After tools.'
`stream_equals_nonstream`	`rand:5:8`	fail	content: nonstream 'Before. ', got 'Before. After tools.'

Minimal repro

uv run canitoolcall run --engine sglang --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-content-after-tool-section --strategy one --strategy rand:1:8 --strategy rand:2:8 --strategy rand:3:8 --strategy rand:4:8 --strategy rand:5:8 --strategy special --strategy token --observed all

Set up the engine first with scripts/engines/sglang.sh; this run used sglang 0.5.20. The fixture is line 7 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream

--- expected
+++ observed
@@ -1,4 +1,4 @@
 {
-  "content": "Before.  After tools.",
+  "content": "Before. ",
   "reasoning_content": null,
   "tool_calls": [

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

Matches the expected parse.

{
  "content": "Before.  After tools.",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Tokyo"
      },
      "name": "get_weather"
    }
  ]
}

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L435-L458. Tags: single-call, text-before-call, text-after-call, x-policy, x-kimi-k2.

Raw output

Before. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{"city": "Tokyo"} <|tool_call_end|><|tool_calls_section_end|> After tools.

Expected parse

{
  "content": "Before.  After tools.",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "city": "Tokyo"
      },
      "name": "get_weather"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-content-after-tool-section", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L435-L458", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}], "raw_output": "Before. <|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0 <|tool_call_argument_begin|>{\"city\": \"Tokyo\"} <|tool_call_end|><|tool_calls_section_end|> After tools.", "output_token_ids": [13295, 13, 220, 163595, 163597, 41937, 1150, 21055, 2800, 25, 15, 220, 163598, 8264, 37666, 1289, 414, 39818, 18009, 16934, 220, 163599, 163596, 6671, 7697, 13], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Before.  After tools.", "reasoning_content": null, "tool_calls": [{"name": "get_weather", "arguments": {"city": "Tokyo"}}]}, "tags": ["single-call", "text-before-call", "text-after-call", "x-policy", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. The deltas of the streaming test, concatenated. vLLM's test asserts the trailing text is DROPPED; this fixture expects it kept, because text outside the tool-calls section is ordinary content and dropping it loses model output. Content is the text before and after the section, concatenated verbatim (spec/README.md, 'Content around tool calls'). Tagged x-policy: the expected value encodes that spec rule, which neither the Kimi format nor the cited test fixes, so the matrix can show it apart from format conformance."}

Parser configuration

{
  "auto_detected": {
    "reasoning_parser": "kimi_k2",
    "tool_call_parser": "kimi_k2"
  },
  "chat_encoding_spec": null,
  "chat_template_kwargs": null,
  "chat_template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "detokenizer": "DetokenizerManager._decode_batch_token_id_output",
  "engine": "sglang",
  "hf_config": {
    "architectures": [
      "DeepseekV3ForCausalLM"
    ],
    "model_type": "kimi_k2"
  },
  "hf_config_error": null,
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "no_stop_trim": false,
  "notes": [],
  "prompt_tail": {
    "ids": [
      163588,
      69702,
      163601
    ],
    "source": "generation_prompt"
  },
  "reasoning_detector": null,
  "reasoning_effort": null,
  "reasoning_enabled": false,
  "reasoning_parser": null,
  "separate_reasoning": true,
  "skip_special_tokens": false,
  "spaces_between_special_tokens": true,
  "stop": {
    "appended": true,
    "finish_reason": "stop",
    "id": 163586,
    "kept_by_engine": false,
    "rule": "first stop token of the reference model",
    "token": "<|im_end|>"
  },
  "stream_reasoning": true,
  "template_force_reasoning": false,
  "template_reasoning_config": null,
  "thinking": null,
  "tokenizer": {
    "class": "TikTokenTokenizer",
    "loader": "sglang.srt.utils.hf_transformers_utils.get_tokenizer(revision=, tokenizer_revision=)",
    "repo": "moonshotai/Kimi-K2-Instruct-0905",
    "revision": "ac6c49f04883bd0a0598b790693a72061c676629",
    "trust_remote_code": true
  },
  "tokenizer_mode": "hf",
  "tool_call_detector": "KimiK2Detector",
  "tool_call_parser": "kimi_k2",
  "tool_choice": "auto",
  "tools_offered": 1,
  "version": "0.5.20"
}

fail kimi/k2i-vllm-noise-between-markers expected_match, stream_equals_nonstream

Checks that did not pass for kimi/k2i-vllm-noise-between-markers
Check	Strategy	Result	Detail
`expected_match`	`one`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`special`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`token`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`rand:1:8`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`rand:2:8`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`rand:3:8`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`rand:4:8`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`expected_match`	`rand:5:8`	fail	content: expected 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`one`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`special`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`token`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`rand:1:8`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`rand:2:8`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`rand:3:8`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`rand:4:8`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '
`stream_equals_nonstream`	`rand:5:8`	fail	content: nonstream 'Reasoning. ', got 'Reasoning. spurious noise '

Minimal repro

uv run canitoolcall run --engine sglang --fixtures fixtures/kimi/imported.jsonl --id kimi/k2i-vllm-noise-between-markers --strategy one --strategy rand:1:8 --strategy rand:2:8 --strategy rand:3:8 --strategy rand:4:8 --strategy rand:5:8 --strategy special --strategy token --observed all

Set up the engine first with scripts/engines/sglang.sh; this run used sglang 0.5.20. The fixture is line 5 of fixtures/kimi/imported.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -1,4 +1,4 @@
 {
-  "content": "Reasoning. ",
+  "content": "Reasoning.  spurious noise ",
   "reasoning_content": null,
   "tool_calls": [

Strategies: nonstream

Matches the expected parse.

{
  "content": "Reasoning. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "k": "v"
      },
      "name": "test"
    }
  ]
}

Fixture

Provenance: engine_test, https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L370-L386. Tags: single-call, text-before-call, malformed, x-kimi-k2.

Raw output

Reasoning. <|tool_calls_section_begin|> spurious noise <|tool_call_begin|>functions.test:0 <|tool_call_argument_begin|>{"k": "v"} <|tool_call_end|><|tool_calls_section_end|>

Expected parse

{
  "content": "Reasoning. ",
  "reasoning_content": null,
  "tool_calls": [
    {
      "arguments": {
        "k": "v"
      },
      "name": "test"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k2i-vllm-noise-between-markers", "family": "kimi", "models": ["moonshotai/Kimi-K2-Instruct-0905"], "spec_version": "0.1", "provenance": {"kind": "engine_test", "source_url": "https://github.com/vllm-project/vllm/blob/ced6857afa0ea7b2e3f0846a62e1394e90f15607/tests/tool_parsers/test_kimi_k2_tool_parser.py#L370-L386", "revision": "ced6857afa0ea7b2e3f0846a62e1394e90f15607", "license": "Apache-2.0", "generator": "scripts/fixtures/kimi/imported.py", "attribution": "Copyright contributors to the vLLM project (Apache-2.0)"}, "tools": [{"type": "function", "function": {"name": "test", "parameters": {"type": "object", "properties": {"k": {"type": "string"}}}}}], "raw_output": "Reasoning. <|tool_calls_section_begin|> spurious noise <|tool_call_begin|>functions.test:0 <|tool_call_argument_begin|>{\"k\": \"v\"} <|tool_call_end|><|tool_calls_section_end|>", "output_token_ids": [24977, 288, 13, 220, 163595, 1284, 26163, 18226, 220, 163597, 41937, 9478, 25, 15, 220, 163598, 8264, 74, 1289, 414, 85, 16934, 220, 163599, 163596], "tokenizer": {"repo": "moonshotai/Kimi-K2-Instruct-0905", "revision": "ac6c49f04883bd0a0598b790693a72061c676629", "mode": "hf"}, "expected": {"content": "Reasoning. ", "reasoning_content": null, "tool_calls": [{"name": "test", "arguments": {"k": "v"}}]}, "tags": ["single-call", "text-before-call", "malformed", "x-kimi-k2"], "notes": "The vLLM test exercises the parser without a tools list; the tool schemas here are minimal stand-ins. The deltas of the streaming test, concatenated. Text inside the section but outside a call is not content (vLLM asserts it does not leak)."}

Parser configuration

{
  "auto_detected": {
    "reasoning_parser": "kimi_k2",
    "tool_call_parser": "kimi_k2"
  },
  "chat_encoding_spec": null,
  "chat_template_kwargs": null,
  "chat_template_sha256": "00938c355ca7195fd4f9c923e93af063846d4d5005e06999274f8db57b65bb0b",
  "detokenizer": "DetokenizerManager._decode_batch_token_id_output",
  "engine": "sglang",
  "hf_config": {
    "architectures": [
      "DeepseekV3ForCausalLM"
    ],
    "model_type": "kimi_k2"
  },
  "hf_config_error": null,
  "model": "moonshotai/Kimi-K2-Instruct-0905",
  "no_stop_trim": false,
  "notes": [],
  "prompt_tail": {
    "ids": [
      163588,
      69702,
      163601
    ],
    "source": "generation_prompt"
  },
  "reasoning_detector": null,
  "reasoning_effort": null,
  "reasoning_enabled": false,
  "reasoning_parser": null,
  "separate_reasoning": true,
  "skip_special_tokens": false,
  "spaces_between_special_tokens": true,
  "stop": {
    "appended": true,
    "finish_reason": "stop",
    "id": 163586,
    "kept_by_engine": false,
    "rule": "first stop token of the reference model",
    "token": "<|im_end|>"
  },
  "stream_reasoning": true,
  "template_force_reasoning": false,
  "template_reasoning_config": null,
  "thinking": null,
  "tokenizer": {
    "class": "TikTokenTokenizer",
    "loader": "sglang.srt.utils.hf_transformers_utils.get_tokenizer(revision=, tokenizer_revision=)",
    "repo": "moonshotai/Kimi-K2-Instruct-0905",
    "revision": "ac6c49f04883bd0a0598b790693a72061c676629",
    "trust_remote_code": true
  },
  "tokenizer_mode": "hf",
  "tool_call_detector": "KimiK2Detector",
  "tool_call_parser": "kimi_k2",
  "tool_choice": "auto",
  "tools_offered": 1,
  "version": "0.5.20"
}

fail kimi/k3-control-marker-text-in-value expected_match

Checks that did not pass for kimi/k3-control-marker-text-in-value
Check	Strategy	Result	Detail
`expected_match`	`nonstream`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`one`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`special`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`token`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`rand:1:8`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`rand:2:8`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`rand:3:8`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`rand:4:8`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'
`expected_match`	`rand:5:8`	fail	tool_calls[0].arguments: expected '{"path": "x.txt", "content": "<\|close\|>argument<\|sep\|> is how args end"}', got '{"path": "x.txt", "content": ""}'

Minimal repro

uv run canitoolcall run --engine sglang --fixtures fixtures/kimi/k3-render.jsonl --id kimi/k3-control-marker-text-in-value --strategy one --strategy rand:1:8 --strategy rand:2:8 --strategy rand:3:8 --strategy rand:4:8 --strategy rand:5:8 --strategy special --strategy token --observed all

Set up the engine first with scripts/engines/sglang.sh; this run used sglang 0.5.20. The fixture is line 9 of fixtures/kimi/k3-render.jsonl.

Observed vs expected

Identical parses are grouped. Empty strings are shown as null, as in strict comparison.

Strategies: nonstream, one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

--- expected
+++ observed
@@ -5,5 +5,5 @@
     {
       "arguments": {
-        "content": "<|close|>argument<|sep|> is how args end",
+        "content": "",
         "path": "x.txt"
       },

Fixture

Provenance: template_render, https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py. Tags: single-call, marker-in-arguments, reasoning, x-control-token-vs-text, x-think-no-open-tag, x-kimi-k3.

Raw output

Write the literal XTML marker text.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool="write_file" index="1"<|sep|><|open|>argument key="path" type="string"<|sep|>x.txt<|close|>argument<|sep|><|open|>argument key="content" type="string"<|sep|><|close|>argument<|sep|> is how args end<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>

Expected parse

{
  "content": null,
  "reasoning_content": "Write the literal XTML marker text.",
  "tool_calls": [
    {
      "arguments": {
        "content": "<|close|>argument<|sep|> is how args end",
        "path": "x.txt"
      },
      "name": "write_file"
    }
  ]
}

Fixture record (JSONL, ready to vendor into an engine's tests)

{"id": "kimi/k3-control-marker-text-in-value", "family": "kimi", "models": ["moonshotai/Kimi-K3"], "spec_version": "0.1", "provenance": {"kind": "template_render", "source_url": "https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/encoding_k3.py", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "license": "LicenseRef-kimi-k3", "generator": "scripts/fixtures/kimi/build.py", "template_sha256": "49ff03305fdc4be26867972788d36150b67f8a9e852e62bb7959d87482223676"}, "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}, {"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}, "filters": {"type": "object", "properties": {"site": {"type": "string"}, "max_results": {"type": "integer"}, "tags": {"type": "array", "items": {"type": "string"}}}}}, "required": ["query"]}}}, {"type": "function", "function": {"name": "get_time", "description": "Get the current UTC time.", "parameters": {"type": "object", "properties": {}}}}, {"type": "function", "function": {"name": "write_file", "description": "Write a text file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "create_event", "description": "Create a calendar event.", "parameters": {"type": "object", "properties": {"title": {"type": "string"}, "attendees": {"type": "array", "items": {"type": "string"}}, "duration_minutes": {"type": "integer"}, "all_day": {"type": "boolean"}, "reminder_minutes": {"type": ["integer", "null"]}, "location": {"type": "object"}}, "required": ["title"]}}}, {"type": "function", "function": {"name": "convert_units", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from_unit": {"type": "string"}, "to_unit": {"type": "string"}}, "required": ["value", "from_unit", "to_unit"]}}}, {"type": "function", "function": {"name": "save_answer", "description": "Store an answer under odd-looking keys.", "parameters": {"type": "object", "properties": {"q&a": {"type": "string"}, "say \"hi\"": {"type": "string"}}}}}], "raw_output": "Write the literal XTML marker text.<|close|>think<|sep|><|open|>response<|sep|><|close|>response<|sep|><|open|>tools<|sep|><|open|>call tool=\"write_file\" index=\"1\"<|sep|><|open|>argument key=\"path\" type=\"string\"<|sep|>x.txt<|close|>argument<|sep|><|open|>argument key=\"content\" type=\"string\"<|sep|><|close|>argument<|sep|> is how args end<|close|>argument<|sep|><|close|>call<|sep|><|close|>tools<|sep|><|close|>message<|sep|>", "output_token_ids": [9570, 276, 34459, 115921, 4920, 24894, 2913, 13, 163588, 39964, 163589, 163587, 12092, 163589, 163588, 12092, 163589, 163587, 25385, 163589, 163587, 10257, 4453, 878, 8124, 6101, 1, 4002, 878, 16, 1, 163589, 163587, 47185, 2355, 878, 4953, 1, 1798, 878, 2033, 1, 163589, 87, 8842, 163588, 47185, 163589, 163587, 47185, 2355, 878, 4204, 1, 1798, 878, 2033, 1, 163589, 27, 91, 10794, 91, 29, 47185, 27, 91, 40433, 91, 29, 387, 1632, 5924, 1565, 163588, 47185, 163589, 163588, 10257, 163589, 163588, 25385, 163589, 163588, 2778, 163589], "tokenizer": {"repo": "moonshotai/Kimi-K3", "revision": "f831ab66814297da540d832a5235f8e904f29d06", "mode": "hf"}, "thinking": true, "expected": {"content": null, "reasoning_content": "Write the literal XTML marker text.", "tool_calls": [{"name": "write_file", "arguments": {"path": "x.txt", "content": "<|close|>argument<|sep|> is how args end"}}]}, "tags": ["single-call", "marker-in-arguments", "reasoning", "x-control-token-vs-text", "x-think-no-open-tag", "x-kimi-k3"], "notes": "encoding_k3 encodes argument text with allow_special=False, so the value's <|close|>/<|sep|> are ORDINARY tokens while the real markers are control tokens. The text is identical; only output_token_ids tell them apart."}

Parser configuration

{
  "auto_detected": {
    "reasoning_parser": null,
    "tool_call_parser": null
  },
  "chat_encoding_spec": "kimi_k3",
  "chat_template_kwargs": {
    "thinking": true
  },
  "chat_template_sha256": null,
  "detokenizer": "DetokenizerManager._decode_batch_token_id_output",
  "engine": "sglang",
  "hf_config": {
    "architectures": [
      "KimiK3ForConditionalGeneration"
    ],
    "model_type": "kimi_k3"
  },
  "hf_config_error": null,
  "model": "moonshotai/Kimi-K3",
  "no_stop_trim": false,
  "notes": [],
  "prompt_tail": {
    "ids": [
      1,
      163589,
      163587,
      39964,
      163589
    ],
    "source": "generation_prompt"
  },
  "reasoning_detector": "KimiK3Detector",
  "reasoning_effort": null,
  "reasoning_enabled": true,
  "reasoning_parser": "kimi_k3",
  "separate_reasoning": true,
  "skip_special_tokens": false,
  "spaces_between_special_tokens": true,
  "stop": {
    "appended": true,
    "finish_reason": "stop",
    "id": 163586,
    "kept_by_engine": false,
    "rule": "first stop token of the reference model",
    "token": "<|end_of_msg|>"
  },
  "stream_reasoning": true,
  "template_force_reasoning": false,
  "template_reasoning_config": null,
  "thinking": true,
  "tokenizer": {
    "class": "TikTokenTokenizer",
    "loader": "sglang.srt.utils.hf_transformers_utils.get_tokenizer(revision=, tokenizer_revision=)",
    "repo": "moonshotai/Kimi-K3",
    "revision": "f831ab66814297da540d832a5235f8e904f29d06",
    "trust_remote_code": true
  },
  "tokenizer_mode": "hf",
  "tool_call_detector": "KimiK3Detector",
  "tool_call_parser": "kimi_k3",
  "tool_choice": "auto",
  "tools_offered": 7,
  "version": "0.5.20"
}

pass strict match on every realistic strategy
soft pass only whitespace differs (normalization soft-v1)
fail a check failed
error the harness failed, not the engine's parser
unsupported the engine has no parser for this family or model