DeepSeek (V3/R1, V3.1, V3.2 DSML, V4 DSML, V4.1 DSML) on ollama 7af39318

pass 100% strict pass 12 pass · 73 unsupported

Run

Engine

ollama 7af39318 (7af393188defd52d370464de0d2064649cab9b41)

Run

2026-09-26T05:36:43Z to 2026-09-26T05:36:50Z

Platform

linux-x86_64, Python 3.12.3, canitoolcall 0.1.0.dev0

Strategies

one, special, token, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8 (normalization soft-v1); not counted for this engine: one, special, rand:1:8, rand:2:8, rand:3:8, rand:4:8, rand:5:8

Fixtures digest

a1e7b55367e195e8aa6e2ef746b2cc141b44f37caad0887ff8f30b104616f836 the fixture corpus used for this site matches the run

Engine details

{
  "go_version": "go1.26.0",
  "llama_cpp_build": "b11081",
  "llama_cpp_commit": "161755f",
  "pinned_version": "7af393188defd52d370464de0d2064649cab9b41",
  "runner": "llama-server token rendering (preserved_tokens) + model/parsers via server/routes.go flow",
  "tokens_per_step": "one"
}

Results file

data/ollama-7af39318.json

Checks

Per fixture, the worst result of each check over the non-streaming parse and every realistic chunking strategy.
Check	Pass	Strict pass rate
`expected_match`	11	100%
`expected_error`	1	100%
`stream_equals_nonstream`	12	100%
`no_leakage`	12	100%
`arguments_json`	10	100%
`arguments_schema`	10	100%
`parallel_order`	1	100%

Fixtures needing attention

Every supported fixture passed strictly on every realistic strategy.

Unsupported fixtures

The adapter declined these fixtures because this engine version has no parser for the family or model.

deepseek/sglang-v32-text-before-empty-invoke: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/sglang-v4-self-closing-invoke: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-empty-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-marker-in-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-nested-json: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-numeric-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-parallel-two-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-text-before-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-truncated-mid-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v3-unicode-emoji: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-empty-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-marker-in-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-numeric-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-parallel-two-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-reasoning-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-reasoning-parallel-text-before: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-reasoning-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-string-false-json: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-text-before-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-truncated-in-reasoning: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-truncated-mid-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-truncated-second-parallel-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-unescaped-string-value: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v32-unicode-emoji: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-empty-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-marker-in-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-multi-turn-after-tool-result: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-numeric-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-parallel-two-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-reasoning-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-reasoning-parallel-text-before: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-reasoning-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-string-false-json: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-text-before-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-truncated-in-reasoning: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-truncated-mid-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-truncated-second-parallel-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-unescaped-string-value: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v4-unicode-emoji: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-empty-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-long-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-marker-in-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-multi-turn-after-tool-result: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-numeric-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-parallel-two-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-reasoning-no-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-reasoning-parallel-text-before: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-reasoning-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-string-false-json: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-text-before-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-truncated-in-reasoning: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-truncated-mid-arguments: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-truncated-second-parallel-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-unescaped-string-value: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-unicode-emoji: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/v41-whitespace-string-value: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-escaped-strings: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-malformed-missing-brace: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-malformed-missing-call-tokens: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-no-tool-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-parallel-no-newline: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-single-call: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v3-various-data-types: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v41-reasoning-text-before-parallel-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered
deepseek/vllm-v41-text-before-parallel-calls: no built-in Ollama parser for this model at the pin; Ollama serves it through a Go template with the legacy tools parser or through llama-server's native chat (see the llamacpp engine); legacy template parser not covered

pass strict match on every realistic strategy
soft pass only whitespace differs (normalization soft-v1)
fail a check failed
error the harness failed, not the engine's parser
unsupported the engine has no parser for this family or model