Truncated tool call (finish_reason: length) is recorded as an empty finding #21

Summary

When a model response ends with finish_reason: "length" partway through a tool call, the arguments fail to parse, become {}, and the hunter still records them through report_finding. The finding the model was writing is lost, and an empty "Unknown" finding goes to the validator in its place.

What happened

Headless openhack --scan (0.2.4) on a small Flask app with three planted bugs, using a BYOK OpenAI-compatible provider and a reasoning model (glm-5.3). One hunter turn emitted three parallel report_finding calls: command injection, SQL injection and path traversal. The response stopped at max_tokens=8192, cutting off the third call 1,794 characters in, inside the exploit script in its description.

Failed to parse tool call arguments: {"category": "Path Traversal", "severity": "high", "file_path": "app.py", "line_number": 20, "description": "Unauthenticated path traversal / arbitrary file read on the /file endpoint. ...

The event log shows tool_arguments_parse_failed (Unterminated string), then report_finding with tool_input: {} → {"status": "recorded", "finding_id": 3}. The validator correctly rejected finding #3 as empty ("category Unknown, no file path…"), and the scan confirmed 2 of the 3 bugs it had found. The validator's own verdict call on the empty finding was also truncated at the same cap.

Over the whole scan, 11 of 40 model calls ended with finish_reason: "length".

Where

• openhack/agents/llm.py (L1051-L1079): on JSONDecodeError, args = {}, and the ToolCall is returned with parse_error set.
• openhack/agents/hunter.py (L404-L405): report_finding is handled with tool_call.arguments and parse_error is never checked. parse_error is put into trace metadata in several agents (base.py, validator.py, the verifiers) but doesn't appear to gate execution anywhere.
• max_tokens=8192 is fixed across the agents (coordinator.py:76, headless_scan.py:166, …), so a reasoning model that writes long finding descriptions reaches it often.

Possible fixes

• When parse_error is set, don't execute the call. Return a tool result telling the model its arguments were truncated and to resend them, perhaps one finding per call.
• Treat finish_reason == "length" with an open tool call as retryable, not complete.
• Optionally, make max_tokens configurable per role, like the *_model_id settings.

AI-assisted: found and written up with Claude Code during a scan run; the event log is available on request.